Horton

Re: [POC] hash partitioning

Andres Freund <andres@anarazel.de>

View thread View on postgr.es
From: Andres Freund <andres@anarazel.de>
To: amul sul <sulamul@gmail.com>
Cc: Robert Haas <robertmhaas@gmail.com>, Ashutosh Bapat <ashutosh.bapat@enterprisedb.com>, Jesper Pedersen <jesper.pedersen@redhat.com>, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>, Thom Brown <thom@linux.com>, Dilip Kumar <dilipbalaut@gmail.com>, Yugo Nagata <nagata@sraoss.co.jp>, David Steele <david@pgmasters.net>, Greg Stark <stark@mit.edu>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2017-10-24T11:30:04Z
Lists: pgsql-hackers 
On 2017-10-24 12:43:12 +0530, amul sul wrote:
> I tried to get suggested SMHasher[1] test result for the hash_combine
> for 32-bit and 64-bit version.
> 
> SMHasher works on hash keys of the form {0}, {0,1}, {0,1,2}... up to
> N=255, using 256-N as the seed, for the hash_combine testing we
> needed two hash value to be combined, for that, I've generated 64
> and 128-bit hash using cityhash functions[2] for the given smhasher
> key then split in two part to test 32-bit and 64-bit hash_combine
> function respectively.   Attached patch for SMHasher code changes &
> output of 32-bit and 64-bit hash_combine testing. Note that I have
> skipped speed test this test which is irrelevant here.
> 
> By referring other hash function results [3], we can see that hash_combine
> test results are not bad either.
> 
> Do let me know if current testing is not good enough or if you want me to do
> more testing, thanks.

This looks very good! Both the tests you did, and the results for
hash_combineXX. I therefore think we can go ahead with that formulation
of hash_combine64?

Greetings,

Andres Freund

Commits

  1. Add hash partitioning.

  2. Add sanity check for pg_proc.provariadic

  3. Add hash_combine64.

  4. Make RelationGetPartitionDispatchInfo expand depth-first.

  5. Introduce 64-bit hash functions with a 64-bit seed.

  6. pg_dump: Add a --load-via-partition-root option.