Horton

Re: [POC] hash partitioning

Amul Sul <sulamul@gmail.com>

View thread View on postgr.es
From: amul sul <sulamul@gmail.com>
To: Andres Freund <andres@anarazel.de>
Cc: Robert Haas <robertmhaas@gmail.com>, Ashutosh Bapat <ashutosh.bapat@enterprisedb.com>, Jesper Pedersen <jesper.pedersen@redhat.com>, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>, Thom Brown <thom@linux.com>, Dilip Kumar <dilipbalaut@gmail.com>, Yugo Nagata <nagata@sraoss.co.jp>, David Steele <david@pgmasters.net>, Greg Stark <stark@mit.edu>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2017-10-24T11:35:26Z
Lists: pgsql-hackers 
On Tue, Oct 24, 2017 at 5:00 PM, Andres Freund <andres@anarazel.de> wrote:
> On 2017-10-24 12:43:12 +0530, amul sul wrote:
>> I tried to get suggested SMHasher[1] test result for the hash_combine
>> for 32-bit and 64-bit version.
>>
>> SMHasher works on hash keys of the form {0}, {0,1}, {0,1,2}... up to
>> N=255, using 256-N as the seed, for the hash_combine testing we
>> needed two hash value to be combined, for that, I've generated 64
>> and 128-bit hash using cityhash functions[2] for the given smhasher
>> key then split in two part to test 32-bit and 64-bit hash_combine
>> function respectively.   Attached patch for SMHasher code changes &
>> output of 32-bit and 64-bit hash_combine testing. Note that I have
>> skipped speed test this test which is irrelevant here.
>>
>> By referring other hash function results [3], we can see that hash_combine
>> test results are not bad either.
>>
>> Do let me know if current testing is not good enough or if you want me to do
>> more testing, thanks.
>
> This looks very good! Both the tests you did, and the results for
> hash_combineXX. I therefore think we can go ahead with that formulation
> of hash_combine64?
>

Thanks, Andres. Yes we can, I've added your suggested hash_combine64 in
the latest patch[1].

Regards,
Amul

1] https://postgr.es/m/CAAJ_b97R2rJinGPAVmZZzpNV%3D-5BgYFxDfY9HYdM1bCYJFGmQw%40mail.gmail.com

Commits

  1. Add hash partitioning.

  2. Add sanity check for pg_proc.provariadic

  3. Add hash_combine64.

  4. Make RelationGetPartitionDispatchInfo expand depth-first.

  5. Introduce 64-bit hash functions with a 64-bit seed.

  6. pg_dump: Add a --load-via-partition-root option.