Horton

Re: [POC] hash partitioning

Andres Freund <andres@anarazel.de>

View thread View on postgr.es
From: Andres Freund <andres@anarazel.de>
To: Robert Haas <robertmhaas@gmail.com>
Cc: amul sul <sulamul@gmail.com>, Ashutosh Bapat <ashutosh.bapat@enterprisedb.com>, Jesper Pedersen <jesper.pedersen@redhat.com>, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>, Thom Brown <thom@linux.com>, Dilip Kumar <dilipbalaut@gmail.com>, Yugo Nagata <nagata@sraoss.co.jp>, David Steele <david@pgmasters.net>, Greg Stark <stark@mit.edu>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2017-10-12T21:30:29Z
Lists: pgsql-hackers 
On 2017-10-12 17:27:52 -0400, Robert Haas wrote:
> On Thu, Oct 12, 2017 at 4:20 PM, Andres Freund <andres@anarazel.de> wrote:
> >> In other words, it's not utterly fixed in stone --- we invented
> >> --load-via-partition-root primarily to cope with circumstances that
> >> could change hash values --- but we sure don't want to be changing it
> >> with any regularity, or for a less-than-excellent reason.
> >
> > Yea, that's what I expected. It'd probably good for somebody to run
> > smhasher or such on the output of the combine function (or even better,
> > on both the 32 and 64 bit variants) in that case.
> 
> Not sure how that test suite works exactly, but presumably the
> characteristics in practice will depend the behavior of the hash
> functions used as input the combine function - so the behavior could
> be good for an (int, int) key but bad for a (text, date) key, or
> whatever.

I don't think that's true, unless you have really bad hash functions on
the the component hashes. A hash combine function can't really do
anything about badly hashed input, what you want is that it doesn't
*reduce* the quality of the hash by combining.

Greetings,

Andres Freund

Commits

  1. Add hash partitioning.

  2. Add sanity check for pg_proc.provariadic

  3. Add hash_combine64.

  4. Make RelationGetPartitionDispatchInfo expand depth-first.

  5. Introduce 64-bit hash functions with a 64-bit seed.

  6. pg_dump: Add a --load-via-partition-root option.