Re: [POC] hash partitioning

Amul Sul <sulamul@gmail.com>

From: amul sul <sulamul@gmail.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Ashutosh Bapat <ashutosh.bapat@enterprisedb.com>, Jesper Pedersen <jesper.pedersen@redhat.com>, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>, Thom Brown <thom@linux.com>, Dilip Kumar <dilipbalaut@gmail.com>, Yugo Nagata <nagata@sraoss.co.jp>, David Steele <david@pgmasters.net>, Greg Stark <stark@mit.edu>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2017-10-31T04:47:33Z
Lists: pgsql-hackers
On Tue, Oct 31, 2017 at 9:54 AM, Robert Haas <robertmhaas@gmail.com> wrote:
> On Mon, Oct 30, 2017 at 5:52 PM, amul sul <sulamul@gmail.com> wrote:
>> Actually, int4[] is also inappropriate type as we have started using a 64bit
>> hash function.  We need something int8[] which is not available, so that I
>> have used ANYARRAYOID in the attached patch(0004).
>
> I don't know why you think int8[] is not available.
>
> rhaas=# select 'int8[]'::regtype;
>  regtype
> ----------
>  bigint[]
> (1 row)
>

I missed _int8, was searching for INT8ARRAYOID in pg_type.h, my bad.

>>> I wrote the following query
>>> to detect problems of this type, and I think we might want to just go
>>> ahead and add this to the regression test suite, verifying that it
>>> returns no rows:
>>>
>>> select oid::regprocedure, provariadic::regtype, proargtypes::regtype[]
>>> from pg_proc where provariadic != 0
>>> and case proargtypes[array_length(proargtypes, 1)-1]
>>>     when 2276 then 2276 -- any -> any
>>>     when 2277 then 2283 -- anyarray -> anyelement
>>>     else (select t.oid from pg_type t where t.typarray =
>>> proargtypes[array_length(proargtypes, 1)-1]) end
>>>     != provariadic;
>>>
>>
>> Added in 0001 patch.
>
> Committed.
>

Thanks !

>> One advantage of current implementation is that we can see which hash
>> function are used for the each partitioning column and also we don't need to
>> worry about user specified opclass and different input types.
>>
>> Something similar I've tried in my initial patch version[1], but I have missed
>> user specified opclass handling for each partitioning column.  Do you want me
>> to handle opclass using RelabelType node? I am afraid that, that would make
>> the \d+ output more horrible than the current one if non-default opclass used.
>
> Maybe we should just pass the OID of the partition (or both the
> partition and the parent, so we can get the lock ordering right?)
> instead.
>
Okay, will try this.


Regards,
Amul


Commits

  1. Add hash partitioning.

  2. Add sanity check for pg_proc.provariadic

  3. Add hash_combine64.

  4. Make RelationGetPartitionDispatchInfo expand depth-first.

  5. Introduce 64-bit hash functions with a 64-bit seed.

  6. pg_dump: Add a --load-via-partition-root option.