Horton

Re: POC: Sharing record typmods between backends

Dilip Kumar <dilipbalaut@gmail.com>

View thread View on postgr.es
From: Dilip Kumar <dilipbalaut@gmail.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Thomas Munro <thomas.munro@enterprisedb.com>, Pg Hackers <pgsql-hackers@postgresql.org>
Date: 2017-05-31T15:16:34Z
Lists: pgsql-hackers 
On Wed, May 31, 2017 at 10:57 AM, Robert Haas <robertmhaas@gmail.com> wrote:
>> Simplehash provides an option to provide your own allocator function
>> to it. So in the allocator function, you can allocate memory from DSA.
>> After it reaches some threshold it expands the size (double) and it
>> will again call the allocator function to allocate the bigger memory.
>> You can refer pagetable_allocate in tidbitmap.c.
>
> That only allows the pagetable to be shared, not the hash table itself.

I agree with you. But, if I understand the use case correctly we need
to store the TupleDesc for the RECORD in shared hash so that it can be
shared across multiple processes.  I think this can be achieved with
the simplehash as well.

For getting this done, we need some fixed shared memory for holding
static members of SH_TYPE and the process which creates the simplehash
will be responsible for copying these static members to the shared
location so that other processes can access the SH_TYPE.  And, the
dynamic part (the actual hash entries) can be allocated using DSA by
registering SH_ALLOCATE function.

-- 
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com

Commits

  1. Remove TupleDesc remapping logic from tqueue.c.

  2. Add support for coordinating record typmods among parallel workers.

  3. Improve division of labor between execParallel.c and nodeGather[Merge].c.

  4. Add minimal regression test for blessed record type transfer.

  5. Consolidate the function pointer types used by dshash.c.

  6. Fix unlikely shared memory leak after failure in dshash_create().

  7. Refactor typcache.c's record typmod hash table.

  8. Add a hash_combine function for mixing hash values.

  9. Backpatch introduction of TupleDescAttr(tupdesc, i).

  10. Partially flatten struct tupleDesc so that it can be used in DSM.

  11. Change tupledesc->attrs[n] to TupleDescAttr(tupledesc, n).