Horton

Re: POC: Sharing record typmods between backends

Thomas Munro <thomas.munro@enterprisedb.com>

View thread View on postgr.es
From: Thomas Munro <thomas.munro@enterprisedb.com>
To: Pg Hackers <pgsql-hackers@postgresql.org>
Date: 2017-05-30T05:09:22Z
Lists: pgsql-hackers

Attachments

On Fri, Apr 7, 2017 at 5:21 PM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:
> * It would be nice for the SharedRecordTypeRegistry to be able to
> survive longer than a single parallel query, perhaps in a per-session
> DSM segment.  Perhaps eventually we will want to consider a
> query-scoped area, a transaction-scoped area and a session-scoped
> area?  I didn't investigate that for this POC.

This seems like the right way to go.  I think there should be one
extra patch in this patch stack, to create a per-session DSA area (and
perhaps a "SharedSessionState" struct?) that worker backends can
attach to.  It could be created when you first run a parallel query,
and then reused for all parallel queries for the rest of your session.
So, after you've run one parallel query, all future record typmod
registrations would get pushed (write-through style) into shmem, for
use by other backends that you might start in future parallel queries.
That will avoid having to copy the leader's registered record typmods
into shmem for every query going forward (the behaviour of the current
POC patch).

> * Perhaps simplehash + an LWLock would be better than dht, but I
> haven't looked into that.  Can it be convinced to work in DSA memory
> and to grow on demand?

Any views on this?

> 1.  Apply dht-v3.patch[3].
> 2.  Apply shared-record-typmod-registry-v1.patch.
> 3.  Apply rip-out-tqueue-remapping-v1.patch.

Here's a rebased version of the second patch (the other two still
apply).  It's still POC code only and still uses a
per-parallel-context DSA area for space, not the per-session one I am
now proposing we develop, if people are in favour of the approach.

In case it wasn't clear from my earlier description, a nice side
effect of using a shared typmod registry is that you can delete 85% of
tqueue.c (see patch #3), so if you don't count the hash table
implementation we come out about even in terms of lines of code.

-- 
Thomas Munro
http://www.enterprisedb.com

Commits

  1. Remove TupleDesc remapping logic from tqueue.c.

  2. Add support for coordinating record typmods among parallel workers.

  3. Improve division of labor between execParallel.c and nodeGather[Merge].c.

  4. Add minimal regression test for blessed record type transfer.

  5. Consolidate the function pointer types used by dshash.c.

  6. Fix unlikely shared memory leak after failure in dshash_create().

  7. Refactor typcache.c's record typmod hash table.

  8. Add a hash_combine function for mixing hash values.

  9. Backpatch introduction of TupleDescAttr(tupdesc, i).

  10. Partially flatten struct tupleDesc so that it can be used in DSM.

  11. Change tupledesc->attrs[n] to TupleDescAttr(tupledesc, n).