Horton

Re: POC: Sharing record typmods between backends

Andres Freund <andres@anarazel.de>

View thread View on postgr.es
From: Andres Freund <andres@anarazel.de>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Thomas Munro <thomas.munro@enterprisedb.com>, Dilip Kumar <dilipbalaut@gmail.com>, Pg Hackers <pgsql-hackers@postgresql.org>
Date: 2017-08-13T03:30:29Z
Lists: pgsql-hackers 
On 2017-08-12 22:52:57 -0400, Robert Haas wrote:
> On Fri, Aug 11, 2017 at 9:55 PM, Andres Freund <andres@anarazel.de> wrote:
> > Well, most of the potential usecases for dsmhash I've heard about so
> > far, don't actually benefit much from incremental growth. In nearly all
> > the implementations I've seen incremental move ends up requiring more
> > total cycles than doing it at once, and for parallelism type usecases
> > the stall isn't really an issue.  So yes, I think this is something
> > worth considering.   If we were to actually use DHT for shared caches or
> > such, this'd be different, but that seems darned far off.
> 
> I think it'd be pretty interesting to look at replacing parts of the
> stats collector machinery with something DHT-based.

That seems to involve a lot more than this though, given that currently
the stats collector data doesn't entirely have to be in memory. I've
seen sites with a lot of databases with quite some per-database stats
data. Don't think we can just require that to be in memory :(

- Andres

Commits

  1. Remove TupleDesc remapping logic from tqueue.c.

  2. Add support for coordinating record typmods among parallel workers.

  3. Improve division of labor between execParallel.c and nodeGather[Merge].c.

  4. Add minimal regression test for blessed record type transfer.

  5. Consolidate the function pointer types used by dshash.c.

  6. Fix unlikely shared memory leak after failure in dshash_create().

  7. Refactor typcache.c's record typmod hash table.

  8. Add a hash_combine function for mixing hash values.

  9. Backpatch introduction of TupleDescAttr(tupdesc, i).

  10. Partially flatten struct tupleDesc so that it can be used in DSM.

  11. Change tupledesc->attrs[n] to TupleDescAttr(tupledesc, n).