Re: postgres_fdw batching vs. (re)creating the tuple slots

Andres Freund <andres@anarazel.de>

From: Andres Freund <andres@anarazel.de>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Tomas Vondra <tomas.vondra@enterprisedb.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2021-05-30T21:26:16Z
Lists: pgsql-hackers
Hi,

On 2021-05-30 17:10:59 -0400, Tom Lane wrote:
> But it does seem like the hashing scheme somebody added to resowners
> is a bit too simplistic.  It ought to be able to cope with lots of
> refs to the same object, or at least not be extra-awful for that case.

It's not really the hashing that's the problem, right? The array
representation would have nearly the same problem, I think?

It doesn't seem trivial to improve it without making resowner.c's
representation a good bit more complicated. Right now there's no space
to store a 'per resowner & tupdesc refcount'. We can't even just make
the tuple desc reference a separate allocation (of (tupdesc, refcount)),
because ResourceArrayRemove() relies on testing for equality with ==.

I think we'd basically need an additional version of ResourceArray (type
+ functions) which can store some additional data for each entry?

Greetings,

Andres Freund



Commits

  1. Fix copying data into slots with FDW batching

  2. Optimize creation of slots for FDW bulk inserts

  3. Adjust batch size in postgres_fdw to not use too many parameters