Re: postgres_fdw batching vs. (re)creating the tuple slots

Tom Lane <tgl@sss.pgh.pa.us>

From: Tom Lane <tgl@sss.pgh.pa.us>
To: Andres Freund <andres@anarazel.de>
Cc: Tomas Vondra <tomas.vondra@enterprisedb.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2021-05-30T23:17:11Z
Lists: pgsql-hackers
Andres Freund <andres@anarazel.de> writes:
> On 2021-05-30 17:10:59 -0400, Tom Lane wrote:
>> But it does seem like the hashing scheme somebody added to resowners
>> is a bit too simplistic.  It ought to be able to cope with lots of
>> refs to the same object, or at least not be extra-awful for that case.

> It's not really the hashing that's the problem, right? The array
> representation would have nearly the same problem, I think?

ResourceArrayAdd would have zero problem.  ResourceArrayRemove is
O(1) as long as resources are removed in reverse order ... which
is effectively true if they're all the same resource.  So while
I've not tested, I believe that this particular case would have
no issue at all with the old resowner implementation, stupid
though that was.

> It doesn't seem trivial to improve it without making resowner.c's
> representation a good bit more complicated.

Dunno, I have not studied the new version at all.

			regards, tom lane



Commits

  1. Fix copying data into slots with FDW batching

  2. Optimize creation of slots for FDW bulk inserts

  3. Adjust batch size in postgres_fdw to not use too many parameters