Re: postgres_fdw batching vs. (re)creating the tuple slots

Tomas Vondra <tomas.vondra@enterprisedb.com>

From: Tomas Vondra <tomas.vondra@enterprisedb.com>
To: Tom Lane <tgl@sss.pgh.pa.us>, Andres Freund <andres@anarazel.de>
Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2021-06-04T11:52:28Z
Lists: pgsql-hackers

Attachments

Argh! I forgot the attachments, of course.

On 6/4/21 1:48 PM, Tomas Vondra wrote:
> Hi,
> 
> Here's two WIP patches that fixes the regression for me. The first part
> is from [1], so make large batches work, 0002 just creates a copy of the
> tupledesc to not cause issues in resource owner, 0003 ensures we only
> initialize the slots once (not per batch).
> 
> With the patches applied, the timings look like this:
> 
>  batch          timing
>  ----------------------
>      1    64194.942 ms
>     10     7233.785 ms
>    100     2244.255 ms
>    32k     1372.175 ms
> 
> which seems fine. I still need to get this properly tested etc. and make
> sure nothing is left over.
> 
> regards
> 
> 
> [1]
> https://postgr.es/m/OS0PR01MB571603973C0AC2874AD6BF2594299%40OS0PR01MB5716.jpnprd01.prod.outlook.com
> 

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Commits

  1. Fix copying data into slots with FDW batching

  2. Optimize creation of slots for FDW bulk inserts

  3. Adjust batch size in postgres_fdw to not use too many parameters