Re: postgres_fdw batching vs. (re)creating the tuple slots
Tomas Vondra <tomas.vondra@enterprisedb.com>
From: Tomas Vondra <tomas.vondra@enterprisedb.com>
To: Tom Lane <tgl@sss.pgh.pa.us>, Andres Freund <andres@anarazel.de>
Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2021-06-04T11:52:28Z
Lists: pgsql-hackers
Attachments
- 0001-Add-PQ_QUERY_PARAM_MAX_LIMIT.patch (text/x-patch)
Argh! I forgot the attachments, of course. On 6/4/21 1:48 PM, Tomas Vondra wrote: > Hi, > > Here's two WIP patches that fixes the regression for me. The first part > is from [1], so make large batches work, 0002 just creates a copy of the > tupledesc to not cause issues in resource owner, 0003 ensures we only > initialize the slots once (not per batch). > > With the patches applied, the timings look like this: > > batch timing > ---------------------- > 1 64194.942 ms > 10 7233.785 ms > 100 2244.255 ms > 32k 1372.175 ms > > which seems fine. I still need to get this properly tested etc. and make > sure nothing is left over. > > regards > > > [1] > https://postgr.es/m/OS0PR01MB571603973C0AC2874AD6BF2594299%40OS0PR01MB5716.jpnprd01.prod.outlook.com > -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
Commits
-
Fix copying data into slots with FDW batching
- 99cea49d6525 14.0 landed
-
Optimize creation of slots for FDW bulk inserts
- b676ac443b6a 14.0 landed
-
Adjust batch size in postgres_fdw to not use too many parameters
- cb92703384e2 14.0 cited