Horton

Re: postgres_fdw batching vs. (re)creating the tuple slots

Tomas Vondra <tomas.vondra@enterprisedb.com>

View thread View on postgr.es
From: Tomas Vondra <tomas.vondra@enterprisedb.com>
To: Alexander Pyhalov <a.pyhalov@postgrespro.ru>
Cc: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>, Andres Freund <andres@anarazel.de>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2021-06-16T21:55:17Z
Lists: pgsql-hackers 
On 6/16/21 4:23 PM, Tomas Vondra wrote:
> On 6/16/21 2:36 PM, Alexander Pyhalov wrote:
>>
>> Hi.
>> It seems this commit
>>
>> commit b676ac443b6a83558d4701b2dd9491c0b37e17c4
>> Author: Tomas Vondra <tomas.vondra@postgresql.org>
>> Date:   Fri Jun 11 20:19:48 2021 +0200
>>
>>      Optimize creation of slots for FDW bulk inserts
>>
>> has broken batch insert for partitions with unique indexes.
>>
> 
> Thanks for the report and reproducer!
> 
> Turns out this is a mind-bogglingly silly bug I made in b676ac443b :-( 
> The data is copied into the slots only in the branch that initializes 
> them, so the subsequent batches just insert the same data over and over.
> 
> The attached patch fixes that, and adds a regression test (a bit smaller 
> version of your reproducer). I'll get this committed shortly.
> 

Pushed, after a bit more cleanup and testing.


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Commits

  1. Fix copying data into slots with FDW batching

  2. Optimize creation of slots for FDW bulk inserts

  3. Adjust batch size in postgres_fdw to not use too many parameters