Re: POC: postgres_fdw insert batching

Tomas Vondra <tomas.vondra@enterprisedb.com>

From: Tomas Vondra <tomas.vondra@enterprisedb.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Amit Langote <amitlangote09@gmail.com>, "tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com>, "Andrey V. Lepikhov" <a.lepikhov@postgrespro.ru>, Tomas Vondra <tomas.vondra@2ndquadrant.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2021-01-21T01:45:12Z
Lists: pgsql-hackers

On 1/21/21 2:22 AM, Tom Lane wrote:
> Tomas Vondra <tomas.vondra@enterprisedb.com> writes:
>> I may be wrong, but the most likely explanation seems to be this is due
>> to the junk filter initialization, which simply moves past the end of
>> the mtstate->resultRelInfo array.
> 
> resultRelInfo is certainly pointing at garbage at that point.
> 

Yup. It's pretty amazing the x86 machines seem to be mostly OK with it.

>> It kinda seems the GetForeignModifyBatchSize call should happen before
>> that block. The attached patch fixes this for me (i.e. regression tests
>> pass with no valgrind reports.
> 
>> Or did I get that wrong?
> 
> Don't we need to initialize ri_BatchSize for *each* resultrelinfo,
> not merely the first one?  That is, this new code needs to be
> somewhere inside a loop over the result rels.
> 

Yeah, I think you're right. That's an embarrassing oversight :-(


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



Commits

  1. Fix tuple routing to initialize batching only for inserts

  2. Fix pointer type in ExecForeignBatchInsert SGML docs

  3. Fix initialization of FDW batching in ExecInitModifyTable

  4. Implement support for bulk inserts in postgres_fdw