Re: POC: postgres_fdw insert batching

Amit Langote <amitlangote09@gmail.com>

From: Amit Langote <amitlangote09@gmail.com>
To: "tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com>
Cc: Tomas Vondra <tomas.vondra@enterprisedb.com>, "Andrey V. Lepikhov" <a.lepikhov@postgrespro.ru>, Tomas Vondra <tomas.vondra@2ndquadrant.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2021-01-19T06:23:51Z
Lists: pgsql-hackers
On Tue, Jan 19, 2021 at 2:06 PM tsunakawa.takay@fujitsu.com
<tsunakawa.takay@fujitsu.com> wrote:
> From: Amit Langote <amitlangote09@gmail.com>
> > I apologize in advance for being maybe overly pedantic, but I noticed
> > that, in ExecInitModifyTable(), you decided to place the call outside
> > the loop that goes over resultRelations (shown below), although my
> > intent was to ask to place it next to the BeginForeignModify() in that
> > loop.
>
> Actually, I tried to do it (adding the GetModifyBatchSize() call after BeginForeignModify()), but it failed.  Because postgresfdwGetModifyBatchSize() wants to know if RETURNING is specified, and ResultRelInfo->projectReturning is created after the above part.  Considering the context where GetModifyBatchSize() implementations may want to know the environment, I placed the call as late as possible in the initialization phase.  As for the future(?) multi-target DML statements, I think we can change this together with other many(?) parts that assume a single target table.

Okay, sometime later then.

I wasn't sure if bringing it up here would be appropriate, but there's
a patch by me to refactor ModfiyTable result relation allocation that
will have to remember to move this code along to an appropriate place
[1].  Thanks for the tip about the dependency on how RETURNING is
handled.  I will remember it when rebasing my patch over this.

--
Amit Langote
EDB: http://www.enterprisedb.com

[1] https://commitfest.postgresql.org/31/2621/



Commits

  1. Fix tuple routing to initialize batching only for inserts

  2. Fix pointer type in ExecForeignBatchInsert SGML docs

  3. Fix initialization of FDW batching in ExecInitModifyTable

  4. Implement support for bulk inserts in postgres_fdw