Re: POC: postgres_fdw insert batching

Amit Langote <amitlangote09@gmail.com>

From: Amit Langote <amitlangote09@gmail.com>
To: Tomas Vondra <tomas.vondra@enterprisedb.com>
Cc: Zhihong Yu <zyu@yugabyte.com>, "tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com>, "Andrey V. Lepikhov" <a.lepikhov@postgrespro.ru>, Tomas Vondra <tomas.vondra@2ndquadrant.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2021-01-21T01:24:55Z
Lists: pgsql-hackers
On Thu, Jan 21, 2021 at 9:56 AM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:
> On 1/21/21 1:17 AM, Zhihong Yu wrote:
> > Hi,
> > The assignment to resultRelInfo is done when junk_filter_needed is true:
> >
> >          if (junk_filter_needed)
> >          {
> >              resultRelInfo = mtstate->resultRelInfo;
> >
> > Should the code for determining batch size access mtstate->resultRelInfo
> > directly ?
> >
>
> IMO the issue is that code iterates over all plans and moves to the next
> for each one:
>
>      resultRelInfo++;
>
> so it ends up pointing past the last element, hence the failures. So
> yeah, either the code needs to move before the loop (per my patch), or
> we need to access mtstate->resultRelInfo directly.

Accessing mtstate->resultRelInfo directly would do.  The only
constraint on where this block should be placed is that
ri_projectReturning must be valid as of calling
GetForeignModifyBatchSize(), as Tsunakawa-san pointed out upthread.
So, after this block in ExecInitModifyTable:

    /*
     * Initialize RETURNING projections if needed.
     */
    if (node->returningLists)
    {
        ....
        /*
         * Build a projection for each result rel.
         */
        resultRelInfo = mtstate->resultRelInfo;
        foreach(l, node->returningLists)
        {
            List       *rlist = (List *) lfirst(l);

            resultRelInfo->ri_returningList = rlist;
            resultRelInfo->ri_projectReturning =
                ExecBuildProjectionInfo(rlist, econtext, slot, &mtstate->ps,
                                        resultRelInfo->ri_RelationDesc->rd_att);
            resultRelInfo++;
        }
    }

-- 
Amit Langote
EDB: http://www.enterprisedb.com



Commits

  1. Fix tuple routing to initialize batching only for inserts

  2. Fix pointer type in ExecForeignBatchInsert SGML docs

  3. Fix initialization of FDW batching in ExecInitModifyTable

  4. Implement support for bulk inserts in postgres_fdw