Horton

Re: Fast COPY FROM based on batch insert

Etsuro Fujita <etsuro.fujita@gmail.com>

View thread View on postgr.es
From: Etsuro Fujita <etsuro.fujita@gmail.com>
To: Andrey Lepikhov <a.lepikhov@postgrespro.ru>
Cc: Zhihong Yu <zyu@yugabyte.com>, Justin Pryzby <pryzby@telsasoft.com>, Amit Langote <amitlangote09@gmail.com>, tanghy.fnst@fujitsu.com, Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>, Pg Hackers <pgsql-hackers@postgresql.org>, Tomas Vondra <tomas.vondra@enterprisedb.com>, houzj.fnst@fujitsu.com
Date: 2022-10-13T09:58:14Z
Lists: pgsql-hackers 
On Thu, Oct 13, 2022 at 1:38 PM Andrey Lepikhov
<a.lepikhov@postgrespro.ru> wrote:
> On 10/12/22 07:56, Etsuro Fujita wrote:
> > On Tue, Oct 11, 2022 at 3:06 PM Andrey Lepikhov
> > <a.lepikhov@postgrespro.ru> wrote:
> >> I reviewed the patch one more time. Only one question: bistate and
> >> ri_FdwRoutine are strongly bounded. Maybe to add some assertion on
> >> (ri_FdwRoutine XOR bistate) ? Just to prevent possible errors in future.
> >
> > You mean the bistate member of CopyMultiInsertBuffer?
> Yes
> >
> > We do not use that member at all for foreign tables, so the patch
> > avoids initializing that member in CopyMultiInsertBufferInit() when
> > called for a foreign table.  So we have bistate = NULL for foreign
> > tables (and bistate != NULL for plain tables), as you mentioned above.
> > I think it is a good idea to add such assertions.  How about adding
> > them to CopyMultiInsertBufferFlush() and
> > CopyMultiInsertBufferCleanup() like the attached?  In the attached I
> > updated comments a bit further as well.
> Yes, quite enough.

I have committed the patch after tweaking comments a little bit further.

Best regards,
Etsuro Fujita

Commits

  1. Allow batch insertion during COPY into a foreign table.

  2. postgres_fdw: Disable batch insertion when there are WCO constraints.

  3. Enforce foreign key correctly during cross-partition updates

  4. Fix permission checks on constraint violation errors on partitions.

  5. Fix compilation of uuid-ossp