Horton

Re: Fast COPY FROM based on batch insert

Etsuro Fujita <etsuro.fujita@gmail.com>

View thread View on postgr.es
From: Etsuro Fujita <etsuro.fujita@gmail.com>
To: Andrey Lepikhov <a.lepikhov@postgrespro.ru>
Cc: Zhihong Yu <zyu@yugabyte.com>, Justin Pryzby <pryzby@telsasoft.com>, Amit Langote <amitlangote09@gmail.com>, tanghy.fnst@fujitsu.com, Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>, Pg Hackers <pgsql-hackers@postgresql.org>, Tomas Vondra <tomas.vondra@enterprisedb.com>, houzj.fnst@fujitsu.com
Date: 2022-10-31T08:50:02Z
Lists: pgsql-hackers 
On Fri, Oct 28, 2022 at 7:53 PM Andrey Lepikhov
<a.lepikhov@postgrespro.ru> wrote:
> On 28/10/2022 16:12, Etsuro Fujita wrote:
> > I think there is another patch that improves performance of COPY FROM
> > for foreign tables using COPY FROM STDIN, but if Andrey (or anyone
> > else) want to work on it again, I think it would be better to create a
> > new CF entry for it (and start a new thread for it).  So I plan to
> > close this in the November CF unless they think otherwise.

> I studied performance of this code in comparison to bulk INSERTions.
> This patch seems to improve speed of insertion by about 20%. Also, this
> patch is very invasive. So, I don't have any plans to work on it now.

Ok, let's leave that for future work.  I closed this entry in the November CF.

Best regards,
Etsuro Fujita

Commits

  1. Allow batch insertion during COPY into a foreign table.

  2. postgres_fdw: Disable batch insertion when there are WCO constraints.

  3. Enforce foreign key correctly during cross-partition updates

  4. Fix permission checks on constraint violation errors on partitions.

  5. Fix compilation of uuid-ossp