Horton

Re: Fast COPY FROM based on batch insert

Andrei Lepikhov <a.lepikhov@postgrespro.ru>

View thread View on postgr.es
From: Andrey Lepikhov <a.lepikhov@postgrespro.ru>
To: "tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com>
Cc: Etsuro Fujita <etsuro.fujita@gmail.com>, Justin Pryzby <pryzby@telsasoft.com>, Zhihong Yu <zyu@yugabyte.com>, Amit Langote <amitlangote09@gmail.com>, "tanghy.fnst@fujitsu.com" <tanghy.fnst@fujitsu.com>, Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>, Pg Hackers <pgsql-hackers@postgresql.org>, Tomas Vondra <tomas.vondra@enterprisedb.com>, "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>
Date: 2021-06-07T06:36:18Z
Lists: pgsql-hackers 
On 4/6/21 13:45, tsunakawa.takay@fujitsu.com wrote:
> From: Andrey Lepikhov <a.lepikhov@postgrespro.ru>
>> We still have slow 'COPY FROM' operation for foreign tables in current master.
>> Now we have a foreign batch insert operation And I tried to rewrite the patch [1]
>> with this machinery.
> 
> I haven't looked at the patch, but nice performance.
> 
> However, I see the following problems.  What do you think about them?
I agree with your fears.
Think about this patch as an intermediate step on the way to fast COPY 
FROM. This patch contains all logic of the previous patch, except of 
transport machinery (bulk insertion api).
It may be simpler to understand advantages of proposed 'COPY' FDW API 
having committed 'COPY FROM ...' feature based on the bulk insert FDW API.

-- 
regards,
Andrey Lepikhov
Postgres Professional

Commits

  1. Allow batch insertion during COPY into a foreign table.

  2. postgres_fdw: Disable batch insertion when there are WCO constraints.

  3. Enforce foreign key correctly during cross-partition updates

  4. Fix permission checks on constraint violation errors on partitions.

  5. Fix compilation of uuid-ossp