Re: Fast COPY FROM based on batch insert

Andrei Lepikhov <a.lepikhov@postgrespro.ru>

From: Andrey Lepikhov <a.lepikhov@postgrespro.ru>

To: Ian Barwick <ian.barwick@enterprisedb.com>

Cc: Justin Pryzby <pryzby@telsasoft.com>, Zhihong Yu <zyu@yugabyte.com>, Amit Langote <amitlangote09@gmail.com>, tanghy.fnst@fujitsu.com, Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>, Pg Hackers <pgsql-hackers@postgresql.org>, Tomas Vondra <tomas.vondra@enterprisedb.com>, Etsuro Fujita <etsuro.fujita@gmail.com>, houzj.fnst@fujitsu.com

Date: 2022-07-11T03:54:08Z

Lists: pgsql-hackers

Attachments

v4-0001-Implementation-of-a-Bulk-COPY-FROM.patch (text/plain) patch v4-0001

On 11/7/2022 04:12, Ian Barwick wrote:
> On 09/07/2022 00:09, Andrey Lepikhov wrote:
>> On 8/7/2022 05:12, Ian Barwick wrote:
>>>      ERROR:  bind message supplies 0 parameters, but prepared 
>>> statement "pgsql_fdw_prep_178" requires 6
>>>      CONTEXT:  remote SQL command: INSERT INTO public.foo_part_1(t, 
>>> v1, v2, v3, v4, v5) VALUES ($1, $2, $3, $4, $5, $6)
>>>      COPY foo, line 88160
>> Thanks, I got it. MultiInsertBuffer are created on the first non-zero 
>> flush of tuples into the partition and isn't deleted from the buffers 
>> list until the end of COPY. And on a subsequent flush in the case of 
>> empty buffer we catch the error.
>> Your fix is correct, but I want to propose slightly different change 
>> (see in attachment).
> 
> LGTM.
New version (with aforementioned changes) is attached.

-- 
regards,
Andrey Lepikhov
Postgres Professional

Commits

Allow batch insertion during COPY into a foreign table.
- 97da48246d34 16.0 landed
postgres_fdw: Disable batch insertion when there are WCO constraints.
- 82593b9a3d5f 16.0 cited
Enforce foreign key correctly during cross-partition updates
- ba9a7e392171 15.0 cited
Fix permission checks on constraint violation errors on partitions.
- 6214e2b22804 14.0 cited
Fix compilation of uuid-ossp
- 525e60b74299 14.0 cited