Re: Fast COPY FROM based on batch insert
Andrei Lepikhov <a.lepikhov@postgrespro.ru>
From: Andrey Lepikhov <a.lepikhov@postgrespro.ru>
To: Etsuro Fujita <etsuro.fujita@gmail.com>
Cc: Justin Pryzby <pryzby@telsasoft.com>, Zhihong Yu <zyu@yugabyte.com>,
Amit Langote <amitlangote09@gmail.com>, tanghy.fnst@fujitsu.com,
Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>,
Pg Hackers <pgsql-hackers@postgresql.org>,
Tomas Vondra <tomas.vondra@enterprisedb.com>, houzj.fnst@fujitsu.com
Date: 2022-07-27T05:42:28Z
Lists: pgsql-hackers
On 7/22/22 13:14, Etsuro Fujita wrote:
> On Fri, Jul 22, 2022 at 3:39 PM Andrey Lepikhov
> <a.lepikhov@postgrespro.ru> wrote:
>> Analyzing multi-level heterogeneous partitioned configurations I
>> realized, that single write into a partition with a trigger will flush
>> buffers for all other partitions of the parent table even if the parent
>> haven't any triggers.
>> It relates to the code:
>> else if (insertMethod == CIM_MULTI_CONDITIONAL &&
>> !CopyMultiInsertInfoIsEmpty(&multiInsertInfo))
>> {
>> /*
>> * Flush pending inserts if this partition can't use
>> * batching, so rows are visible to triggers etc.
>> */
>> CopyMultiInsertInfoFlush(&multiInsertInfo, resultRelInfo);
>> }
>>
>> Why such cascade flush is really necessary, especially for BEFORE and
>> INSTEAD OF triggers?
>
> BEFORE triggers on the chosen partition might query the parent table,
> not just the partition, so I think we need to do this so that such
> triggers can see all the rows that have been inserted into the parent
> table until then.
if you'll excuse me, I will add one more argument.
It wasn't clear, so I've made an experiment: result of a SELECT in an
INSERT trigger function shows only data, existed in the parent table
before the start of COPY.
So, we haven't tools to access newly inserting rows in neighboring
partition and don't need to flush tuple buffers immediately.
Where am I wrong?
--
Regards
Andrey Lepikhov
Postgres Professional
Commits
-
Allow batch insertion during COPY into a foreign table.
- 97da48246d34 16.0 landed
-
postgres_fdw: Disable batch insertion when there are WCO constraints.
- 82593b9a3d5f 16.0 cited
-
Enforce foreign key correctly during cross-partition updates
- ba9a7e392171 15.0 cited
-
Fix permission checks on constraint violation errors on partitions.
- 6214e2b22804 14.0 cited
-
Fix compilation of uuid-ossp
- 525e60b74299 14.0 cited