Re: [POC] Fast COPY FROM command for the table with foreign partitions

19.06.2020 19:58, Etsuro Fujita пишет:
> On Tue, Jun 2, 2020 at 2:51 PM Andrey Lepikhov
> <a.lepikhov@postgrespro.ru> wrote:
>> Hiding the COPY code under the buffers management machinery allows us to
>> generalize buffers machinery, execute one COPY operation on each buffer
>> and simplify error handling.
> 
> I'm not sure that it's really a good idea that the bulk-insert API is
> designed the way it's tightly coupled with the bulk-insert machinery
> in the core, because 1) some FDWs might want to send tuples provided
> by the core to the remote, one by one, without storing them in a
> buffer, or 2) some other FDWs might want to store the tuples in the
> buffer and send them in a lump as postgres_fdw in the proposed patch
> but might want to do so independently of MAX_BUFFERED_TUPLES and/or
> MAX_BUFFERED_BYTES defined in the bulk-insert machinery.
> 
> I agree that we would need special handling for cases you mentioned
> above if we design this API based on something like the idea I
> proposed in that thread.
Agreed
> 
>> As i understand, main idea of the thread, mentioned by you, is to add
>> "COPY FROM" support without changes in FDW API.
> 
> I don't think so; I think we should introduce new API for this feature
> to keep the ExecForeignInsert() API simple.
Ok
> 
>> All that I can offer in this place now is to introduce one new
>> ExecForeignBulkInsert(buf) routine that will execute single "COPY FROM
>> STDIN" operation, send tuples and close the operation. We can use the
>> ExecForeignInsert() routine for each buffer tuple if
>> ExecForeignBulkInsert() is not supported.
> 
> Agreed.
In the next version (see attachment) of the patch i removed Begin/End 
fdwapi routines. Now we have only the ExecForeignBulkInsert() routine.

-- 
Andrey Lepikhov
Postgres Professional