Horton

Re: Make COPY format extendable: Extract COPY TO format implementations

Sutou Kouhei <kou@clear-code.com>

View thread View on postgr.es
From: Sutou Kouhei <kou@clear-code.com>
To: sawada.mshk@gmail.com
Cc: andres@anarazel.de, michael@paquier.xyz, david.g.johnston@gmail.com, tgl@sss.pgh.pa.us, zhjwpku@gmail.com, pgsql-hackers@postgresql.org
Date: 2025-07-18T09:49:12Z
Lists: pgsql-hackers 
Hi,

In <CAD21AoAQkjU=o0nX4y0jtX0BnsrqA04g2ABqrUwjT88YeEWarA@mail.gmail.com>
  "Re: Make COPY format extendable: Extract COPY TO format implementations" on Thu, 17 Jul 2025 13:33:13 -0700,
  Masahiko Sawada <sawada.mshk@gmail.com> wrote:

>> I've not followed the development of this patch - but I continue to be
>> concerned about the performance impact it has as-is and the amount of COPY
>> performance improvements it forecloses.
>>
>> This seems to add yet another layer of indirection to a lot of hot functions
>> like CopyGetData() etc.
>>
> 
> The most refactoring works have been done by commit 7717f6300 and
> 2e4127b6d with a slight performance gain. At this stage, we're trying
> to introduce the registration API so that extensions can provide their
> callbacks to the core. Some functions required for I/O such as
> CopyGetData() and CopySendEndOfRow() would be exposed but I'm not
> going to add additional indirection function call layers.

I think Andres is talking about any indirection not only
indirection function call. In this case, "cstate->XXX" ->
"cstate->edata->XXX".

It's also mentioned in my e-mail. I'm not sure whether it
has performance impact but it's better that we benchmark to
confirm whether there is any performance impact or not with
the Copy{From,To}ExecutionData approach.


Thanks,
-- 
kou

Commits

  1. Refactor Copy{From|To}GetRoutine() to use pass-by-reference argument.

  2. Refactor COPY FROM to use format callback functions.

  3. Refactor COPY TO to use format callback functions.

  4. Another try to fix BF failure introduced in commit ddd5f4f54a.

  5. Revert "Refactor CopyReadAttributes{CSV,Text}() to use a callback in COPY FROM"

  6. Improve COPY TO performance when server and client encodings match

  7. Simplify signature of CopyAttributeOutCSV() in copyto.c

  8. Revert "Refactor CopyAttributeOut{CSV,Text}() to use a callback in COPY TO"

  9. Refactor CopyAttributeOut{CSV,Text}() to use a callback in COPY TO

  10. Refactor CopyReadAttributes{CSV,Text}() to use a callback in COPY FROM

  11. Add progress reporting of skipped tuples during COPY FROM.

  12. pgbench: Add \syncpipeline

  13. meson: Make gzip and tar optional

  14. Export the external file reader used in COPY FROM as APIs.