Horton

Re: Make COPY format extendable: Extract COPY TO format implementations

Sutou Kouhei <kou@clear-code.com>

View thread View on postgr.es
From: Sutou Kouhei <kou@clear-code.com>
To: sawada.mshk@gmail.com
Cc: pgsql-hackers@postgresql.org
Date: 2024-10-08T08:39:18Z
Lists: pgsql-hackers

Attachments

Hi,

In <CAD21AoD67TAO6KkBecKBsLgR1tgYJS1AwiN9NQJSLE0WYw8pDA@mail.gmail.com>
  "Re: Make COPY format extendable: Extract COPY TO format implementations" on Mon, 7 Oct 2024 15:23:08 -0700,
  Masahiko Sawada <sawada.mshk@gmail.com> wrote:

> I've run the same benchmark script on my various machines (Mac, Linux
> (with Intel CPU and Ryzen CPU) and Raspberry Pi etc). I've not
> investigated the results in depth yet but let me share the results.
> Please find the attached file, extensible_copy_benchmark_20241007.pdf.
> 
> In the benchmark, I've applied the v20 patch set and 'master' in the
> result refers to a19f83f87966. And I disabled CPU turbo boost where
> possible. Overall, v20 patch got a similar or better performance in
> both COPY FROM and COPY TO compared to master except for on MacOS. I'm
> not sure that changes made to master since the last benchmark run by
> Tomas and Suto-san might contribute to these results. I'll try to
> investigate the performance regression that happened on MacOS. I think
> that other performance differences in my results seem to be within
> noises and could be acceptable. Of course, it would be great if others
> also could try to run benchmark tests.

Thanks! They're interesting...

I've run the same benchmark script for the v20 patch set on
the same Intel Core machine when I used for the v19 patch
set:
https://www.postgresql.org/message-id/20240805.072012.2006870620510018355.kou%40clear-code.com

See the attached v{19,20}-*-result.pdf. (v19-*-result.pdf is
the same PDF attached in the above e-mail.) There are no
significant differences. So I think that there are no
related changes in master...


Thanks,
-- 
kou

Commits

  1. Refactor Copy{From|To}GetRoutine() to use pass-by-reference argument.

  2. Refactor COPY FROM to use format callback functions.

  3. Refactor COPY TO to use format callback functions.

  4. Another try to fix BF failure introduced in commit ddd5f4f54a.

  5. Revert "Refactor CopyReadAttributes{CSV,Text}() to use a callback in COPY FROM"

  6. Improve COPY TO performance when server and client encodings match

  7. Simplify signature of CopyAttributeOutCSV() in copyto.c

  8. Revert "Refactor CopyAttributeOut{CSV,Text}() to use a callback in COPY TO"

  9. Refactor CopyAttributeOut{CSV,Text}() to use a callback in COPY TO

  10. Refactor CopyReadAttributes{CSV,Text}() to use a callback in COPY FROM

  11. Add progress reporting of skipped tuples during COPY FROM.

  12. pgbench: Add \syncpipeline

  13. meson: Make gzip and tar optional

  14. Export the external file reader used in COPY FROM as APIs.