Horton

Re: Make COPY format extendable: Extract COPY TO format implementations

Sutou Kouhei <kou@clear-code.com>

View thread View on postgr.es
From: Sutou Kouhei <kou@clear-code.com>
To: kuroda.hayato@fujitsu.com
Cc: zhjwpku@gmail.com, andrew@dunslane.net, michael@paquier.xyz, sawada.mshk@gmail.com, nathandbossart@gmail.com, pgsql-hackers@postgresql.org
Date: 2023-12-09T20:44:07Z
Lists: pgsql-hackers 
Hi,

Thanks for reviewing our latest patch!

In 
 <TY3PR01MB9889C9234CD220A3A7075F0DF589A@TY3PR01MB9889.jpnprd01.prod.outlook.com>
  "RE: Make COPY format extendable: Extract COPY TO format implementations" on Sat, 9 Dec 2023 02:43:49 +0000,
  "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com> wrote:

> (I remember that this theme was talked at Japan PostgreSQL conference)

Yes. I should have talked to you more at the conference...
I will do it next time!


Can we discuss how to proceed this improvement?

There are 2 approaches for it:

1. Do the followings concurrently:
   a. Implementing small changes that got a consensus and
      merge them step-by-step
      (e.g. We got a consensus that we need to extract the
      current format related routines.)
   b. Discuss design

   (v1-v3 patches use this approach.)

2. Implement one (large) complete patch set with design
   discussion and merge it

   (v4- patches use this approach.)

Which approach is preferred? (Or should we choose another
approach?)

I thought that 1. is preferred because it will reduce review
cost. So I chose 1.

If 2. is preferred, I'll use 2. (I'll add more changes to
the latest patch.)


Thanks,
-- 
kou

Commits

  1. Refactor Copy{From|To}GetRoutine() to use pass-by-reference argument.

  2. Refactor COPY FROM to use format callback functions.

  3. Refactor COPY TO to use format callback functions.

  4. Another try to fix BF failure introduced in commit ddd5f4f54a.

  5. Revert "Refactor CopyReadAttributes{CSV,Text}() to use a callback in COPY FROM"

  6. Improve COPY TO performance when server and client encodings match

  7. Simplify signature of CopyAttributeOutCSV() in copyto.c

  8. Revert "Refactor CopyAttributeOut{CSV,Text}() to use a callback in COPY TO"

  9. Refactor CopyAttributeOut{CSV,Text}() to use a callback in COPY TO

  10. Refactor CopyReadAttributes{CSV,Text}() to use a callback in COPY FROM

  11. Add progress reporting of skipped tuples during COPY FROM.

  12. pgbench: Add \syncpipeline

  13. meson: Make gzip and tar optional

  14. Export the external file reader used in COPY FROM as APIs.