Horton

Re: Make COPY format extendable: Extract COPY TO format implementations

Michael Paquier <michael@paquier.xyz>

View thread View on postgr.es
From: Michael Paquier <michael@paquier.xyz>
To: Sutou Kouhei <kou@clear-code.com>
Cc: andres@anarazel.de, sawada.mshk@gmail.com, zhjwpku@gmail.com, andrew@dunslane.net, nathandbossart@gmail.com, pgsql-hackers@postgresql.org
Date: 2024-03-06T06:34:04Z
Lists: pgsql-hackers 
On Tue, Mar 05, 2024 at 05:18:08PM +0900, Sutou Kouhei wrote:
> I'll send the following patches after this patch is
> merged.

I am not sure that my schedule is on track to allow that for this
release, unfortunately, especially with all the other items to review
and discuss to make this thread feature-complete.  There should be
a bit more than four weeks until the feature freeze (date not set in
stone, should be around the 8th of April AoE), but I have less than
the half due to personal issues.  Perhaps if somebody jumps on this
thread, that will be possible..

> They are based on the v6 patch[1]:
> 
> 1. Add copy_handler
>    * This also adds a pg_proc lookup for custom FORMAT
>    * This also adds a test for copy_handler
> 2. Export CopyToStateData
>    * We need it to implement custom copy TO handler
> 3. Add needed APIs to implement custom copy TO handler
>    * Add CopyToStateData::opaque
>    * Export CopySendEndOfRow()
> 4. Export CopyFromStateData
>    * We need it to implement custom copy FROM handler
> 5. Add needed APIs to implement custom copy FROM handler
>    * Add CopyFromStateData::opaque
>    * Export CopyReadBinaryData()

Hmm.  Sounds like a good plan for a split.
--
Michael

Commits

  1. Refactor Copy{From|To}GetRoutine() to use pass-by-reference argument.

  2. Refactor COPY FROM to use format callback functions.

  3. Refactor COPY TO to use format callback functions.

  4. Another try to fix BF failure introduced in commit ddd5f4f54a.

  5. Revert "Refactor CopyReadAttributes{CSV,Text}() to use a callback in COPY FROM"

  6. Improve COPY TO performance when server and client encodings match

  7. Simplify signature of CopyAttributeOutCSV() in copyto.c

  8. Revert "Refactor CopyAttributeOut{CSV,Text}() to use a callback in COPY TO"

  9. Refactor CopyAttributeOut{CSV,Text}() to use a callback in COPY TO

  10. Refactor CopyReadAttributes{CSV,Text}() to use a callback in COPY FROM

  11. Add progress reporting of skipped tuples during COPY FROM.

  12. pgbench: Add \syncpipeline

  13. meson: Make gzip and tar optional

  14. Export the external file reader used in COPY FROM as APIs.