Horton

Re: Make COPY format extendable: Extract COPY TO format implementations

David G. Johnston <david.g.johnston@gmail.com>

View thread View on postgr.es
From: "David G. Johnston" <david.g.johnston@gmail.com>
To: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: Sutou Kouhei <kou@clear-code.com>, "tgl@sss.pgh.pa.us" <tgl@sss.pgh.pa.us>, "zhjwpku@gmail.com" <zhjwpku@gmail.com>, "michael@paquier.xyz" <michael@paquier.xyz>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2025-05-03T14:42:08Z
Lists: pgsql-hackers 
On Saturday, May 3, 2025, Masahiko Sawada <sawada.mshk@gmail.com> wrote:


> I think that we need to ensure that if users specify text/csv/binary
> the built-in formats are always used, to keep backward compatibility.


That was my original thinking, but it’s inconsistent with how functions
behave today.  We don’t promise that installing extensions won’t cause
existing code to change.


>
>
> > I’m all for registration to enable additional options and features - but
> am against moving away from turning format into a namespaced identifier.
> This is a query-facing feature where namespaces are common and
> fundamentally required.
>
> That's a fair concern. But isn't the format name ultimately just an
> option value, but not like a database object?


We get to decide that.  And deciding in favor of “extensible database
object in a namespace’ makes more sense - leveraging all that pre-existing
design to play more nicely with extensions and give DBAs control.  The SQL
command to add one is “create function” instead of “create copy format”.

David J.

Commits

  1. Refactor Copy{From|To}GetRoutine() to use pass-by-reference argument.

  2. Refactor COPY FROM to use format callback functions.

  3. Refactor COPY TO to use format callback functions.

  4. Another try to fix BF failure introduced in commit ddd5f4f54a.

  5. Revert "Refactor CopyReadAttributes{CSV,Text}() to use a callback in COPY FROM"

  6. Improve COPY TO performance when server and client encodings match

  7. Simplify signature of CopyAttributeOutCSV() in copyto.c

  8. Revert "Refactor CopyAttributeOut{CSV,Text}() to use a callback in COPY TO"

  9. Refactor CopyAttributeOut{CSV,Text}() to use a callback in COPY TO

  10. Refactor CopyReadAttributes{CSV,Text}() to use a callback in COPY FROM

  11. Add progress reporting of skipped tuples during COPY FROM.

  12. pgbench: Add \syncpipeline

  13. meson: Make gzip and tar optional

  14. Export the external file reader used in COPY FROM as APIs.