Horton

Re: Make COPY format extendable: Extract COPY TO format implementations

David G. Johnston <david.g.johnston@gmail.com>

View thread View on postgr.es
From: "David G. Johnston" <david.g.johnston@gmail.com>
To: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: Sutou Kouhei <kou@clear-code.com>, "tgl@sss.pgh.pa.us" <tgl@sss.pgh.pa.us>, "zhjwpku@gmail.com" <zhjwpku@gmail.com>, "michael@paquier.xyz" <michael@paquier.xyz>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2025-05-03T06:37:36Z
Lists: pgsql-hackers 
On Friday, May 2, 2025, Masahiko Sawada <sawada.mshk@gmail.com> wrote:

>
> I'm concerned about allowing multiple 'text' format implementations
> with identical names within the database, as this could lead to
> considerable confusion. When users specify 'text', it would be more
> logical to guarantee that the built-in 'text' format is consistently
> used.


Do you want to only give text/csv/binary this special treatment or also any
future format name we ever decide to implement in core.  If an extension
takes up “xml” and we try to do that in core do we fail an upgrade because
of the conflict, and make it impossible to actually use said extension?

This principle aligns with other customizable components, such
> as custom resource managers, wait events, lightweight locks, and
> custom scans. These components maintain their built-in data/types and
> explicitly prevent the registration of duplicate names.
>

I am totally lost on how any of those resemble this feature.

I’m all for registration to enable additional options and features - but am
against moving away from turning format into a namespaced identifier.  This
is a query-facing feature where namespaces are common and fundamentally
required.  I have some sympathy for the fact that until now one could not
prefix text/binary/csv with pg_catalog to be fully safe, but in reality
DBAs/query authors either put pg_catalog first in their search_path or make
an informed decision when they deviate.  That is the established precedent
relevant to this feature.  The power, and responsibility for education,
lies with the user.

David J.

Commits

  1. Refactor Copy{From|To}GetRoutine() to use pass-by-reference argument.

  2. Refactor COPY FROM to use format callback functions.

  3. Refactor COPY TO to use format callback functions.

  4. Another try to fix BF failure introduced in commit ddd5f4f54a.

  5. Revert "Refactor CopyReadAttributes{CSV,Text}() to use a callback in COPY FROM"

  6. Improve COPY TO performance when server and client encodings match

  7. Simplify signature of CopyAttributeOutCSV() in copyto.c

  8. Revert "Refactor CopyAttributeOut{CSV,Text}() to use a callback in COPY TO"

  9. Refactor CopyAttributeOut{CSV,Text}() to use a callback in COPY TO

  10. Refactor CopyReadAttributes{CSV,Text}() to use a callback in COPY FROM

  11. Add progress reporting of skipped tuples during COPY FROM.

  12. pgbench: Add \syncpipeline

  13. meson: Make gzip and tar optional

  14. Export the external file reader used in COPY FROM as APIs.