Re: Make COPY format extendable: Extract COPY TO format implementations
Sutou Kouhei <kou@clear-code.com>
Attachments
- v42-0001-Export-CopyDest-as-private-data.patch (text/x-patch)
Hi,
In <CAD21AoC19fV5Ujs-1r24MNU+hwTQUeZMEnaJDjSFwHLMMdFi0Q@mail.gmail.com>
"Re: Make COPY format extendable: Extract COPY TO format implementations" on Wed, 25 Jun 2025 00:48:46 +0900,
Masahiko Sawada <sawada.mshk@gmail.com> wrote:
>> >> It's natural to add more related APIs with this
>> >> approach. The single registration API provides one feature
>> >> by one operation. If we use the RegisterCopyRoutine() for
>> >> FROM and TO formats API, it's not natural that we add more
>> >> related APIs. In this case, some APIs may provide multiple
>> >> features by one operation and other APIs may provide single
>> >> feature by one operation. Developers may be confused with
>> >> the API. For example, developers may think "what does mean
>> >> NULL here?" or "can we use NULL here?" for
>> >> "RegisterCopyRoutine("new-format", NewFormatFromRoutine,
>> >> NULL)".
>> >
>> > We can document it in the comment for the registration function.
>>
>> I think that API that can be understandable without the
>> additional note is better API than API that needs some
>> notes.
>
> I don't see much difference in this case.
OK. It seems that we can't agree on which API is better.
I've implemented your idea as the v42 patch set. Can we
proceed this proposal with this approach? What is the next
step?
> No. I think that if extensions are likely to support both
> CopyToRoutine and CopyFromRoutine in most cases, it would be simpler
> to register the custom format using a single API. Registering
> CopyToRoutine and CopyFromRoutine separately seems redundant to me.
I don't think so. In general, extensions are implemented
step by step. Extension developers will not implement
CopyToRoutine and CopyFromRoutine at once even if extensions
implement both of CopyToRoutine and CopyFromRoutine
eventually.
> Could you provide some examples? It seems to me that even if we
> provide the single API for the registration we can provide other APIs
> differently. For example, if we want to provide an API to register a
> custom option, we can provide RegisterCopyToOption() and
> RegisterCopyFromOption().
Yes. We can mix different style APIs. In general, consistent
style APIs is easier to use than mixed style APIs. If it's
not an important point in PostgreSQL API design, my point is
meaningless. (Sorry, I'm not familiar with PostgreSQL API
design.)
> My point is about the consistency of registration behavior. I think
> that we should raise an error if the custom format name that an
> extension tries to register already exists. Therefore I'm not sure why
> installing extension-A+B is okay but installing extension-C+A or
> extension-C+B is not okay? We can think that's an extension-A's choice
> not to implement CopyFromRoutine for the 'myformat' format so
> extension-B should not change it.
I think that it's the users' responsibility. I think that
it's more convenient that users can mix extension-A+B (A
provides only TO format and B provides only FROM format)
than users can't mix them. I think that extension-A doesn't
want to prohibit FROM format in the case. Extension-A just
doesn't care about FROM format.
FYI: Both of extension-C+A and extension-C+B are OK when we
update not raising an error existing format.
Thanks,
--
kou
Commits
-
Refactor Copy{From|To}GetRoutine() to use pass-by-reference argument.
- bacbc4863b3b 18.0 landed
-
Refactor COPY FROM to use format callback functions.
- 7717f6300693 18.0 landed
-
Refactor COPY TO to use format callback functions.
- 2e4127b6d2d8 18.0 landed
-
Another try to fix BF failure introduced in commit ddd5f4f54a.
- 9bc1eee988c3 17.0 cited
-
Revert "Refactor CopyReadAttributes{CSV,Text}() to use a callback in COPY FROM"
- 06bd311bce24 17.0 landed
-
Improve COPY TO performance when server and client encodings match
- b619852086ed 17.0 cited
-
Simplify signature of CopyAttributeOutCSV() in copyto.c
- b9d6038d7048 17.0 landed
-
Revert "Refactor CopyAttributeOut{CSV,Text}() to use a callback in COPY TO"
- 1aa8324b81fa 17.0 landed
-
Refactor CopyAttributeOut{CSV,Text}() to use a callback in COPY TO
- 2889fd23be56 17.0 landed
-
Refactor CopyReadAttributes{CSV,Text}() to use a callback in COPY FROM
- 95fb5b49024a 17.0 landed
-
Add progress reporting of skipped tuples during COPY FROM.
- 729439607ad2 17.0 cited
-
pgbench: Add \syncpipeline
- 94edfe250c6a 17.0 cited
-
meson: Make gzip and tar optional
- 9ca6e7b9411e 17.0 cited
-
Export the external file reader used in COPY FROM as APIs.
- 8ddc05fb01ee 9.1.0 cited