Re: Make COPY format extendable: Extract COPY TO format implementations
Sutou Kouhei <kou@clear-code.com>
Hi, In <CAD21AoBjzkL2Lv7j4teaHBZvNmKctQtH6X71kN_sj6Fm-+VvJQ@mail.gmail.com> "Re: Make COPY format extendable: Extract COPY TO format implementations" on Tue, 25 Feb 2025 14:05:28 -0800, Masahiko Sawada <sawada.mshk@gmail.com> wrote: > The first two patches are refactoring patches (+ small performance > improvements). I've reviewed these patches again and attached the > updated patches. I reorganized the function order and updated comments > etc. I find that these patches are reasonably ready to push. Could you > review these versions? I'm going to push them, barring objections and > further comments. Sure. Here are some minor comments: 0001: Commit message: > or CSV mode. The performance benchmark results showed ~5% performance > gain intext or CSV mode. intext -> in text > --- a/src/backend/commands/copyto.c > +++ b/src/backend/commands/copyto.c > @@ -20,6 +20,7 @@ > #include "commands/copy.h" > +#include "commands/copyapi.h" We can remove '#include "commands/copy.h"' because it's included in copyapi.h. (0002 does it.) > @@ -254,6 +502,35 @@ CopySendEndOfRow(CopyToState cstate) > +/* > + * Wrapper function of CopySendEndOfRow for text and CSV formats. Sends the > + * the line termination and do common appropriate things for the end of row. > + */ Sends the the line -> Sends the line > --- /dev/null > +++ b/src/include/commands/copyapi.h > + /* End a COPY TO. This callback is called once at the end of COPY FROM */ The last "." is missing: ... COPY FROM. 0002: Commit message: > This change is a preliminary step towards making the COPY TO command > extensible in terms of output formats. COPY TO -> COPY FROM > --- a/src/backend/commands/copyfromparse.c > +++ b/src/backend/commands/copyfromparse.c > @@ -1087,7 +1132,7 @@ NextCopyFrom(CopyFromState cstate, ExprContext *econtext, > static bool > -CopyReadLine(CopyFromState cstate) > +CopyReadLine(CopyFromState cstate, bool is_csv) > @@ -1163,7 +1208,7 @@ CopyReadLine(CopyFromState cstate) > static bool > -CopyReadLineText(CopyFromState cstate) > +CopyReadLineText(CopyFromState cstate, bool is_csv) We may want to add a comment why we don't use "inline" nor "pg_attribute_always_inline" here: https://www.postgresql.org/message-id/CAD21AoBNfKDbJnu-zONNpG820ZXYC0fuTSLrJ-UdRqU4qp2wog%40mail.gmail.com > Yes, I'm not sure it's really necessary to make it inline since the > benchmark results don't show much difference. Probably this is because > the function has 'is_csv' in some 'if' branches but the compiler > cannot optimize out the whole 'if' branches as most 'if' branches > check 'is_csv' and other variables. Or we can add "inline" not "pg_attribute_always_inline" here as a hint for compiler. > --- a/src/include/commands/copyapi.h > +++ b/src/include/commands/copyapi.h > @@ -52,4 +52,50 @@ typedef struct CopyToRoutine > + /* End a COPY FROM. This callback is called once at the end of COPY FROM */ The last "." is missing: ... COPY FROM. I think that these patches are ready to push too. Thanks, -- kou
Commits
-
Refactor Copy{From|To}GetRoutine() to use pass-by-reference argument.
- bacbc4863b3b 18.0 landed
-
Refactor COPY FROM to use format callback functions.
- 7717f6300693 18.0 landed
-
Refactor COPY TO to use format callback functions.
- 2e4127b6d2d8 18.0 landed
-
Another try to fix BF failure introduced in commit ddd5f4f54a.
- 9bc1eee988c3 17.0 cited
-
Revert "Refactor CopyReadAttributes{CSV,Text}() to use a callback in COPY FROM"
- 06bd311bce24 17.0 landed
-
Improve COPY TO performance when server and client encodings match
- b619852086ed 17.0 cited
-
Simplify signature of CopyAttributeOutCSV() in copyto.c
- b9d6038d7048 17.0 landed
-
Revert "Refactor CopyAttributeOut{CSV,Text}() to use a callback in COPY TO"
- 1aa8324b81fa 17.0 landed
-
Refactor CopyAttributeOut{CSV,Text}() to use a callback in COPY TO
- 2889fd23be56 17.0 landed
-
Refactor CopyReadAttributes{CSV,Text}() to use a callback in COPY FROM
- 95fb5b49024a 17.0 landed
-
Add progress reporting of skipped tuples during COPY FROM.
- 729439607ad2 17.0 cited
-
pgbench: Add \syncpipeline
- 94edfe250c6a 17.0 cited
-
meson: Make gzip and tar optional
- 9ca6e7b9411e 17.0 cited
-
Export the external file reader used in COPY FROM as APIs.
- 8ddc05fb01ee 9.1.0 cited