Thread

Re: libpq: PQgetCopyData() and allocation overhead

Jeroen Vermeulen <jtvjtv@gmail.com> — 2023-03-03T18:25:48Z

On Fri, 3 Mar 2023 at 18:14, Tom Lane <tgl@sss.pgh.pa.us> wrote:

> Jeroen Vermeulen <jtvjtv@gmail.com> writes:
> > On Fri, 3 Mar 2023 at 17:33, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >> Let's not do that.  Declare it const char *, or maybe better const void
> *.
>
> > Personally I would much prefer "char" over "void" here:
> > * It really is a char buffer, containing text.
>
> Not in binary-mode COPY.
>

True.  And in that case zero-termination doesn't matter much either.  But
overall libpq's existing choice seems reasonable.


> As for const, I would definitely have preferred that.  But if the caller
> > needs a zero-terminated string, forcing them into a memcpy() would kind
> of
> > defeat the purpose.
>
> I'm willing to grant that avoiding malloc-and-free is worth the trouble.
> I'm not willing to allow applications to scribble on libpq buffers to
> avoid memcpy.  Even your not-a-patch patch fails to make the case that
> this is essential, because you could have used fwrite() instead of
> printf() (which would be significantly faster yet btw, printf formatting
> ain't cheap).
>

Your house, your rules.  For my own use-case "const" is just peachy.

The printf() is just the simplest example that sprang to mind though.
There may be other use-cases out there involving  libraries that require
zero-terminated strings, and I figured an ability to set a sentinel could
help those.


> Can do that, sure.  I'll also try benchmarking a variant that doesn't take
> > a callback at all, but gives you the buffer pointer in addition to the
> > size/status return.  I don't generally like callbacks.
>
> Um ... that would require an assumption that libpq neither changes nor
> moves that buffer before returning to the caller.  I don't much like
> that either.
>

Not an assumption about _before returning to the caller_ I guess, because
the function would be on top of that anyway.  The concern would be libpq
changing or moving the buffer _before the caller is done with the line._
Which would require some kind of clear rule about what invalidates the
buffer.  Yes, that is easier with the callback.


Jeroen