Thread

Re: libpq: PQgetCopyData() and allocation overhead

Jeroen Vermeulen <jtvjtv@gmail.com> — 2023-03-03T17:04:22Z

On Fri, 3 Mar 2023 at 17:33, Tom Lane <tgl@sss.pgh.pa.us> wrote:

>
> If you mean exposing PQExpBuffer to users of libpq-fe.h, I'd be very
> seriously against that.  I realize that libpq exposes it at an ABI
> level, but that doesn't mean we want non-Postgres code to use it.
> I also don't see what it'd add for this particular use-case.
>

Fair enough.  Never even got around to checking whether it was in the API
already.



> One thing I don't care for at all in the proposed API spec is the bit
> about how the handler function can scribble on the passed buffer.
> Let's not do that.  Declare it const char *, or maybe better const void *.
>

Personally I would much prefer "char" over "void" here:
* It really is a char buffer, containing text.
* If there is to be any type punning, best have it explicit.
* Reduces risk of getting the two pointer arguments the wrong way around.

As for const, I would definitely have preferred that.  But if the caller
needs a zero-terminated string, forcing them into a memcpy() would kind of
defeat the purpose.

I even tried poking a terminating zero in there from inside the function,
but that made the code significantly less efficient.  Optimiser
assumptions, I suppose.


Rather than duplicating most of pqGetCopyData3, I'd suggest revising
> it to take a callback, where the callback is either user-supplied
> or is supplied by PQgetCopyData to emulate the existing behavior.
> This would both avoid duplicate coding and provide a simple check that
> you've made a usable callback API (in particular, one that can do
> something sane for error cases).
>

Can do that, sure.  I'll also try benchmarking a variant that doesn't take
a callback at all, but gives you the buffer pointer in addition to the
size/status return.  I don't generally like callbacks.


Jeroen