Thread

Re: libpq: PQgetCopyData() and allocation overhead

Jeroen Vermeulen <jtvjtv@gmail.com> — 2023-02-10T12:19:29Z

OK, I've updated the PR with a benchmark (in the main directory).

On this benchmark I'm seeing about a 24% reduction in "user" CPU time, and
a 8% reduction in "system" CPU time.  (Almost no reduction in wall-clock
time.)


Jeroen

On Fri, 10 Feb 2023 at 11:32, Jeroen Vermeulen <jtvjtv@gmail.com> wrote:

> Here's the patch (as a PR just to make it easy to read):
> https://github.com/jtv/postgres/pull/1
>
> I don't have an easily readable benchmark yet, since I've been timing the
> potential impact on libpqxx.  But can do that next.
>
>
> Jeroen
>
> On Fri, Feb 10, 2023, 11:26 Bharath Rupireddy <
> bharath.rupireddyforpostgres@gmail.com> wrote:
>
>> On Fri, Feb 10, 2023 at 3:43 PM Jeroen Vermeulen <jtvjtv@gmail.com>
>> wrote:
>> >
>> > Would there be interest in a variant of PQgetCopyData() that re-uses
>> the same buffer for a new row, rather than allocating a new buffer on each
>> iteration?
>> >
>> > I tried it on a toy benchmark, and it reduced client-side CPU time by
>> about 12%.  (Less of a difference in wall-clock time of course; the client
>> was only using the CPU for a bit over half the time.)
>>
>> Interesting. It might improve logical replication performance too as
>> it uses COPY protocol.
>>
>> Do you mind sharing a patch, test case that you used and steps to
>> verify the benefit?
>>
>> --
>> Bharath Rupireddy
>> PostgreSQL Contributors Team
>> RDS Open Source Databases
>> Amazon Web Services: https://aws.amazon.com
>>
>