Horton

Re: SendRowDescriptionMessage() is slow for queries with a lot of columns

Andres Freund <andres@anarazel.de>

View thread View on postgr.es
From: Andres Freund <andres@anarazel.de>
To: pgsql-hackers@postgresql.org
Date: 2017-09-27T17:20:19Z
Lists: pgsql-hackers

Attachments

Hi,

On 2017-09-13 23:34:18 -0700, Andres Freund wrote:
>    I'm not yet super sure about the implementation. For one, I'm not
>    sure this shouldn't instead be stringinfo.h functions, with very very
>    tiny pqformat.h wrappers. But conversely I think it'd make a lot of
>    sense for the pqformat integer functions to get rid of the
>    continually maintained trailing null-byte - I was hoping the compiler
>    could optimize that away, but alas, no luck.  As soon as a single
>    integer is sent, you can't rely on 0 terminated strings anyway.

I'd been wondering about missing CPU optimizations after the patch, and
hunted it down. Turns out the problem is that htons/ntohs are, on pretty
much all glibc versions, implemented using inline assembler. Which in
turns allows the compiler very little freedom to perform optimizations,
because it doesn't know what's actually happening.

Attached is an extension of the already existing pg_bswap.h that
a) adds 16 bit support
b) moves everything to inline functions, removing multiple evaluation
   hazards that were present everywhere.
c) adds pg_nto{s,l,ll} and pg_hton{s,l,ll} wrappers that only do work
   if necessary.

This'll allow the later patches to allow the compiler to perform the
relevant optimizations. It also allows to optimize e.g. pq_sendint64()
to avoid having to do multiple byteswaps.

Greetings,

Andres Freund

Commits

  1. Replace remaining uses of pq_sendint with pq_sendint{8,16,32}.

  2. Improve performance of SendRowDescriptionMessage.

  3. Use one stringbuffer for all rows printed in printtup.c.

  4. Add configure infrastructure to detect support for C99's restrict.

  5. Add more efficient functions to pqformat API.

  6. Allow to avoid NUL-byte management for stringinfos and use in format.c.

  7. Replace most usages of ntoh[ls] and hton[sl] with pg_bswap.h.

  8. Extend & revamp pg_bswap.h infrastructure.