Horton

Re: Streaming I/O, vectored I/O (WIP)

Heikki Linnakangas <hlinnaka@iki.fi>

View thread View on postgr.es
From: Heikki Linnakangas <hlinnaka@iki.fi>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: Dilip Kumar <dilipbalaut@gmail.com>, Robert Haas <robertmhaas@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, Andres Freund <andres@anarazel.de>, Melanie Plageman <melanieplageman@gmail.com>
Date: 2024-03-29T16:28:29Z
Lists: pgsql-hackers 
On 29/03/2024 09:01, Thomas Munro wrote:
> On Fri, Mar 29, 2024 at 9:45 AM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
>> master (213c959a29):            8.0 s
>> streaming-api v13:              9.5 s
> 
> Hmm, that's not great, and I think I know one factor that has
> confounded my investigation and the conflicting reports I have
> received from a couple of people: some are using meson, which is
> defaulting to -O3 by default, and others are using make which gives
> you -O2 by default, but at -O2, GCC doesn't inline that
> StartReadBuffer specialisation that is used in the "fast path", and
> possibly more.  Some of that gap is closed by using
> pg_attribute_inline_always.  Clang fails to inline at any level.  So I
> should probably use the "always" macro there because that is the
> intention.  Still processing the rest of your email...

Ah yeah, I also noticed that the inlining didn't happen with some 
compilers and flags. I use a mix of gcc and clang and meson and autoconf 
in my local environment.

The above micro-benchmarks were with meson and gcc -O3. GCC version:

$ gcc --version
gcc (Debian 12.2.0-14) 12.2.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


-- 
Heikki Linnakangas
Neon (https://neon.tech)

Commits

  1. Fix typos and incorrect type in read_stream.c

  2. Use streaming I/O in pg_prewarm.

  3. Provide API for streaming relation data.

  4. Provide vectored variant of ReadBuffer().

  5. Provide vectored variants of smgrread() and smgrwrite().

  6. Provide multi-block smgrprefetch().

  7. Provide vectored variants of FileRead() and FileWrite().

  8. Provide helper for retrying partial vectored I/O.

  9. Optimize pg_readv/pg_pwritev single vector case.

  10. bufmgr: Support multiple in-progress IOs by using resowner