Horton

Re: Streaming I/O, vectored I/O (WIP)

Melanie Plageman <melanieplageman@gmail.com>

View thread View on postgr.es
From: Melanie Plageman <melanieplageman@gmail.com>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: Heikki Linnakangas <hlinnaka@iki.fi>, pgsql-hackers <pgsql-hackers@postgresql.org>, Andres Freund <andres@anarazel.de>
Date: 2023-12-31T18:36:28Z
Lists: pgsql-hackers 
I've written a new version of the vacuum streaming read user on top of
the rebased patch set [1]. It differs substantially from Andres' and
includes several refactoring patches that can apply on top of master.
As such, I've proposed those in a separate thread [2]. I noticed mac
and windows fail to build on CI for my branch with the streaming read
code. I haven't had a chance to investigate  -- but I must have done
something wrong on rebase.

- Melanie

[1] https://github.com/melanieplageman/postgres/tree/stepwise_vac_streaming_read
[2] https://www.postgresql.org/message-id/CAAKRu_Yf3gvXGcCnqqfoq0Q8LX8UM-e-qbm_B1LeZh60f8WhWA%40mail.gmail.com

Commits

  1. Fix typos and incorrect type in read_stream.c

  2. Use streaming I/O in pg_prewarm.

  3. Provide API for streaming relation data.

  4. Provide vectored variant of ReadBuffer().

  5. Provide vectored variants of smgrread() and smgrwrite().

  6. Provide multi-block smgrprefetch().

  7. Provide vectored variants of FileRead() and FileWrite().

  8. Provide helper for retrying partial vectored I/O.

  9. Optimize pg_readv/pg_pwritev single vector case.

  10. bufmgr: Support multiple in-progress IOs by using resowner