Horton

Re: Streaming I/O, vectored I/O (WIP)

Thomas Munro <thomas.munro@gmail.com>

View thread View on postgr.es
From: Thomas Munro <thomas.munro@gmail.com>
To: Melanie Plageman <melanieplageman@gmail.com>
Cc: Heikki Linnakangas <hlinnaka@iki.fi>, Dilip Kumar <dilipbalaut@gmail.com>, Robert Haas <robertmhaas@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, Andres Freund <andres@anarazel.de>
Date: 2024-03-28T11:16:47Z
Lists: pgsql-hackers

Attachments

On Fri, Mar 29, 2024 at 12:06 AM Thomas Munro <thomas.munro@gmail.com> wrote:
> Small bug fix: the condition in the final test at the end of
> read_stream_look_ahead() wasn't quite right.  In general when looking
> ahead, we don't need to start a read just because the pending read
> would bring us up to stream->distance if submitted now (we'd prefer to
> build it all the way up to size io_combine_limit if we can), but if
> that condition is met AND we have nothing pinned yet, then there is no
> chance for the read to grow bigger by a pinned buffer being consumed.
> Fixed, comment updated.

Oops, I sent the wrong/unfixed version.  This version has the fix
described above.

Commits

  1. Fix typos and incorrect type in read_stream.c

  2. Use streaming I/O in pg_prewarm.

  3. Provide API for streaming relation data.

  4. Provide vectored variant of ReadBuffer().

  5. Provide vectored variants of smgrread() and smgrwrite().

  6. Provide multi-block smgrprefetch().

  7. Provide vectored variants of FileRead() and FileWrite().

  8. Provide helper for retrying partial vectored I/O.

  9. Optimize pg_readv/pg_pwritev single vector case.

  10. bufmgr: Support multiple in-progress IOs by using resowner