Horton

Re: Streaming I/O, vectored I/O (WIP)

Cedric Villemain <cedric.villemain+pgsql@abcsql.com>

View thread View on postgr.es
From: Cédric Villemain <cedric.villemain+pgsql@abcsql.com>
To: Thomas Munro <thomas.munro@gmail.com>, Heikki Linnakangas <hlinnaka@iki.fi>
Cc: Andres Freund <andres@anarazel.de>, pgsql-hackers <pgsql-hackers@postgresql.org>, Melanie Plageman <melanieplageman@gmail.com>
Date: 2023-12-30T12:01:04Z
Lists: pgsql-hackers 
Le 11/12/2023 à 10:12, Thomas Munro a écrit :
> 3.  smgrreadv/smgrwritev patch:
> 
>   * improved ENOSPC handling
>   * improve description of EOF and ENOSPC handling
>   * fixed the sizes reported in dtrace static probes
>   * fixed some words in the docs about that
>   * changed error messages to refer to "blocks %u..%u"
> 
> 4.  smgrprefetch-with-nblocks patch has no change, hasn't drawn any
> comments hopefully because it is uncontroversial.
> 
> I'm planning to commit these fairly soon.


Thanks, very useful additions.
Not sure what you have already done to come next...

I have 2 smalls patches here:
* to use range prefetch in pg_prewarm (smgrprefetch only at the moment, 
using smgrreadv to come next).
* to support nblocks=0 in smgrprefetch (posix_fadvise supports a len=0 
to apply flag from offset to end of file).

Should I add to commitfest ?
---
Cédric Villemain +33 (0)6 20 30 22 52
https://Data-Bene.io
PostgreSQL Expertise, Support, Training, R&D

Commits

  1. Fix typos and incorrect type in read_stream.c

  2. Use streaming I/O in pg_prewarm.

  3. Provide API for streaming relation data.

  4. Provide vectored variant of ReadBuffer().

  5. Provide vectored variants of smgrread() and smgrwrite().

  6. Provide multi-block smgrprefetch().

  7. Provide vectored variants of FileRead() and FileWrite().

  8. Provide helper for retrying partial vectored I/O.

  9. Optimize pg_readv/pg_pwritev single vector case.

  10. bufmgr: Support multiple in-progress IOs by using resowner