Horton
Menu

Re: AIO writes vs hint bits vs checksums

Thomas Munro <thomas.munro@gmail.com>

View thread
From: Thomas Munro <thomas.munro@gmail.com>
To: Andres Freund <andres@anarazel.de>
Cc: Noah Misch <noah@leadboat.com>, pgsql-hackers@postgresql.org, Heikki Linnakangas <hlinnaka@iki.fi>, Robert Haas <robertmhaas@gmail.com>
Date: 2024-09-25T00:45:10Z
Lists: pgsql-hackers 
On Wed, Sep 25, 2024 at 8:30 AM Andres Freund <andres@anarazel.de> wrote:
> Just ran that. There probably is a performance difference, but it's small
> (<0.5%) making it somewhat hard to be certain. It looks like the main reason
> for that is ConditionVariableBroadcast() on the iocv shows up even though
> nobody is waiting.

. o O { Gotta fix that.  Memory barriers might be enough to check for
empty wait list?, and even in the slow path, atomic wait lists or
something better than spinlocks... }

> However, our habit of modifying buffers while IO is going on is
> causing issues with filesystem level checksums as well, as evidenced by the
> fact that debug_io_direct = data on btrfs causes filesystem corruption. So I
> tend to think it'd be better to just stop doing that alltogether (we also do
> that for WAL, when writing out a partial page, but a potential fix there would
> be different, I think).

+many.  Interesting point re the WAL variant.  For the record, here's
some discussion and a repro for that problem, which Andrew currently
works around in a build farm animal with mount options:

https://www.postgresql.org/message-id/CA%2BhUKGKSBaz78Fw3WTF3Q8ArqKCz1GgsTfRFiDPbu-j9OFz-jw%40mail.gmail.com

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Require share-exclusive lock to set hint bits and to flush

  2. heapam: Move logic to handle HEAP_MOVED into a helper function

  3. Add very basic test for kill_prior_tuples

  4. aio: Add README.md explaining higher level design

  5. heapam: Only set tuple's block once per page in pagemode

  6. bufmgr: Use AIO in StartReadBuffers()

  7. bufmgr: Implement AIO read support

  8. aio: Implement support for reads in smgr/md/fd

  9. aio: Add io_method=io_uring

  10. aio: Add io_method=worker

  11. aio: Infrastructure for io_method=worker

  12. aio: Add core asynchronous I/O infrastructure

  13. bufmgr: Make it easier to change number of buffer state bits