Horton
Menu

Re: AIO writes vs hint bits vs checksums

Nico Williams <nico@cryptonector.com>

View thread
From: Nico Williams <nico@cryptonector.com>
To: Heikki Linnakangas <hlinnaka@iki.fi>
Cc: Andres Freund <andres@anarazel.de>, pgsql-hackers@postgresql.org
Date: 2025-04-01T15:29:31Z
Lists: pgsql-hackers 
On Wed, Oct 30, 2024 at 02:16:51PM +0200, Heikki Linnakangas wrote:
> Acquiring the exclusive lock in step 4 is just a way to wait for all the
> existing share-lockers to release the lock. You wouldn't need to block new
> share-lockers. We have LW_WAIT_UNTIL_FREE, which is almost what we need, but
> it currently ignores share-lockers. So doing this "properly" would require
> more changes to LWLocks. Briefly acquiring an exclusive lock seems
> acceptable though.

The problem is starvation.  For this you really want something more like
rwlocks that do not have the writer starvation problem.  But rwlocks
have other problems too, like in this case forcing readers to wait.

What you want here is something more like a barrier where readers that
did not see that the page has BM_IO_IN_PROGRESS set get to act as though
it's not set while readers that did see that the page has
BM_IO_IN_PROGRESS set don't, and the process that did set that bit gets
to wait for the first set of readers all without blocking the second set
of readers.  That's something akin to an rwlock, but better -- in fact,
it resembles RCU.

Nico
--

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Require share-exclusive lock to set hint bits and to flush

  2. heapam: Move logic to handle HEAP_MOVED into a helper function

  3. Add very basic test for kill_prior_tuples

  4. aio: Add README.md explaining higher level design

  5. heapam: Only set tuple's block once per page in pagemode

  6. bufmgr: Use AIO in StartReadBuffers()

  7. bufmgr: Implement AIO read support

  8. aio: Implement support for reads in smgr/md/fd

  9. aio: Add io_method=io_uring

  10. aio: Add io_method=worker

  11. aio: Infrastructure for io_method=worker

  12. aio: Add core asynchronous I/O infrastructure

  13. bufmgr: Make it easier to change number of buffer state bits