Horton
Menu

Re: AIO writes vs hint bits vs checksums

Jeff Davis <pgsql@j-davis.com>

View thread
From: Jeff Davis <pgsql@j-davis.com>
To: Andres Freund <andres@anarazel.de>, pgsql-hackers@postgresql.org
Cc: Noah Misch <noah@leadboat.com>, Heikki Linnakangas <hlinnaka@iki.fi>, Robert Haas <robertmhaas@gmail.com>, Thomas Munro <thomas.munro@gmail.com>
Date: 2024-10-30T17:47:35Z
Lists: pgsql-hackers 
On Tue, 2024-09-24 at 11:55 -0400, Andres Freund wrote:
> What I suspect we might want instead is something inbetween a share
> and an
> exclusive lock, which is taken while setting a hint bit and which
> conflicts
> with having an IO in progress.

I am starting to wonder if a shared content locks are really the right
concept at all. It makes sense for simple mutexes, but we are already
more complex than that, and you are suggesting adding on to that
complexity. Which I agree is a good idea, I'm just wondering if we
could go even further.

The README states that a shared lock is necessary for visibility
checking, but can we just be more careful with the ordering and
atomicity of visibility changes in the page?

 * carefully order reads and writes of xmin/xmax/hints (would
   that create too many read barriers in the tqual.c code?)
 * write line pointer after tuple is written

We would still have pins and cleanup locks to prevent data removal.
We'd have the logic you suggest that would prevent modification during
IO. And there would still need to be an exclusive content locks so that
two inserts don't try to allocate the same line pointer, or lock the
same tuple.

If PD_ALL_VISIBLE is set it's even simpler.

Am I missing some major hazards?

Regards,
	Jeff Davis

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Require share-exclusive lock to set hint bits and to flush

  2. heapam: Move logic to handle HEAP_MOVED into a helper function

  3. Add very basic test for kill_prior_tuples

  4. aio: Add README.md explaining higher level design

  5. heapam: Only set tuple's block once per page in pagemode

  6. bufmgr: Use AIO in StartReadBuffers()

  7. bufmgr: Implement AIO read support

  8. aio: Implement support for reads in smgr/md/fd

  9. aio: Add io_method=io_uring

  10. aio: Add io_method=worker

  11. aio: Infrastructure for io_method=worker

  12. aio: Add core asynchronous I/O infrastructure

  13. bufmgr: Make it easier to change number of buffer state bits