Re: AIO writes vs hint bits vs checksums
Heikki Linnakangas <hlinnaka@iki.fi>
From: Heikki Linnakangas <hlinnaka@iki.fi>
To: Andres Freund <andres@anarazel.de>
Cc: pgsql-hackers@postgresql.org, Noah Misch <noah@leadboat.com>,
Robert Haas <robertmhaas@gmail.com>, Thomas Munro <thomas.munro@gmail.com>
Date: 2025-04-01T10:34:53Z
Lists: pgsql-hackers
Attachments
- 0001-Add-very-basic-test-for-kill_prior_tuples.patch (text/x-patch) patch 0001
- 0002-heapam-Move-logic-to-handle-HEAP_MOVED-into-a-helper.patch (text/x-patch) patch 0002
- 0003-bufmgr-Add-BufferLockHeldByMe.patch (text/x-patch) patch 0003
- 0004-heapam-Use-exclusive-lock-on-old-page-in-CLUSTER.patch (text/x-patch) patch 0004
- 0005-bufmgr-Separate-slow-fast-path-of-LockBufHdr.patch (text/x-patch) patch 0005
- 0006-heapam-Add-batch-mode-mvcc-check-and-use-it-in-page-.patch (text/x-patch) patch 0006
- 0007-bufmgr-Add-interface-to-acquire-right-to-set-hint-bi.patch (text/x-patch) patch 0007
- 0008-heapam-Acquire-right-to-set-hint-bits.patch (text/x-patch) patch 0008
- 0009-Acquire-right-to-set-hint-bits-in-the-remaining-plac.patch (text/x-patch) patch 0009
- 0010-bufmgr-Don-t-copy-pages-while-writing-out.patch (text/x-patch) patch 0010
- 0011-bufmgr-Detect-some-missing-BufferPrepareToSetHintBit.patch (text/x-patch) patch 0011
- 0012-WIP-bufmgr-Detect-some-bad-buffer-accesses.patch (text/x-patch) patch 0012
Here's a rebase of these patches. The last patch, to use mprotect() to detect bad buffer accesses, needed some changes because of AIO. I disabled the protection completely for I/O worker processes, it was difficult to be more fine-grained than that. I didn't test it with io_uring. I went ahead and committed the "heapam: Only set tuple's block once per page in pagemode" patch, because it was trivial and independent of the rest. -- Heikki Linnakangas Neon (https://neon.tech)
Commits
Same data as JSON:
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Require share-exclusive lock to set hint bits and to flush
- 82467f627bd4 19 (unreleased) landed
-
heapam: Move logic to handle HEAP_MOVED into a helper function
- 548de59d93d5 19 (unreleased) landed
-
Add very basic test for kill_prior_tuples
- 377b7ab14524 19 (unreleased) landed
-
aio: Add README.md explaining higher level design
- fdd146a8ef2b 18.0 landed
-
heapam: Only set tuple's block once per page in pagemode
- 2904324a88f6 18.0 landed
-
bufmgr: Use AIO in StartReadBuffers()
- 12ce89fd0708 18.0 landed
-
bufmgr: Implement AIO read support
- 047cba7fa0f8 18.0 landed
-
aio: Implement support for reads in smgr/md/fd
- 50cb7505b301 18.0 landed
-
aio: Add io_method=io_uring
- c325a7633fcb 18.0 landed
-
aio: Add io_method=worker
- 247ce06b883d 18.0 landed
-
aio: Infrastructure for io_method=worker
- 55b454d0e140 18.0 landed
-
aio: Add core asynchronous I/O infrastructure
- da7226993fd4 18.0 landed
-
bufmgr: Make it easier to change number of buffer state bits
- 418451bfe161 18.0 landed