Horton

Re: index prefetching

Peter Geoghegan <pg@bowt.ie>

View thread View on postgr.es
From: Peter Geoghegan <pg@bowt.ie>
To: Tomas Vondra <tomas@vondra.me>
Cc: Andres Freund <andres@anarazel.de>, Thomas Munro <thomas.munro@gmail.com>, Nazir Bilal Yavuz <byavuz81@gmail.com>, Robert Haas <robertmhaas@gmail.com>, Melanie Plageman <melanieplageman@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Georgios <gkokolatos@protonmail.com>, Konstantin Knizhnik <knizhnik@garret.ru>, Dilip Kumar <dilipbalaut@gmail.com>
Date: 2026-01-07T18:50:53Z
Lists: pgsql-hackers

Attachments

On Sun, Dec 21, 2025 at 12:55 PM Peter Geoghegan <pg@bowt.ie> wrote:
> Attached is v5.

Attached is v6. I'm posting this a little earlier than planned in
order to make sure that the patch continues to apply cleanly against
HEAD (v5 stopped cleanly applying about a week back).

v6 focusses on simplifying the batch management code in
heapam_batch_getnext_tid. Importantly, heapam_batch_getnext_tid no
longer uses a loop to process items from the currently loaded batch/to
load the next batch. The control flow in heapam_batch_getnext_tid is a
lot simpler in general compared to v5.

I still haven't had time to produce an implementation of the "heap
buffer locking minimization" optimization that's clean enough to
present to the list. The heapam_batch_getnext_tid refactoring in v6 is
groundwork for that, though; the previous complicated loop-based
control flow in heapam_batch_getnext_tid made that harder. Hopefully
I'll manage to have that in shape in the next revision.

The corresponding "batch management" control flow used by the stream
callback (added by the second/prefetching patch) still needs to use a
loop -- that can't be avoided/actually makes sense. But I did manage
to make those parts a bit simpler and clearer for v6. Work remains to
simplify that stream callback control flow. We still need to invent
clear invariants that describe the relationship between
readPos/readBatch and streamPos/streamBatch. Some verifying assertions
would also go a long way towards making the readstream callback clean
and maintainable.

-- 
Peter Geoghegan

Commits

  1. read stream: Split decision about look ahead for AIO and combining

  2. read_stream: Only increase read-ahead distance when waiting for IO

  3. aio: io_uring: Trigger async processing for large IOs

  4. heapam: Keep buffer pins across index scan resets.

  5. heapam: Track heap block in IndexFetchHeapData.

  6. Move heapam_handler.c index scan code to new file.

  7. Rename heapam_index_fetch_tuple argument for clarity.

  8. Optimize fast-path FK checks with batched index probes

  9. read_stream: Prevent distance from decaying too quickly

  10. read_stream: Issue IO synchronously while in fast path

  11. bufmgr: Return whether WaitReadBuffers() needed to wait

  12. aio: io_uring: Allow IO methods to check if IO completed in the background

  13. bufmgr: Make UnlockReleaseBuffer() more efficient

  14. Add fake LSN support to hash index AM.

  15. Make IndexScanInstrumentation a pointer in executor scan nodes.

  16. Use fake LSNs to improve nbtree dropPin behavior.

  17. Move fake LSN infrastructure out of GiST.

  18. Use simplehash for backend-private buffer pin refcounts.

  19. nbtree: Avoid allocating _bt_search stack.

  20. bufmgr: Fix use of wrong variable in GetPrivateRefCountEntrySlow()

  21. Conditional locking in pgaio_worker_submit_internal

  22. Reduce ExecSeqScan* code size using pg_assume()

  23. Fix rare bug in read_stream.c's split IO handling.

  24. Remove HeapBitmapScan's skip_fetch optimization

  25. Optimize nbtree backwards scans.

  26. Fix multiranges to behave more like dependent types.

  27. Add EXPLAIN (MEMORY) to report planner memory consumption

  28. Optimize nbtree backward scan boundary cases.

  29. Increment xactCompletionCount during subtransaction abort.

  30. Add nbtree Valgrind buffer lock checks.

  31. Add nbtree high key "continuescan" optimization.

  32. Reduce pinning and buffer content locking for btree scans.

  33. Teach btree to handle ScalarArrayOpExpr quals natively.