Horton

Re: index prefetching

Peter Geoghegan <pg@bowt.ie>

View thread View on postgr.es
From: Peter Geoghegan <pg@bowt.ie>
To: Andres Freund <andres@anarazel.de>
Cc: Tomas Vondra <tomas@vondra.me>, Alexandre Felipe <o.alexandre.felipe@gmail.com>, Thomas Munro <thomas.munro@gmail.com>, Nazir Bilal Yavuz <byavuz81@gmail.com>, Robert Haas <robertmhaas@gmail.com>, Melanie Plageman <melanieplageman@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Georgios <gkokolatos@protonmail.com>, Konstantin Knizhnik <knizhnik@garret.ru>, Dilip Kumar <dilipbalaut@gmail.com>
Date: 2026-03-23T01:14:23Z
Lists: pgsql-hackers

Attachments

On Sat, Mar 21, 2026 at 7:01 PM Peter Geoghegan <pg@bowt.ie> wrote:
> I can immediately act on most of what you've said here. I'm planning
> to commit the first patch (and maybe the hash index fake LSN patch) in
> the next couple of days, ahead of posting a new v17 of the patch set.
> You can expect any item I reply to with "fixed" or similar to be in
> that version. Other items might not be addressed in v17 -- generally
> because I require more context or feedback to act.
>
> I'm going to work through both this email and the later one from today
> before posting this v17. What I've said applies equally to both
> reviews/emails.

I committed the fake LSN hash index patch (as commit e5836f7b), as
well as the "Make IndexScanInstrumentation a pointer in executor scan
nodes" patch (as commit f026fbf0).

V17 is attached. This addresses most of your feedback, but defers
dealing with the trickier parts for now. I didn't want to delay
producing a new version that cleanly applies on top of master.
Performance validation takes at least as long as writing the code,
with these sorts of changes.

Notable things that I have *not* done just yet:

* No new heapam_index.c file in this version (though there is an indexbatch.h).

* No split of table AM revisions that pushes down VM accesses for
index-only scans into its own commit (though there are a couple of new
patches that were split from the big one to keep the size of the
commits slightly more manageable, as promised).

* No progress on splitting the batch management code currently in
heapam_handler.c into smaller, reusable/modular pieces. Pieces that
other table AMs can reuse.

* Recently, Andres asked, "Does it really make sense to not just have
2 separate loops from within heapam_batch_resolve_visibility?". That
is a question that remains unanswered, for now.

Notable changes (things that I *have* done) include:

* Cleaned up the mess of carrying around four full-length, specialized
versions of the table AM index scan slot callback. We still
specialize, but now there's only one generic
pg_attribute_always_inline function definition, which is called by all
four specializations.  The compiler can create specialized versions
because the amgetbatch + index_only_scan parameters are constants
(four specializations for all four combinations of amgetbatch +
index_only_scan boolean parameters).

* Lots of renaming and polishing, along the lines requested in your
review. Basically, everything I specifically told you I would do soon
in my initial responses over the past couple of days has been
completed. A large number of small things, that aren't worth noting
individually.

-- 
Peter Geoghegan

Commits

  1. read stream: Split decision about look ahead for AIO and combining

  2. read_stream: Only increase read-ahead distance when waiting for IO

  3. aio: io_uring: Trigger async processing for large IOs

  4. heapam: Keep buffer pins across index scan resets.

  5. heapam: Track heap block in IndexFetchHeapData.

  6. Move heapam_handler.c index scan code to new file.

  7. Rename heapam_index_fetch_tuple argument for clarity.

  8. Optimize fast-path FK checks with batched index probes

  9. read_stream: Prevent distance from decaying too quickly

  10. read_stream: Issue IO synchronously while in fast path

  11. bufmgr: Return whether WaitReadBuffers() needed to wait

  12. aio: io_uring: Allow IO methods to check if IO completed in the background

  13. bufmgr: Make UnlockReleaseBuffer() more efficient

  14. Add fake LSN support to hash index AM.

  15. Make IndexScanInstrumentation a pointer in executor scan nodes.

  16. Use fake LSNs to improve nbtree dropPin behavior.

  17. Move fake LSN infrastructure out of GiST.

  18. Use simplehash for backend-private buffer pin refcounts.

  19. nbtree: Avoid allocating _bt_search stack.

  20. bufmgr: Fix use of wrong variable in GetPrivateRefCountEntrySlow()

  21. Conditional locking in pgaio_worker_submit_internal

  22. Reduce ExecSeqScan* code size using pg_assume()

  23. Fix rare bug in read_stream.c's split IO handling.

  24. Remove HeapBitmapScan's skip_fetch optimization

  25. Optimize nbtree backwards scans.

  26. Fix multiranges to behave more like dependent types.

  27. Add EXPLAIN (MEMORY) to report planner memory consumption

  28. Optimize nbtree backward scan boundary cases.

  29. Increment xactCompletionCount during subtransaction abort.

  30. Add nbtree Valgrind buffer lock checks.

  31. Add nbtree high key "continuescan" optimization.

  32. Reduce pinning and buffer content locking for btree scans.

  33. Teach btree to handle ScalarArrayOpExpr quals natively.