Horton

Re: index prefetching

Peter Geoghegan <pg@bowt.ie>

View thread View on postgr.es
From: Peter Geoghegan <pg@bowt.ie>
To: Andres Freund <andres@anarazel.de>
Cc: Tomas Vondra <tomas@vondra.me>, Alexandre Felipe <o.alexandre.felipe@gmail.com>, Thomas Munro <thomas.munro@gmail.com>, Nazir Bilal Yavuz <byavuz81@gmail.com>, Robert Haas <robertmhaas@gmail.com>, Melanie Plageman <melanieplageman@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Georgios <gkokolatos@protonmail.com>, Konstantin Knizhnik <knizhnik@garret.ru>, Dilip Kumar <dilipbalaut@gmail.com>
Date: 2026-02-28T19:32:35Z
Lists: pgsql-hackers 
On Fri, Feb 27, 2026 at 7:37 PM Andres Freund <andres@anarazel.de> wrote:
> I do continue to wonder if we ought to pass down some hints from the planner
> about how much data an indexscan is likely to read to influence readahead
> aggressiveness.

As you know, I already have some of that, but it isn't particularly
well developed. I agree that pushing the hinting mechanism further and
using something like your proposed READ_STREAM_SLOW_START mechanism
(which you mention in your later email from today) seems most
promising.

> I do agree it's right beign concerned about the increase in index fetches with
> such mark/restore cases.

> Is this, by any chance, with starting the server and running these queries in
> that order?  Are you repeating these runs within one server start, evicting
> the buffers inbetween?

Yes, I repeat the runs 3 times, evicting/prewarming as needed each
time. I go with the fastest run.

> Are you using huge pages?  I see rather differing performance results
> with/without when not prefetching.

Yes, I always use huge_pages (and never use transparent huge pages).

> It really shouldn't be sensitive - that query will never be able to reuse
> heapam pages within a query, and evicting a clean buffer isn't that expensive.

Could have been my previous failure to use "cpupower idle-set -D 0" to
get maximally stable performance for my microbenchmarks. As discussed
privately, I've added this as yet another step in the script that
automates all this for me/that prepares my system to run a
microbenchmark.

-- 
Peter Geoghegan

Commits

  1. read stream: Split decision about look ahead for AIO and combining

  2. read_stream: Only increase read-ahead distance when waiting for IO

  3. aio: io_uring: Trigger async processing for large IOs

  4. heapam: Keep buffer pins across index scan resets.

  5. heapam: Track heap block in IndexFetchHeapData.

  6. Move heapam_handler.c index scan code to new file.

  7. Rename heapam_index_fetch_tuple argument for clarity.

  8. Optimize fast-path FK checks with batched index probes

  9. read_stream: Prevent distance from decaying too quickly

  10. read_stream: Issue IO synchronously while in fast path

  11. bufmgr: Return whether WaitReadBuffers() needed to wait

  12. aio: io_uring: Allow IO methods to check if IO completed in the background

  13. bufmgr: Make UnlockReleaseBuffer() more efficient

  14. Add fake LSN support to hash index AM.

  15. Make IndexScanInstrumentation a pointer in executor scan nodes.

  16. Use fake LSNs to improve nbtree dropPin behavior.

  17. Move fake LSN infrastructure out of GiST.

  18. Use simplehash for backend-private buffer pin refcounts.

  19. nbtree: Avoid allocating _bt_search stack.

  20. bufmgr: Fix use of wrong variable in GetPrivateRefCountEntrySlow()

  21. Conditional locking in pgaio_worker_submit_internal

  22. Reduce ExecSeqScan* code size using pg_assume()

  23. Fix rare bug in read_stream.c's split IO handling.

  24. Remove HeapBitmapScan's skip_fetch optimization

  25. Optimize nbtree backwards scans.

  26. Fix multiranges to behave more like dependent types.

  27. Add EXPLAIN (MEMORY) to report planner memory consumption

  28. Optimize nbtree backward scan boundary cases.

  29. Increment xactCompletionCount during subtransaction abort.

  30. Add nbtree Valgrind buffer lock checks.

  31. Add nbtree high key "continuescan" optimization.

  32. Reduce pinning and buffer content locking for btree scans.

  33. Teach btree to handle ScalarArrayOpExpr quals natively.