Horton

Re: index prefetching

Alexandre Felipe <o.alexandre.felipe@gmail.com>

View thread View on postgr.es
From: Alexandre Felipe <o.alexandre.felipe@gmail.com>
To: Peter Geoghegan <pg@bowt.ie>
Cc: Tomas Vondra <tomas@vondra.me>, Andres Freund <andres@anarazel.de>, Thomas Munro <thomas.munro@gmail.com>, Nazir Bilal Yavuz <byavuz81@gmail.com>, Robert Haas <robertmhaas@gmail.com>, Melanie Plageman <melanieplageman@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Georgios <gkokolatos@protonmail.com>, Konstantin Knizhnik <knizhnik@garret.ru>, Dilip Kumar <dilipbalaut@gmail.com>
Date: 2026-02-15T08:00:23Z
Lists: pgsql-hackers

Attachments

Wow, quick response.


OS eviction is only implemented in the python version. But the most
intriguing part is that the python version also shows a degradation by
simply evicting with pg_buffeercache_evict. I checked many times, first I
thought that my differences had a sign inversion. Then I thought it could
be something about the python speed, could it be? I am timing with explain
analyse, if I was fetching the rows this would be the obvious culprit.


How did you account for the OS filesystem cache? It looks like you
> didn't, based on this run_benchmarks.sh code:
>

Sorry I attached the wrong file. I didn't use this run_benchmark.sh.


> How should I go about recreating your result? This was my best guess
> at how to do so. But it doesn't feel like a good guess.


Could you try running the python script or should I provide it in a
different way?


> Are the numbers you showed comparing the patch to the master branch?
> Or is it just comparing enable_indexscan_prefetch=on to
> enable_indexscan_prefetch=off with the patch?


Just changing the parameter.


> Did you write all this test code yourself?
>

In the sense of typing it no, I used cursor, so I just edited what I could
see it didn't get right.

The results you've shown put the patch in a very negative light -- at
> least if taken at face value.


Please notice that I tried to be neutral in the narrative.

Maybe because I tested on a MacOS maybe it works differently from the type
of
"modern storage" mentioned at the start of the thread.

There's no point in speculating what might have happened here until I
> can reproduce your results
>

I totally agree, that is why my first step was to try to reproduce your
results independently.

I see another message, checking that.

--
Alexandre

Commits

  1. read stream: Split decision about look ahead for AIO and combining

  2. read_stream: Only increase read-ahead distance when waiting for IO

  3. aio: io_uring: Trigger async processing for large IOs

  4. heapam: Keep buffer pins across index scan resets.

  5. heapam: Track heap block in IndexFetchHeapData.

  6. Move heapam_handler.c index scan code to new file.

  7. Rename heapam_index_fetch_tuple argument for clarity.

  8. Optimize fast-path FK checks with batched index probes

  9. read_stream: Prevent distance from decaying too quickly

  10. read_stream: Issue IO synchronously while in fast path

  11. bufmgr: Return whether WaitReadBuffers() needed to wait

  12. aio: io_uring: Allow IO methods to check if IO completed in the background

  13. bufmgr: Make UnlockReleaseBuffer() more efficient

  14. Add fake LSN support to hash index AM.

  15. Make IndexScanInstrumentation a pointer in executor scan nodes.

  16. Use fake LSNs to improve nbtree dropPin behavior.

  17. Move fake LSN infrastructure out of GiST.

  18. Use simplehash for backend-private buffer pin refcounts.

  19. nbtree: Avoid allocating _bt_search stack.

  20. bufmgr: Fix use of wrong variable in GetPrivateRefCountEntrySlow()

  21. Conditional locking in pgaio_worker_submit_internal

  22. Reduce ExecSeqScan* code size using pg_assume()

  23. Fix rare bug in read_stream.c's split IO handling.

  24. Remove HeapBitmapScan's skip_fetch optimization

  25. Optimize nbtree backwards scans.

  26. Fix multiranges to behave more like dependent types.

  27. Add EXPLAIN (MEMORY) to report planner memory consumption

  28. Optimize nbtree backward scan boundary cases.

  29. Increment xactCompletionCount during subtransaction abort.

  30. Add nbtree Valgrind buffer lock checks.

  31. Add nbtree high key "continuescan" optimization.

  32. Reduce pinning and buffer content locking for btree scans.

  33. Teach btree to handle ScalarArrayOpExpr quals natively.