Re: index prefetching

Peter Geoghegan <pg@bowt.ie>

From: Peter Geoghegan <pg@bowt.ie>

To: Andres Freund <andres@anarazel.de>

Cc: Tomas Vondra <tomas@vondra.me>, Robert Haas <robertmhaas@gmail.com>, Melanie Plageman <melanieplageman@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Georgios <gkokolatos@protonmail.com>, Thomas Munro <thomas.munro@gmail.com>, Konstantin Knizhnik <knizhnik@garret.ru>, Dilip Kumar <dilipbalaut@gmail.com>

Date: 2025-07-16T21:47:53Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

aio: io_uring: Trigger async processing for large IOs
- a9ee66881744 19 (unreleased) landed
read stream: Split decision about look ahead for AIO and combining
- 8ca147d582a5 19 (unreleased) landed
read_stream: Only increase read-ahead distance when waiting for IO
- f63ca3379025 19 (unreleased) landed
read_stream: Prevent distance from decaying too quickly
- 6e36930f9aaf 19 (unreleased) landed
Reduce ExecSeqScan* code size using pg_assume()
- b227b0bb4e03 19 (unreleased) cited
Fix rare bug in read_stream.c's split IO handling.
- b421223172a2 19 (unreleased) cited
Fix multiranges to behave more like dependent types.
- 3e8235ba4f9c 17.0 cited
Add EXPLAIN (MEMORY) to report planner memory consumption
- 5de890e3610d 17.0 cited
Optimize nbtree backward scan boundary cases.
- c9c0589fda0e 17.0 cited
Increment xactCompletionCount during subtransaction abort.
- 90c885cdab8b 14.0 cited
Add nbtree Valgrind buffer lock checks.
- 4a70f829d86c 14.0 cited
Add nbtree high key "continuescan" optimization.
- 29b64d1de7c7 12.0 cited
Reduce pinning and buffer content locking for btree scans.
- 2ed5b87f96d4 9.5.0 cited
Teach btree to handle ScalarArrayOpExpr quals natively.
- 9e8da0f75731 9.2.0 cited

On Wed, Jul 16, 2025 at 5:41 PM Andres Freund <andres@anarazel.de> wrote:
> I don't mean the index tids, but how the read stream is fed block numbers. In
> the "complex" patch that's done by index_scan_stream_read_next(). And the
> block number it returns is simply
>
>       return ItemPointerGetBlockNumber(tid);
>
> without the table AM having any way of influencing that. Which means that if
> your table AM does not use the block number of the tid 1:1 as the real block
> number, the fetched block will be completely bogus.

How is that handled when such a table AM uses the existing amgettuple
interface? I think that it shouldn't be hard to implement an opt-out
of prefetching for such table AMs, so at least you won't fetch random
garbage.

Right now, the amgetbatch interface is oriented around returning TIDs.
Obviously it works that way because that's what heapam expects, and
what amgettuple (which I'd like to replace with amgetbatch) does.

-- 
Peter Geoghegan