Horton

Re: index prefetching

Tomas Vondra <tomas@vondra.me>

View thread
From: Tomas Vondra <tomas@vondra.me>
To: Peter Geoghegan <pg@bowt.ie>
Cc: Andres Freund <andres@anarazel.de>, Thomas Munro <thomas.munro@gmail.com>, Nazir Bilal Yavuz <byavuz81@gmail.com>, Robert Haas <robertmhaas@gmail.com>, Melanie Plageman <melanieplageman@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Georgios <gkokolatos@protonmail.com>, Konstantin Knizhnik <knizhnik@garret.ru>, Dilip Kumar <dilipbalaut@gmail.com>
Date: 2025-11-03T00:06:03Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. aio: io_uring: Trigger async processing for large IOs

  2. read stream: Split decision about look ahead for AIO and combining

  3. read_stream: Only increase read-ahead distance when waiting for IO

  4. read_stream: Prevent distance from decaying too quickly

  5. Reduce ExecSeqScan* code size using pg_assume()

  6. Fix rare bug in read_stream.c's split IO handling.

  7. Fix multiranges to behave more like dependent types.

  8. Add EXPLAIN (MEMORY) to report planner memory consumption

  9. Optimize nbtree backward scan boundary cases.

  10. Increment xactCompletionCount during subtransaction abort.

  11. Add nbtree Valgrind buffer lock checks.

  12. Add nbtree high key "continuescan" optimization.

  13. Reduce pinning and buffer content locking for btree scans.

  14. Teach btree to handle ScalarArrayOpExpr quals natively.


On 11/3/25 00:49, Peter Geoghegan wrote:
> On Sun, Oct 12, 2025 at 2:52 PM Peter Geoghegan <pg@bowt.ie> wrote:
>> Attached in a new revision, mostly just to keep CFBot happy following
>> a recent trivial conflict introduced on master.
> 
> Attached is another revision, also just to keep CFBot happy following
> a conflict introduced on master. Nothing really new here (I've been
> working on batching on the table AM side, but nothing to show on that
> just yet).
> 

Thanks.

> One minor thing to note about this revision: I added a comment to
> selfuncs.c's that notes that there's an unfixed bug there. That code
> more or less copies the approach used by nodeIndexonlyscan.c, but
> neglects to take the same precautions around the read
> stream/prefetching see different pages as all-visible that the view
> seen on the consumer side.
> 
> ISTM that the right fix there is to totally rethink the interface such
> that the read stream is directly owned by the table AM. That way we
> won't have to work around inconsistent ideas around which heap pages
> are all-visible because there'll only be one view of that, in a single
> place. We won't have to do anything special in either selfuncs.c or in
> nodeIndexonlyscan.c.
> 

I think we've already more or less agreed that the read_stream should be
managed by the table AM (rather than by indexam.c), because it's up to
the table AM to interpret the TID.

If that also clarifies the IOS handling, that'd be a bonus. I've not
been very happy with having to check visibility in the stream callback
and passing it to the executor. If this gets "nicer", great.


regards

-- 
Tomas Vondra