Horton

Re: index prefetching

Thomas Munro <thomas.munro@gmail.com>

View thread
From: Thomas Munro <thomas.munro@gmail.com>
To: Tomas Vondra <tomas@vondra.me>
Cc: Peter Geoghegan <pg@bowt.ie>, Andres Freund <andres@anarazel.de>, Robert Haas <robertmhaas@gmail.com>, Melanie Plageman <melanieplageman@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Georgios <gkokolatos@protonmail.com>, Konstantin Knizhnik <knizhnik@garret.ru>, Dilip Kumar <dilipbalaut@gmail.com>
Date: 2025-07-19T04:03:19Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. aio: io_uring: Trigger async processing for large IOs

  2. read stream: Split decision about look ahead for AIO and combining

  3. read_stream: Only increase read-ahead distance when waiting for IO

  4. read_stream: Prevent distance from decaying too quickly

  5. Reduce ExecSeqScan* code size using pg_assume()

  6. Fix rare bug in read_stream.c's split IO handling.

  7. Fix multiranges to behave more like dependent types.

  8. Add EXPLAIN (MEMORY) to report planner memory consumption

  9. Optimize nbtree backward scan boundary cases.

  10. Increment xactCompletionCount during subtransaction abort.

  11. Add nbtree Valgrind buffer lock checks.

  12. Add nbtree high key "continuescan" optimization.

  13. Reduce pinning and buffer content locking for btree scans.

  14. Teach btree to handle ScalarArrayOpExpr quals natively.

On Sat, Jul 19, 2025 at 6:31 AM Tomas Vondra <tomas@vondra.me> wrote:
> Perhaps the ReadStream should do something like this? Of course, the
> simple patch resets the stream very often, likely mcuh more often than
> anything else in the code. But wouldn't it be beneficial for streams
> reset because of a rescan? Possibly needs to be optional.

Right, that's also discussed, with a similar patch, here:

https://www.postgresql.org/message-id/CA%2BhUKG%2Bx2BcqWzBC77cN0ewhzMF0kYhC6c4G_T2gJLPbqYQ6Ow%40mail.gmail.com

Resetting the distance was a short-sighted mistake: I was thinking
about rescans, the original use case for the reset operation, and
guessing that the data would remain cached.  But all the new users of
_reset() have a completely different motivation, namely temporary
exhaustion in their source data, so that guess was simply wrong.
There was also some discussion at the time about whether "reset so I
can rescan", and "reset so I can continue after a temporary stop"
should be different operations requiring different APIs.  It now seems
like one operation is sufficient, but it should preserve the distance
as you showed and then let the algorithm learn about already-cached
data in the rescan case (if it is even true then, which is also
debatable since it depends on the size of the scan).  So, I think we
should just go ahead and commit a patch like that.