Horton

Re: index prefetching

Robert Haas <robertmhaas@gmail.com>

View thread
From: Robert Haas <robertmhaas@gmail.com>
To: Peter Geoghegan <pg@bowt.ie>
Cc: Tomas Vondra <tomas@vondra.me>, Andres Freund <andres@anarazel.de>, Melanie Plageman <melanieplageman@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Georgios <gkokolatos@protonmail.com>, Thomas Munro <thomas.munro@gmail.com>, Konstantin Knizhnik <knizhnik@garret.ru>, Dilip Kumar <dilipbalaut@gmail.com>
Date: 2024-11-11T18:33:37Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. aio: io_uring: Trigger async processing for large IOs

  2. read stream: Split decision about look ahead for AIO and combining

  3. read_stream: Only increase read-ahead distance when waiting for IO

  4. read_stream: Prevent distance from decaying too quickly

  5. Reduce ExecSeqScan* code size using pg_assume()

  6. Fix rare bug in read_stream.c's split IO handling.

  7. Fix multiranges to behave more like dependent types.

  8. Add EXPLAIN (MEMORY) to report planner memory consumption

  9. Optimize nbtree backward scan boundary cases.

  10. Increment xactCompletionCount during subtransaction abort.

  11. Add nbtree Valgrind buffer lock checks.

  12. Add nbtree high key "continuescan" optimization.

  13. Reduce pinning and buffer content locking for btree scans.

  14. Teach btree to handle ScalarArrayOpExpr quals natively.

On Mon, Nov 11, 2024 at 1:03 PM Peter Geoghegan <pg@bowt.ie> wrote:
> I almost think of "pin held" and "buffer lock held" as synonymous when
> working on the nbtree code, even though you have this one obscure page
> deletion case where that isn't quite true (plus the TID recycle safety
> business imposed by heapam). As far as protecting the structure of the
> index itself is concerned, holding on to buffer pins alone does not
> matter at all.

That makes sense from the point of view of working with the btree code
itself, but from a system-wide perspective, it's weird to pretend like
the pins don't exist or don't matter just because a buffer lock is
also held. I had actually forgotten that the btree code tends to
pin+lock together; now that you mention it, I remember that I knew it
at one point, but it fell out of my head a long time ago...

> I think that this is exactly what I propose to do, said in a different
> way. (Again, I wouldn't have expressed it in this way because it seems
> obvious to me that buffer pins don't have nearly the same significance
> to an index AM as they do to heapam -- they have no value in
> protecting the index structure, or helping an index scan to reason
> about concurrency that isn't due to a heapam issue.)
>
> Does that make sense?

Yeah, it just really throws me for a loop that you're using "pin" to
mean "pin at a time when we don't also hold a lock." The fundamental
purpose of a pin is to prevent a buffer from being evicted while
someone is in the middle of looking at it, and nothing that uses
buffers can possibly work correctly without that guarantee. Everything
you've written in parentheses there is, AFAICT, 100% wrong if you mean
"any pin" and 100% correct if you mean "a pin held without a
corresponding lock."

-- 
Robert Haas
EDB: http://www.enterprisedb.com