Horton
Menu

Thread

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. bufmgr: Fix ordering of checks in PinBuffer()

  2. Use UnlockReleaseBuffer() in more places

  3. bufmgr: Make UnlockReleaseBuffer() more efficient

  4. bufmgr: Don't copy pages while writing out

  5. Fix use of wrong variable in _hash_kill_items()

  6. Fix bug due to confusion about what IsMVCCSnapshot means

  7. bufmgr: Switch to standard order in MarkBufferDirtyHint()

  8. bufmgr: Remove the, now obsolete, BM_JUST_DIRTIED

  9. Require share-exclusive lock to set hint bits and to flush

  10. heapam: Don't mimic MarkBufferDirtyHint() in inplace updates

  11. bufmgr: Allow conditionally locking of already locked buffer

  12. bufmgr: Avoid spurious compiler warning after fcb9c977aa5

  13. lwlock: Remove ForEachLWLockHeldByMe

  14. lwlock: Remove support for disowned lwlwocks

  15. bufmgr: Implement buffer content locks independently of lwlocks

  16. bufmgr: Change BufferDesc.state to be a 64-bit atomic

  17. lwlock: Improve local variable name

  18. lwlock: Invert meaning of LW_FLAG_RELEASE_OK

  19. bufmgr: Make definitions related to buffer descriptor easier to modify

  20. heapam: Add batch mode mvcc check and use it in page mode

  21. freespace: Don't modify page without any lock

  22. heapam: Move logic to handle HEAP_MOVED into a helper function

  23. bufmgr: Optimize & harmonize LockBufHdr(), LWLockWaitListLock()

  24. bufmgr: Add one-entry cache for private refcount

  25. bufmgr: Separate keys for private refcount infrastructure

  26. Add pg_atomic_unlocked_write_u64

  27. Rename BUFFERPIN wait event class to BUFFER

  28. bufmgr: Turn BUFFER_LOCK_* into an enum

  29. lwlock: Fix, currently harmless, bug in LWLockWakeup()

  30. bufmgr: Use atomic sub for unpinning buffers

  31. bufmgr: Allow some buffer state modifications while holding header lock

  32. bufmgr: Fix valgrind checking for buffers pinned in StrategyGetBuffer()

  33. bufmgr: Don't lock buffer header in StrategyGetBuffer()

  34. bufmgr: fewer calls to BufferDescriptorGetContentLock

  35. bufmgr: Fix signedness of mask variable in BufferSync()

  36. bufmgr: Introduce FlushUnlockedBuffer

  37. Improve ReadRecentBuffer() scalability

  1. Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2025-08-22T19:44:48Z

    Hi,


I'm working on making bufmgr.c ready for AIO writes. That requires some
infrastructure changes that I wanted to discuss... In this email I'm trying to
outline the problems and what I currently think we should do.


== Problem 1 - hint bits ==

Currently pages that are being written out are copied if checksums are
enabled. The copy is needed because hint bits can be set with just a share
lock. Writing out a buffer only requires a share lock. If we didn't copy the
buffer, the computed checksum can be falsified by the checksum.

Even without checksums hint bits being set while IO is ongoing is an issue,
e.g. btrfs assumes that pages do not change while being written out with
direct-io, and corrupts itself if they do [1].

The copy we make to avoid checksum failure are not at all cheap [2], but are
particularly problematic for AIO, where a lot of buffers can undergo AIO at
the same time. For AIO the issue is that the longer-lived bounce buffers that
I had initially prototyped actually end up using a lot of memory, making it
hard to figure out what defaults to set.

In [2] I had worked on an approach to avoid copying pages during writes. It
avoided the need to copy buffers by not allowing hint bits to be set while IO
is ongoing. It did so by skipping hint bit writes while IO is ongoing (plus a
fair bit of infrastructure to make that cheap enough).


In that thread Heikki was wondering whether we should instead not go for a
more fundamental solution to the problem, like introducing a separate lock
level that prevents concurrent modifications but allows share lockers. At the
time I was against that, because it seemed like a large project.

However, since then I realized there's a architecturally related second issue:


== Problem 2 - AIO writes vs exclusive locks ==

Separate from the hint bit issue, there is a second issue that I didn't have a
good answer for: Making acquiring an exclusive lock concurrency safe in the
presence of asynchronous writes:

The problem is that while a buffer is being written out, it obviously has to
be share locked. That's true even with AIO. With AIO the share lock is held
once the IO is completed. The problem is that if a backend wants to
exclusively lock a buffer undergoing AIO, it can't just wait for the content
lock as today, it might have to actually reap the IO completion from the
operating system. If one just were to wait for the content lock, there's no
forward progress guarantee.

The buffer's state "knows" that it's undergoing write IO (BM_VALID and
BM_IO_IN_PROGRESS are set). To ensure forward progress guarantee, an exclusive
locker needs to wait for the IO (pgaio_wref_wait(BufferDesc->->io_wref)). The
problem is that it's surprisingly hard to do so race free:

If a backend A were to just check if a buffer is undergoing IO before locking
it, a backend B could start IO on the buffer between A checking for
BM_IO_IN_PROGRESS and acquiring the content lock.  We obviously can't just
hold the buffer header spinlock across a blocking lwlock acquisition.

There potentially are ways to synchronize the buffer state and the content
lock, but it requires deep integration between bufmgr.c and lwlock.c.


== Problem 3 - Cacheline contention ==

This is unrelated to AIO, but might influence the architecture for potential
solutions.  It might make sense to skip over this section on a quicker
read-through.

The problem is that in some workloads the cacheline containing the BufferDesc
becomes very hot. The most common case are workloads with lots of index nested
loops, the root page and some of the other inner pages in the index can become
very contended.

I've seen cases where running the same workload in four separate copies in the
same postgres instance yields ~8x the throughput - on a machine with forty
cores, there's machines with many more these days [4]/

There are three main issues:

a) We manipulate the same cache line multiple times. E.g. btree always first
   pins the page and then locks the page, which performs two atomic operations
   on the same cacheline.

   I've experimented with putting the content lock on a separate cacheline,
   but that causes regressions in other cases.

   Leaving nontrivial implementation issues aside, we could release the lock
   and pin at the same time. With a bit increased difficulty we could do the
   same for pin and lock acquisition.  nbtree almost always does the two
   together, so it'd not be hard to make it benefit from such an optimization.


b) Many operations, like unpinning a buffer, need to use CAS, instead of an
   atomic subtraction.

   atomic add/sub scales a *lot* better than compare and swap, as there is no
   need to retry. With increased contention the number of retries increases
   further and further.

   The reason we need to use CAS in so many places is the following:

   Historically postgres' spinlocks don't use an atomic operation to release
   the spinlock on x86. When making buffer header locks their own thing, I
   carried that forward, to avoid performance regressions.  Because of that
   BufferDesc.state may not be modified while the buffer header spinlock is
   held - which is incompatible with using atomic-sub.

   However, since then we converted most of the "hot" uses of buffer header
   spinlocks into CAS loops (see e.g. PinBuffer()). That makes it feasible to
   use an atomic operation for the buffer header lock release, which in turn
   allows e.g. unpinning with an atomic sub.


c) Read accesses to the BufferDesc cause contention

   Some code, like nbtree, relies on functions like
   BufferGetBlockNumber(). Unfortunately that contends with concurrent
   modifications of the buffer descriptor (like pinning). Potential solutions
   are to rely less on functions like BufferGetBlockNumber() or to split out
   the memory for that into a separate (denser?) array.


d) Even after addressing all of the above, there's still a lot of contention

   I think the solution here would be something roughly to fastpath locks. If
   a buffer is very contended, we can mark it as super-pinned & share locked,
   avoiding any atomic operation on the buffer descriptor itself. Instead the
   current lock and pincount would be stored in each backends PGPROC.
   Obviously evicting or exclusively-locking such a buffer would be a lot more
   expensive.

   I've prototyped it and it helps a *lot*.  The reason I mention this here is
   that this seems impossible to do while using the generic lwlocks for the
   content lock.



== Solution ? ==

My conclusion from the above is that we ought to:


A) Make Buffer Locks something separate from lwlocks

   As part of that introduce a new lock level in-between share and exclusive
   locks (provisionally called share-exclusive, but I hate it). The new lock
   level allows other share lockers, but can only be held by one backend.

   This allows to change the rules so that:

   1) Share lockers are not allowed to modify buffers anymore
   2) Hint bits need the new lock mode (conditionally upgrading the lock in SetHintBits())
   3) Write IO needs to the new lock level

   This addresses 1) from above


B) Merge BufferDesc.state and the content lock

   This allows to address 2) from above, as we now atomically can check if IO
   was concurrently initiated.

   Obviously BufferDesc.state is not currently wide enough, therefore the
   buffer state has to be updated to a 64bit variable.


C) Allow some modifications of BufferDesc.state while holding spinlock

   Just naively doing the above two things reduces scalability, as the
   likelihood of CAS failures increases, due to increased number of
   modifications of the same atomic variable.

   However, by allowing unpinning while the buffer header spinlock is held,
   scalability considerably improves in my tests.

   Doing so requires changing all uses of LockBufHdr(), but by introducing a
   static inline helper the complexity can largely be encapsulated.



I've prototyped the above. The current state is pretty rough, but before I
spend the non-trivial time to make it into an understandable sequence of
changes, I wanted to get feedback.


Does this plan sound reasonable?


The hardest part about this change is that everything kind of depends on each
other. The changes are large enough that they clearly can't just be committed
at once, but doing them over time risks [temporary] performance regressions.




The order of changes I think makes the most sense is the following:

1) Allow some modifications while holding the buffer header spinlock

2) Reduce buffer pin with just an atomic-sub

   This needs to happen first, otherwise there are performance regressions
   during the later steps.

3) Widen BufferDesc.state to 64 bits

4) Implement buffer locking inside BufferDesc.state

5) Do IO while holding share-exclusive lock and require all buffer
   modifications to at least hold share exclusive lock

6) Wait for AIO when acquiring an exclusive content lock

(some of these will likely have parts of their own, but that's details)


Sane?


DOES ANYBODY HAVE A BETTER NAME THAN SHARE-EXCLUSIVE???!?


Greetings,

Andres Freund

[1] https://www.postgresql.org/message-id/CA%2BhUKGKSBaz78Fw3WTF3Q8ArqKCz1GgsTfRFiDPbu-j9OFz-jw%40mail.gmail.com
[2] https://www.postgresql.org/message-id/stj36ea6yyhoxtqkhpieia2z4krnam7qyetc57rfezgk4zgapf%40gcnactj4z56m
[3] In some cases it causes slowdowns for checkpointer close to 50%!
[4] https://anarazel.de/talks/2024-10-23-pgconf-eu-numa-vs-postgresql/numa-vs-postgresql.pdf - slide 19

  2. Re: Buffer locking is special (hints, checksums, AIO writes)

    Mihail Nikalayeu <mihailnikalayeu@gmail.com> — 2025-08-23T10:31:53Z

    Hello, Andres!

Andres Freund <andres@anarazel.de>:
>
>    As part of that introduce a new lock level in-between share and exclusive
>    locks (provisionally called share-exclusive, but I hate it). The new lock
>    level allows other share lockers, but can only be held by one backend.
>
>    This allows to change the rules so that:
>
>    1) Share lockers are not allowed to modify buffers anymore
>    2) Hint bits need the new lock mode (conditionally upgrading the lock in SetHintBits())
>    3) Write IO needs to the new lock level

IIUC, it may be mapped to existing locking system:

BUFFER_LOCK_SHARE          ---->      AccessShareLock
new lock mode                           ---->     ExclusiveLock
BUFFER_LOCK_EXCLUSIVE   ---->     AccessExclusiveLock

So, it all may be named:

BUFFER_LOCK_ACCESS_SHARE
BUFFER_LOCK_EXCLUSIVE
BUFFER_LOCK_ACCESS_EXCLUSIVE

being more consistent with table-level locking system.

Greetings,
Mikhail.

  3. Re: Buffer locking is special (hints, checksums, AIO writes)

    Robert Haas <robertmhaas@gmail.com> — 2025-08-26T20:21:36Z

    On Fri, Aug 22, 2025 at 3:45 PM Andres Freund <andres@anarazel.de> wrote:
> My conclusion from the above is that we ought to:
>
> A) Make Buffer Locks something separate from lwlocks
> B) Merge BufferDesc.state and the content lock
> C) Allow some modifications of BufferDesc.state while holding spinlock

+1 to (A) and (B). No particular opinion on (C) but if it works well, great.

> The order of changes I think makes the most sense is the following:
>
> 1) Allow some modifications while holding the buffer header spinlock
> 2) Reduce buffer pin with just an atomic-sub
> 3) Widen BufferDesc.state to 64 bits
> 4) Implement buffer locking inside BufferDesc.state
> 5) Do IO while holding share-exclusive lock and require all buffer
>    modifications to at least hold share exclusive lock
> 6) Wait for AIO when acquiring an exclusive content lock

No strong objections. I certainly like getting to (5) and (6) and I
think those are in the right order. I'm not sure about the rest. I
thought (1) and (2) were the same change after reading your email; and
it surprises me a little bit that (2) is separate from (4). But I'm
sure you have a much better sense of this than I do.

> DOES ANYBODY HAVE A BETTER NAME THAN SHARE-EXCLUSIVE???!?

AFAIK "share exclusive" or "SX" is standard terminology. While I'm not
wholly hostile to the idea of coming up with something else, I don't
think our tendency to invent our own way to do everything is one of
our better tendencies as a project.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  4. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2025-08-26T21:00:13Z

    Hi,

On 2025-08-26 16:21:36 -0400, Robert Haas wrote:
> On Fri, Aug 22, 2025 at 3:45 PM Andres Freund <andres@anarazel.de> wrote:
> > My conclusion from the above is that we ought to:
> >
> > A) Make Buffer Locks something separate from lwlocks
> > B) Merge BufferDesc.state and the content lock
> > C) Allow some modifications of BufferDesc.state while holding spinlock
> 
> +1 to (A) and (B). No particular opinion on (C) but if it works well, great.

Without it I see performance regressions due to the increased rate of CAS
failures due to having more changes to one atomic variable :/



> > The order of changes I think makes the most sense is the following:
> >
> > 1) Allow some modifications while holding the buffer header spinlock
> > 2) Reduce buffer pin with just an atomic-sub
> > 3) Widen BufferDesc.state to 64 bits
> > 4) Implement buffer locking inside BufferDesc.state
> > 5) Do IO while holding share-exclusive lock and require all buffer
> >    modifications to at least hold share exclusive lock
> > 6) Wait for AIO when acquiring an exclusive content lock
> 
> No strong objections. I certainly like getting to (5) and (6) and I
> think those are in the right order. I'm not sure about the rest.


> I thought (1) and (2) were the same change after reading your email

They are certainly related. I thought it'd make sense to split them as
outlined above, as (1) is relatively verbose on its own, but far more
mechanical.


> and it surprises me a little bit that (2) is separate from (4).

Without doing 2) first, I see performance/scalability regressions doing
(4). Doing (3) without (2) also hurts...



> > DOES ANYBODY HAVE A BETTER NAME THAN SHARE-EXCLUSIVE???!?
> 
> AFAIK "share exclusive" or "SX" is standard terminology. While I'm not
> wholly hostile to the idea of coming up with something else, I don't
> think our tendency to invent our own way to do everything is one of
> our better tendencies as a project.

I guess it bothers me that we'd use share-exclusive to mean the buffer can't
be modified, but for real (vs share, which does allow some modifications). But
it's very well plausible that there's no meaningfully better name, in which
case we certainly shouldn't differ from what's somewhat commonly used.


Greetings,

Andres Freund

  5. Re: Buffer locking is special (hints, checksums, AIO writes)

    Noah Misch <noah@leadboat.com> — 2025-08-27T00:14:49Z

    On Fri, Aug 22, 2025 at 03:44:48PM -0400, Andres Freund wrote:
> I'm working on making bufmgr.c ready for AIO writes.

Nice!

> == Problem 2 - AIO writes vs exclusive locks ==
> 
> Separate from the hint bit issue, there is a second issue that I didn't have a
> good answer for: Making acquiring an exclusive lock concurrency safe in the
> presence of asynchronous writes:
> 
> The problem is that while a buffer is being written out, it obviously has to
> be share locked. That's true even with AIO. With AIO the share lock is held
> once the IO is completed. The problem is that if a backend wants to
> exclusively lock a buffer undergoing AIO, it can't just wait for the content
> lock as today, it might have to actually reap the IO completion from the
> operating system. If one just were to wait for the content lock, there's no
> forward progress guarantee.
> 
> The buffer's state "knows" that it's undergoing write IO (BM_VALID and
> BM_IO_IN_PROGRESS are set). To ensure forward progress guarantee, an exclusive
> locker needs to wait for the IO (pgaio_wref_wait(BufferDesc->->io_wref)). The
> problem is that it's surprisingly hard to do so race free:
> 
> If a backend A were to just check if a buffer is undergoing IO before locking
> it, a backend B could start IO on the buffer between A checking for
> BM_IO_IN_PROGRESS and acquiring the content lock.  We obviously can't just
> hold the buffer header spinlock across a blocking lwlock acquisition.
> 
> There potentially are ways to synchronize the buffer state and the content
> lock, but it requires deep integration between bufmgr.c and lwlock.c.

You may have considered and rejected simpler alternatives for (2) before
picking the approach you go on to outline.  Anything interesting?  For
example, I imagine these might work with varying degrees of inefficiency:

- Use LWLockConditionalAcquire() with some nonstandard waiting protocol when
  there's a non-I/O lock conflict.
- Take BM_IO_IN_PROGRESS before exclusive-locking, then release it.

> == Problem 3 - Cacheline contention ==

> c) Read accesses to the BufferDesc cause contention
> 
>    Some code, like nbtree, relies on functions like
>    BufferGetBlockNumber(). Unfortunately that contends with concurrent
>    modifications of the buffer descriptor (like pinning). Potential solutions
>    are to rely less on functions like BufferGetBlockNumber() or to split out
>    the memory for that into a separate (denser?) array.

Agreed.  BufferGetBlockNumber() could even use a new local (non-shmem) data
structure, since the buffer's mapping can't change until we unpin.

> d) Even after addressing all of the above, there's still a lot of contention
> 
>    I think the solution here would be something roughly to fastpath locks. If
>    a buffer is very contended, we can mark it as super-pinned & share locked,
>    avoiding any atomic operation on the buffer descriptor itself. Instead the
>    current lock and pincount would be stored in each backends PGPROC.
>    Obviously evicting or exclusively-locking such a buffer would be a lot more
>    expensive.
> 
>    I've prototyped it and it helps a *lot*.  The reason I mention this here is
>    that this seems impossible to do while using the generic lwlocks for the
>    content lock.

Nice.

On Tue, Aug 26, 2025 at 05:00:13PM -0400, Andres Freund wrote:
> On 2025-08-26 16:21:36 -0400, Robert Haas wrote:
> > On Fri, Aug 22, 2025 at 3:45 PM Andres Freund <andres@anarazel.de> wrote:
> > > The order of changes I think makes the most sense is the following:

No concerns so far.  I won't claim I can picture all the implications and be
sure this is the right thing, but it sounds promising.  I like your principle
of ordering changes to avoid performance regressions.

> > > DOES ANYBODY HAVE A BETTER NAME THAN SHARE-EXCLUSIVE???!?

I would consider {AccessShare, Exclusive, AccessExclusive}.  What the $SUBJECT
proposal calls SHARE-EXCLUSIVE would become Exclusive.  That has the same
conflict matrix as the corresponding heavyweight locks, which seems good.  I
don't love our mode names, particularly ShareRowExclusive being unsharable.
However, learning one special taxonomy is better than learning two.

> > AFAIK "share exclusive" or "SX" is standard terminology.

Can you say more about that?  I looked around at
https://google.com/search?q=share+exclusive+%22sx%22+lock but didn't find
anything well-aligned with the proposal:

https://dev.mysql.com/doc/dev/mysql-server/latest//PAGE_LOCK_ORDER.html looked
most relevant, but it doesn't give the big idea.
https://mysqlonarm.github.io/Understanding-InnoDB-rwlock-stats/ is less
authoritative but does articulate the big idea, as "Shared-Exclusive (SX):
offer write access to the resource with inconsistent read. (relaxed
exclusive)."  That differs from $SUBJECT semantics, in which SHARE-EXCLUSIVE
can't see inconsistent reads.

https://docs.oracle.com/en/database/oracle/oracle-database/19/arpls/DBMS_LOCK.html
has term SX = "sub exclusive".  I gather an SX lock on a table lets one do
SELECT FOR UPDATE on that table (each row is the "sub"component being locked).

https://man.freebsd.org/cgi/man.cgi?query=sx_slock&sektion=9&format=html uses
the term "SX", but it's more like our lwlocks.  One acquires S or X, not
blends of them.

  6. Re: Buffer locking is special (hints, checksums, AIO writes)

    Robert Haas <robertmhaas@gmail.com> — 2025-08-27T14:03:08Z

    On Tue, Aug 26, 2025 at 8:14 PM Noah Misch <noah@leadboat.com> wrote:
> > > AFAIK "share exclusive" or "SX" is standard terminology.
>
> Can you say more about that?

Looks like I was misremembering. I was thinking of Gray & Reuter,
Transaction Processing: Concepts and Techniques, 1993. However,
opening it up, I find that his vocabulary is slightly different. He
offers the following six lock modes: IS, IX, S, SIX, Update, X. "I"
means "intent" and acts as a modifier to the letter that follows.
Hence, SIX means "a course-granularity shared lock with intent to set
finer-granularity exclusive locks" (p. 408). His lock manager is
hierarchical, so taking a SIX lock on a table means that you are
allowed to read all the rows in the table and you are allowed to
exclusive-lock individual rows as desired and nobody is allowed to
exclusive-lock any rows in the table. It is compatible only with IS;
that is, it does not preclude other people from share-locking
individual rows (which might delay your exclusive locks on those
rows). Since we don't have intent-locking in PostgreSQL, I think my
brain mentally flattened this hierarchy down to S, X, SX, but that's
not what he actually wrote.

His "Update" locks are also somewhat interesting: an update lock is
exactly like an exclusive lock except that it permits PAST
share-locks. You take an update lock when you currently need a
share-lock but anticipate the possibility of needing an
exclusive-lock. This is a deadlock avoidance strategy: updaters will
take turns, and some of them will ultimately want exclusive locks and
others won't, but they can't deadlock against each other as long as
they all take "Update" locks initially and don't try to upgrade to
that level later. An updater's attempt to upgrade to an exclusive lock
can still be delayed by, or deadlock against, share lockers, but those
typically won't try to higher lock levels later.

If we were to use the existing PostgreSQL naming convention, I think
I'd probably argue that the nearest parallel to this level is
ShareUpdateExclusive: a self-exclusive lock level that permits
ordinary table access to continue while blocking exclusive locks, used
for an in-flight maintenance operation. But that's arguable, of
course.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  7. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2025-08-27T16:18:27Z

    Hi,

On 2025-08-26 17:14:49 -0700, Noah Misch wrote:
> On Fri, Aug 22, 2025 at 03:44:48PM -0400, Andres Freund wrote:
> > == Problem 2 - AIO writes vs exclusive locks ==
> >
> > Separate from the hint bit issue, there is a second issue that I didn't have a
> > good answer for: Making acquiring an exclusive lock concurrency safe in the
> > presence of asynchronous writes:
> >
> > The problem is that while a buffer is being written out, it obviously has to
> > be share locked. That's true even with AIO. With AIO the share lock is held
> > once the IO is completed. The problem is that if a backend wants to
> > exclusively lock a buffer undergoing AIO, it can't just wait for the content
> > lock as today, it might have to actually reap the IO completion from the
> > operating system. If one just were to wait for the content lock, there's no
> > forward progress guarantee.
> >
> > The buffer's state "knows" that it's undergoing write IO (BM_VALID and
> > BM_IO_IN_PROGRESS are set). To ensure forward progress guarantee, an exclusive
> > locker needs to wait for the IO (pgaio_wref_wait(BufferDesc->->io_wref)). The
> > problem is that it's surprisingly hard to do so race free:
> >
> > If a backend A were to just check if a buffer is undergoing IO before locking
> > it, a backend B could start IO on the buffer between A checking for
> > BM_IO_IN_PROGRESS and acquiring the content lock.  We obviously can't just
> > hold the buffer header spinlock across a blocking lwlock acquisition.
> >
> > There potentially are ways to synchronize the buffer state and the content
> > lock, but it requires deep integration between bufmgr.c and lwlock.c.
>
> You may have considered and rejected simpler alternatives for (2) before
> picking the approach you go on to outline.

I tried a few things...


> Anything interesting?

Not really.

The first one you propose is what I looked at for a while:

> For example, I imagine these might work with varying degrees of
> inefficiency:
>
> - Use LWLockConditionalAcquire() with some nonstandard waiting protocol when
>   there's a non-I/O lock conflict.

It's nontrivial to make this race free - the problem is the case where we *do*
have to wait for an exclusive content lock. It's possible for the lwlock to be
released by the owning backend and for IO to be started, after checking
whether IO is in progress (after LWLockConditionalAcquire() had failed).

I came up with a complicated scheme, where setting IO in progress would
afterwards wake up all lwlock waiters and all exclusive content lock waits
were done with LWLockAcquireOrWait().  I think that was working - but it's
also a slower and seems really fragile and ugly.


> - Take BM_IO_IN_PROGRESS before exclusive-locking, then release it.

That just seems expensive. We could make it cheaper by doing it only if a
LWLockConditionalAcquire() doesn't succeed. But it still seems not great.  And
it doesn't really help with addressing the 'setting hint bits while IO is in
progress" part...


> > == Problem 3 - Cacheline contention ==
>
> > c) Read accesses to the BufferDesc cause contention
> >
> >    Some code, like nbtree, relies on functions like
> >    BufferGetBlockNumber(). Unfortunately that contends with concurrent
> >    modifications of the buffer descriptor (like pinning). Potential solutions
> >    are to rely less on functions like BufferGetBlockNumber() or to split out
> >    the memory for that into a separate (denser?) array.
>
> Agreed.  BufferGetBlockNumber() could even use a new local (non-shmem) data
> structure, since the buffer's mapping can't change until we unpin.

Hm. I didn't think about a backend local datastructure for that, perhaps
because it seems not cheap to maintain (both from a runtime and a space
perspective).

If we store the read-only data for buffers separately from the read-write
data, we could access that from backends without a lock, since it can't change
with the buffer pinned.

One way to do that would be to maintain a back-pointer from the BufferDesc to
the BufferLookupEnt, since the latter *already* contains the BufferTag. We
probably don't want to add another indirection to the buffer mapping hash
table, otherwise we could deduplicate the other way round and just put padding
between the modified and read-only part of a buffer desc.



> On Tue, Aug 26, 2025 at 05:00:13PM -0400, Andres Freund wrote:
> > On 2025-08-26 16:21:36 -0400, Robert Haas wrote:
> > > On Fri, Aug 22, 2025 at 3:45 PM Andres Freund <andres@anarazel.de> wrote:
> > > > The order of changes I think makes the most sense is the following:
>
> No concerns so far.  I won't claim I can picture all the implications and be
> sure this is the right thing, but it sounds promising.  I like your principle
> of ordering changes to avoid performance regressions.

I suspect we'll have to merge this incrementally to stay sane, I don't want to
end up with a period of worse performance due to this, that could make it
harder to evaluate other work.


> > > > DOES ANYBODY HAVE A BETTER NAME THAN SHARE-EXCLUSIVE???!?

> I would consider {AccessShare, Exclusive, AccessExclusive}.

One thing I forgot to mention is that with the proposed re-architecture in
place, we could subsequently go further and make pinning just be a very
lightweight lock level, instead of that being a separate dedicated
infrstructure.  One nice outgrowth of that would be that that acquiring a
cleanup lock would just be a real lock acquisition, instead of the dedicated
limited machinery we have right now.

Which would leave us with:
- reference (pins today)
- share
- share-exclusive
- exclusive
- cleanup

This doesn't quite seem to map onto the heavyweight lock levels in a sensible
way...



> What the $SUBJECT proposal calls SHARE-EXCLUSIVE would become Exclusive.

There are a few hundred references to the lock levels though, seems painful to
rename them :(


> That has the same conflict matrix as the corresponding heavyweight locks,
> which seems good.

> I don't love our mode names, particularly ShareRowExclusive being
> unsharable.

I hate them with a passion :). Except for the most basic ones they just don't
stay in my head for more than a few hours.


> However, learning one special taxonomy is better than learning two.

But that's fair.


Greetings,

Andres Freund

  8. Re: Buffer locking is special (hints, checksums, AIO writes)

    Noah Misch <noah@leadboat.com> — 2025-08-27T19:14:41Z

    On Wed, Aug 27, 2025 at 12:18:27PM -0400, Andres Freund wrote:
> On 2025-08-26 17:14:49 -0700, Noah Misch wrote:
> > On Fri, Aug 22, 2025 at 03:44:48PM -0400, Andres Freund wrote:
> > > == Problem 3 - Cacheline contention ==
> >
> > > c) Read accesses to the BufferDesc cause contention
> > >
> > >    Some code, like nbtree, relies on functions like
> > >    BufferGetBlockNumber(). Unfortunately that contends with concurrent
> > >    modifications of the buffer descriptor (like pinning). Potential solutions
> > >    are to rely less on functions like BufferGetBlockNumber() or to split out
> > >    the memory for that into a separate (denser?) array.
> >
> > Agreed.  BufferGetBlockNumber() could even use a new local (non-shmem) data
> > structure, since the buffer's mapping can't change until we unpin.
> 
> Hm. I didn't think about a backend local datastructure for that, perhaps
> because it seems not cheap to maintain (both from a runtime and a space
> perspective).

Yes, paying off the cost of maintaining it could be tricky.  It could be the
kind of thing where the overhead loses at 10 cores and wins at 40 cores.  It
could also depend heavily on the workload's concurrent pins per backend.

> If we store the read-only data for buffers separately from the read-write
> data, we could access that from backends without a lock, since it can't change
> with the buffer pinned.

Good point.  That alone may be enough of a win.

> One way to do that would be to maintain a back-pointer from the BufferDesc to
> the BufferLookupEnt, since the latter *already* contains the BufferTag. We
> probably don't want to add another indirection to the buffer mapping hash
> table, otherwise we could deduplicate the other way round and just put padding
> between the modified and read-only part of a buffer desc.

I think you're saying clients would save the back-pointer once and dereference
it many times, with each dereference of a saved back-pointer avoiding a shmem
read of BufferDesc.tag.  Is that right?

> > On Tue, Aug 26, 2025 at 05:00:13PM -0400, Andres Freund wrote:
> > > On 2025-08-26 16:21:36 -0400, Robert Haas wrote:
> > > > On Fri, Aug 22, 2025 at 3:45 PM Andres Freund <andres@anarazel.de> wrote:
> > > > > DOES ANYBODY HAVE A BETTER NAME THAN SHARE-EXCLUSIVE???!?
> 
> > I would consider {AccessShare, Exclusive, AccessExclusive}.
> 
> One thing I forgot to mention is that with the proposed re-architecture in
> place, we could subsequently go further and make pinning just be a very
> lightweight lock level, instead of that being a separate dedicated
> infrstructure.  One nice outgrowth of that would be that that acquiring a
> cleanup lock would just be a real lock acquisition, instead of the dedicated
> limited machinery we have right now.
> 
> Which would leave us with:
> - reference (pins today)
> - share
> - share-exclusive
> - exclusive
> - cleanup
> 
> This doesn't quite seem to map onto the heavyweight lock levels in a sensible
> way...

Could map it like this:

AccessShare - pins today
RowShare - check tuple visibility (BUFFER_LOCK_SHARE today)
Share - set hint bits
ShareUpdateExclusive - clean/write out (borrowing Robert's idea)
Exclusive - add tuples, change xmax, etc. (BUFFER_LOCK_EXCLUSIVE today)
AccessExclusive - cleanup lock or evict the buffer

That has a separate level for hint bits vs. I/O, so multiple backends could
set hint bits.  I don't know whether the benchmarks would favor maintaining
that distinction.

> > What the $SUBJECT proposal calls SHARE-EXCLUSIVE would become Exclusive.
> 
> There are a few hundred references to the lock levels though, seems painful to
> rename them :(

Yes, especially in comments and extensions.  Likely more important than that
for the long-term, your latest proposal has the advantage of keeping short
names for the most-commonly-referenced lock types.  (We could keep
BUFFER_LOCK_SHARE with the lower layers translating that into RowShare, but
that weakens or eliminates the benefit of reducing what readers need to
learn.)  For what it's worth, 6 PGXN modules reference BUFFER_LOCK_SHARE
and/or BUFFER_LOCK_EXCLUSIVE.

Compared to share-exclusive, I think I'd prefer a name that describes the use
cases, "set-hints-or-write" (or separate "write" and "set-hints" levels).
What do you think of that?  I don't know whether that should win vs. names
like ShareUpdateExclusive, though.

  9. Re: Buffer locking is special (hints, checksums, AIO writes)

    Robert Haas <robertmhaas@gmail.com> — 2025-08-27T19:22:55Z

    On Wed, Aug 27, 2025 at 12:18 PM Andres Freund <andres@anarazel.de> wrote:
> Which would leave us with:
> - reference (pins today)
> - share
> - share-exclusive
> - exclusive
> - cleanup
>
> This doesn't quite seem to map onto the heavyweight lock levels in a sensible
> way...

Could do: ACCESS SHARE, SHARE, SHARE UPDATE EXCLUSIVE, EXCLUSIVE,
ACCESS EXCLUSIVE.

I've always thought that a pin was a lot like an access share lock and
a cleanup lock was a lot like an access exclusive lock.

But then again, using the same terminology for two different things
might be confusing.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  10. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2025-08-27T19:29:02Z

    Hi,

On 2025-08-27 12:14:41 -0700, Noah Misch wrote:
> On Wed, Aug 27, 2025 at 12:18:27PM -0400, Andres Freund wrote:
> > One way to do that would be to maintain a back-pointer from the BufferDesc to
> > the BufferLookupEnt, since the latter *already* contains the BufferTag. We
> > probably don't want to add another indirection to the buffer mapping hash
> > table, otherwise we could deduplicate the other way round and just put padding
> > between the modified and read-only part of a buffer desc.
> 
> I think you're saying clients would save the back-pointer once and dereference
> it many times, with each dereference of a saved back-pointer avoiding a shmem
> read of BufferDesc.tag.  Is that right?

I was thinking that we'd not have BufferDesc.tag, instead just storing it
solely in BufferLookupEnt. To get the tag of a BufferDesc, you'd every time
have to follow the back-reference.   But that's actually why it doesn't work -
reading the back-reference pointer would have the same issue as just reading
BufferDesc.tag...


> > > On Tue, Aug 26, 2025 at 05:00:13PM -0400, Andres Freund wrote:
> > > > On 2025-08-26 16:21:36 -0400, Robert Haas wrote:
> > > > > On Fri, Aug 22, 2025 at 3:45 PM Andres Freund <andres@anarazel.de> wrote:
> > > > > > DOES ANYBODY HAVE A BETTER NAME THAN SHARE-EXCLUSIVE???!?
> > 
> > > I would consider {AccessShare, Exclusive, AccessExclusive}.
> > 
> > One thing I forgot to mention is that with the proposed re-architecture in
> > place, we could subsequently go further and make pinning just be a very
> > lightweight lock level, instead of that being a separate dedicated
> > infrstructure.  One nice outgrowth of that would be that that acquiring a
> > cleanup lock would just be a real lock acquisition, instead of the dedicated
> > limited machinery we have right now.
> > 
> > Which would leave us with:
> > - reference (pins today)
> > - share
> > - share-exclusive
> > - exclusive
> > - cleanup
> > 
> > This doesn't quite seem to map onto the heavyweight lock levels in a sensible
> > way...
> 
> Could map it like this:
> 
> AccessShare - pins today
> RowShare - check tuple visibility (BUFFER_LOCK_SHARE today)
> Share - set hint bits
> ShareUpdateExclusive - clean/write out (borrowing Robert's idea)
> Exclusive - add tuples, change xmax, etc. (BUFFER_LOCK_EXCLUSIVE today)
> AccessExclusive - cleanup lock or evict the buffer

I tend think having things like RowShare for buffer locking is confusing
enough to actually make the similarity to the heavyweight locks to not be a
win...


> That has a separate level for hint bits vs. I/O, so multiple backends could
> set hint bits.  I don't know whether the benchmarks would favor maintaining
> that distinction.

I don't think it would - I actually found multiple backends setting the same
hint bits to *hurt* performance a bit.  But what's more important, we don't
have the space for it, I think.  Every lock that can be acquired multiple
times needs a lock count of 18 bits. And we need to store the buffer state
flags (10 bits). There's just not enough space in 64bit to have three 18bit
counters as well as flag bits etc.


> Compared to share-exclusive, I think I'd prefer a name that describes the use
> cases, "set-hints-or-write" (or separate "write" and "set-hints" levels).

I would too, I just couldn't come up with something that conveys the meanings
in a sufficiently concise way :)


> What do you think of that?  I don't know whether that should win vs. names
> like ShareUpdateExclusive, though.

I think it'd be a win compared to the heavyweight lock names...

Greetings,

Andres Freund

  11. Re: Buffer locking is special (hints, checksums, AIO writes)

    Noah Misch <noah@leadboat.com> — 2025-08-27T23:04:51Z

    On Wed, Aug 27, 2025 at 03:29:02PM -0400, Andres Freund wrote:
> On 2025-08-27 12:14:41 -0700, Noah Misch wrote:
> > On Wed, Aug 27, 2025 at 12:18:27PM -0400, Andres Freund wrote:
> > > > On Tue, Aug 26, 2025 at 05:00:13PM -0400, Andres Freund wrote:
> > > > > On 2025-08-26 16:21:36 -0400, Robert Haas wrote:
> > > > > > On Fri, Aug 22, 2025 at 3:45 PM Andres Freund <andres@anarazel.de> wrote:
> > > > > > > DOES ANYBODY HAVE A BETTER NAME THAN SHARE-EXCLUSIVE???!?

> > > Which would leave us with:
> > > - reference (pins today)
> > > - share
> > > - share-exclusive
> > > - exclusive
> > > - cleanup

> > Compared to share-exclusive, I think I'd prefer a name that describes the use
> > cases, "set-hints-or-write" (or separate "write" and "set-hints" levels).

Another name idea is "self-exclusive", to contrast with "exclusive" excluding
all of (exclusive, self-exclusive, share).

Fortunately, not much code will acquire this lock type.  Hence, there's
relatively little damage if the name is less obvious than older lock types or
if the name changes later.

  12. Re: Buffer locking is special (hints, checksums, AIO writes)

    Michael Paquier <michael@paquier.xyz> — 2025-09-01T02:03:55Z

    On Wed, Aug 27, 2025 at 10:03:08AM -0400, Robert Haas wrote:
> If we were to use the existing PostgreSQL naming convention, I think
> I'd probably argue that the nearest parallel to this level is
> ShareUpdateExclusive: a self-exclusive lock level that permits
> ordinary table access to continue while blocking exclusive locks, used
> for an in-flight maintenance operation. But that's arguable, of
> course.

ShareUpdateExclusive is a term that's been used for some time now and
relates to knowledge that's quite spread in the tree, so it feels like
a natural fit for the use-case described on this thread as we'd want a
self-conflicting lock.  share-exclusive did not sound that bad to me,
TBH, quite the contrary, when applied to buffer locking for aio.

"intent" is also a word I've bumped quite a lot into while looking at
some naming convention, but this is more related to the fact that a
lock is going to be taken, which we don't really have.  So that feels
off.
--
Michael

  13. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2025-09-15T23:05:37Z

    Hi,

On 2025-08-22 15:44:48 -0400, Andres Freund wrote:
> The hardest part about this change is that everything kind of depends on each
> other. The changes are large enough that they clearly can't just be committed
> at once, but doing them over time risks [temporary] performance regressions.
> 
> 
> 
> 
> The order of changes I think makes the most sense is the following:
> 
> 1) Allow some modifications while holding the buffer header spinlock
> 
> 2) Reduce buffer pin with just an atomic-sub
> 
>    This needs to happen first, otherwise there are performance regressions
>    during the later steps.

Here are the first few cleaned up patches implementing the above steps, as
well as some cleanups.  I included a commit from another thread, as it
conflicts with these changes, and we really should apply it - and it's
arguably required to make the changes viable, as it removes one more use of
PinBuffer_Locked().

Another change included is to not return the buffer with the spinlock held
from StrategyGetBuffer(), and instead pin the buffer in freelist.c. The reason
for that is to reduce the most common PinBuffer_locked() call. By definition
PinBuffer_locked() will become a bit slower due to 0003. But even without 0003
it 0002 is faster than master. And the previous approach also just seems
pretty unclean.   I don't love that it requires the new TrackNewBufferPin(),
but I don't really have a better idea.

I invite particular attention to the commit message for 0003 as well as the
comment changes in buf_internals.h within.

I'm not sure I like the TODOs added in 0003 and removed in 0004, but squashing
the changes doesn't really seem better either.

Greetings,

Andres Freund

  14. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2025-09-22T22:14:12Z

    Hi,

On 2025-09-15 19:05:37 -0400, Andres Freund wrote:
> On 2025-08-22 15:44:48 -0400, Andres Freund wrote:
> > The hardest part about this change is that everything kind of depends on each
> > other. The changes are large enough that they clearly can't just be committed
> > at once, but doing them over time risks [temporary] performance regressions.
> >
> >
> >
> >
> > The order of changes I think makes the most sense is the following:
> >
> > 1) Allow some modifications while holding the buffer header spinlock
> >
> > 2) Reduce buffer pin with just an atomic-sub
> >
> >    This needs to happen first, otherwise there are performance regressions
> >    during the later steps.
>
> Here are the first few cleaned up patches implementing the above steps, as
> well as some cleanups.  I included a commit from another thread, as it
> conflicts with these changes, and we really should apply it - and it's
> arguably required to make the changes viable, as it removes one more use of
> PinBuffer_Locked().
>
> Another change included is to not return the buffer with the spinlock held
> from StrategyGetBuffer(), and instead pin the buffer in freelist.c. The reason
> for that is to reduce the most common PinBuffer_locked() call. By definition
> PinBuffer_locked() will become a bit slower due to 0003. But even without 0003
> it 0002 is faster than master. And the previous approach also just seems
> pretty unclean.   I don't love that it requires the new TrackNewBufferPin(),
> but I don't really have a better idea.
>
> I invite particular attention to the commit message for 0003 as well as the
> comment changes in buf_internals.h within.

Robert looked at the patches while we were chatting, and I addressed his
feedback in this new version.

Changes:

- Updated patch description for 0002, giving a lot more background

- Improved BufferDesc comments a fair bit more in 0003

- Reduced the size of 0003 a bit, by using UnlockBufHdrExt() instead of a CAS
  loop in buffer_stage_common() and reordering some things in
  TerminateBufferIO()

- Made 0004 only use the non-looping atomic op in UnpinBufferNoOwner(), not
  MarkBufferDirty(). I realized the latter would take additional complexity to
  make safe (a CAS loop in TerminateBufferIO()).  I am somewhat doubtful that
  there are workloads where it matters...


Greetings,

Andres Freund

  15. Re: Buffer locking is special (hints, checksums, AIO writes)

    Matthias van de Meent <boekewurm+postgres@gmail.com> — 2025-10-04T07:05:45Z

    On Tue, 23 Sept 2025 at 00:14, Andres Freund <andres@anarazel.de> wrote:
> On 2025-09-15 19:05:37 -0400, Andres Freund wrote:
> > Here are the first few cleaned up patches implementing the above steps, as
> > well as some cleanups.  I included a commit from another thread, as it
> > conflicts with these changes, and we really should apply it - and it's
> > arguably required to make the changes viable, as it removes one more use of
> > PinBuffer_Locked().
> >
> > Another change included is to not return the buffer with the spinlock held
> > from StrategyGetBuffer(), and instead pin the buffer in freelist.c. The reason
> > for that is to reduce the most common PinBuffer_locked() call. By definition
> > PinBuffer_locked() will become a bit slower due to 0003. But even without 0003
> > it 0002 is faster than master. And the previous approach also just seems
> > pretty unclean.   I don't love that it requires the new TrackNewBufferPin(),
> > but I don't really have a better idea.
> >
> > I invite particular attention to the commit message for 0003 as well as the
> > comment changes in buf_internals.h within.
>
> Robert looked at the patches while we were chatting, and I addressed his
> feedback in this new version.

I like these changes, and have some minor comments:

0001 ensures that ReadRecentBuffer increments the usage counter, which
someone who uses an access strategy may want to prevent. I know this
isn't exactly new behaviour, but something I noticed anyway. Apart
from that observation, LGTM

0002 has a FIXME in a comment in GetVictimBuffer. Assuming it's about
the comment itself needing updates, how about:

+     * Ensure, before we pin a victim buffer, that there's a free refcount
+     * entry, and a resource owner slot for the pin.

Again, LGTM.

0003's UnlockBufHdrExt:
This is implemented with CAS, even when we only want to change bits we
know the state of (or could know, if we spent the effort).
Given its inline nature, wouldn't it be better to use atomic_sub
instructions? Or is this to handle cases where the bits we want to
(un)set might be (un)set by a concurrent process?
If the latter, could we specialize this to do a single atomic_sub
whenever we want to change state bits that we know can be only changed
whilst holding the spinlock?

0004: LGTM

0005: LGTM

0006: LGTM

Kind regards,

Matthias van de Meent

  16. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2025-10-06T22:55:23Z

    Hi,

On 2025-10-04 09:05:45 +0200, Matthias van de Meent wrote:
> On Tue, 23 Sept 2025 at 00:14, Andres Freund <andres@anarazel.de> wrote:
> > On 2025-09-15 19:05:37 -0400, Andres Freund wrote:
> > > Here are the first few cleaned up patches implementing the above steps, as
> > > well as some cleanups.  I included a commit from another thread, as it
> > > conflicts with these changes, and we really should apply it - and it's
> > > arguably required to make the changes viable, as it removes one more use of
> > > PinBuffer_Locked().
> > >
> > > Another change included is to not return the buffer with the spinlock held
> > > from StrategyGetBuffer(), and instead pin the buffer in freelist.c. The reason
> > > for that is to reduce the most common PinBuffer_locked() call. By definition
> > > PinBuffer_locked() will become a bit slower due to 0003. But even without 0003
> > > it 0002 is faster than master. And the previous approach also just seems
> > > pretty unclean.   I don't love that it requires the new TrackNewBufferPin(),
> > > but I don't really have a better idea.
> > >
> > > I invite particular attention to the commit message for 0003 as well as the
> > > comment changes in buf_internals.h within.
> >
> > Robert looked at the patches while we were chatting, and I addressed his
> > feedback in this new version.
> 
> I like these changes, and have some minor comments:

Thank for reviewing!


> 0001 ensures that ReadRecentBuffer increments the usage counter, which
> someone who uses an access strategy may want to prevent. I know this
> isn't exactly new behaviour, but something I noticed anyway. Apart
> from that observation, LGTM

Are you proposing to change behaviour? Right now ReadRecentBuffer doesn't even
accept a strategy, so I don't really see this as something that needs to be
tackled at this point.

I'm not sure I see any real use cases for ReadRecentBuffer() that would
benefit from a strategy, but I very well might just not be thinking wide
enough.


> 0002 has a FIXME in a comment in GetVictimBuffer. Assuming it's about
> the comment itself needing updates

Indeed.


> , how about:
> 
> +     * Ensure, before we pin a victim buffer, that there's a free refcount
> +     * entry, and a resource owner slot for the pin.
> 
> Again, LGTM.

WFM.


> 0003's UnlockBufHdrExt:
> This is implemented with CAS, even when we only want to change bits we
> know the state of (or could know, if we spent the effort).
> Given its inline nature, wouldn't it be better to use atomic_sub
> instructions? Or is this to handle cases where the bits we want to
> (un)set might be (un)set by a concurrent process?

Yes, it's to handle concurrent changes to the buffer state.

> If the latter, could we specialize this to do a single atomic_sub
> whenever we want to change state bits that we know can be only changed
> whilst holding the spinlock?

We probably could optimize some cases as an atomic-sub, some others as an
atomic-and and others again as an atomic-or. The latter to however are
implemented as a CAS on x86 anyway...

After 0004 I don't think any of the paths using this are actually particularly
hot, so I'm somewhat doubtful it's worth to try to optimize this too much. If
there are hot paths, we really should try to work towards not even needing the
buffer header spinlock, that has a bigger impact that improving the code for
unlocking the buffer header...

Greetings,

Andres Freund

  17. Re: Buffer locking is special (hints, checksums, AIO writes)

    Matthias van de Meent <boekewurm+postgres@gmail.com> — 2025-10-07T16:40:48Z

    Hi,

On Tue, 7 Oct 2025 at 00:55, Andres Freund <andres@anarazel.de> wrote:
> On 2025-10-04 09:05:45 +0200, Matthias van de Meent wrote:
> > 0001 ensures that ReadRecentBuffer increments the usage counter, which
> > someone who uses an access strategy may want to prevent. I know this
> > isn't exactly new behaviour, but something I noticed anyway. Apart
> > from that observation, LGTM
>
> Are you proposing to change behaviour?

Eventually, yes, but not necessarily now or with this patchset.

> Right now ReadRecentBuffer doesn't even
> accept a strategy, so I don't really see this as something that needs to be
> tackled at this point.
>
> I'm not sure I see any real use cases for ReadRecentBuffer() that would
> benefit from a strategy, but I very well might just not be thinking wide
> enough.

I think it's rather strange that there is no Extended variant of
ReadRecentBuffer, like how there is a ReadBufferExtended for
ReadBuffer. Yes, ReadRecentBuffer has more arguments to fill and so
has a smaller difference versus ReadBufferExtended, but it's no
complete replacement for ReadBuffer[Ext] when you're aware of a recent
buffer of the page.

I'm not saying it will definitely happen, but I could see that e.g.
amcheck might want to keep track of buffer IDs of recent heap pages it
accessed to verify index's results, without holding a pin on all the
pages; and instead using ReadRecentBuffer[Extended] with a
BufferAccessStrategy to allow re-acquiring the buffer pin without
blowing out shared buffers or making parts of the pool take forever to
evict again.

> > 0003's UnlockBufHdrExt:
> > This is implemented with CAS, even when we only want to change bits we
> > know the state of (or could know, if we spent the effort).
> > Given its inline nature, wouldn't it be better to use atomic_sub
> > instructions? Or is this to handle cases where the bits we want to
> > (un)set might be (un)set by a concurrent process?
>
> Yes, it's to handle concurrent changes to the buffer state.
>
> > If the latter, could we specialize this to do a single atomic_sub
> > whenever we want to change state bits that we know can be only changed
> > whilst holding the spinlock?
>
> We probably could optimize some cases as an atomic-sub, some others as an
> atomic-and and others again as an atomic-or. The latter to however are
> implemented as a CAS on x86 anyway...
>
> After 0004 I don't think any of the paths using this are actually particularly
> hot, so I'm somewhat doubtful it's worth to try to optimize this too much. If
> there are hot paths, we really should try to work towards not even needing the
> buffer header spinlock, that has a bigger impact that improving the code for
> unlocking the buffer header...

Fair enough; I guess we'll see if further optimization would have much
impact once this all has been committed.

Kind regards,

Matthias van de Meent
Databricks

  18. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2025-10-09T20:35:44Z

    Hi,

I pushed a few commits from this patchset after Matthias' review
(thanks!). Unfortunately in 5e899859287 I missed that the valgrind annotations
would not be done anymore for the buffers returned by
StrategyGetBuffer(). Which turned skink red.

The attached 0001 patch centralizes the valgrind initialization in
TrackNewBufferPin(), which 5e899859287 had added. The nice side effect of that
is that there are fewer VALGRIND_MAKE_MEM_DEFINED() calls than before. The
naming isn't the perfect match, but it seems fine to me.


0002 is a new version of "Allow some buffer state modifications while holding
header lock", mainly with a fair bit of comment polishing around BufferDesc
and one small oversight fixed (didn't update a buffer_state variable in one
place).

Greetings,

Andres Freund

  19. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2025-10-09T21:16:49Z

    On 2025-10-09 16:35:44 -0400, Andres Freund wrote:
> I pushed a few commits from this patchset after Matthias' review
> (thanks!). Unfortunately in 5e899859287 I missed that the valgrind annotations
> would not be done anymore for the buffers returned by
> StrategyGetBuffer(). Which turned skink red.
> 
> The attached 0001 patch centralizes the valgrind initialization in
> TrackNewBufferPin(), which 5e899859287 had added. The nice side effect of that
> is that there are fewer VALGRIND_MAKE_MEM_DEFINED() calls than before. The
> naming isn't the perfect match, but it seems fine to me.

Forgot to say: I'll push this patch soon, to get skink back to green. Unless
somebody says something.  We can adjust this later, if the comment and/or
placement of VALGRIND_MAKE_MEM_DEFINED() isn't to everyones liking.

  20. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2025-11-20T02:47:49Z

    Hi,

On 2025-10-09 17:16:49 -0400, Andres Freund wrote:
> On 2025-10-09 16:35:44 -0400, Andres Freund wrote:
> > I pushed a few commits from this patchset after Matthias' review
> > (thanks!). Unfortunately in 5e899859287 I missed that the valgrind annotations
> > would not be done anymore for the buffers returned by
> > StrategyGetBuffer(). Which turned skink red.
> > 
> > The attached 0001 patch centralizes the valgrind initialization in
> > TrackNewBufferPin(), which 5e899859287 had added. The nice side effect of that
> > is that there are fewer VALGRIND_MAKE_MEM_DEFINED() calls than before. The
> > naming isn't the perfect match, but it seems fine to me.
> 
> Forgot to say: I'll push this patch soon, to get skink back to green. Unless
> somebody says something.  We can adjust this later, if the comment and/or
> placement of VALGRIND_MAKE_MEM_DEFINED() isn't to everyones liking.

I have pushed that fix as well as the subsequent buffer header locking changes
a while ago.


Attached is a patchset that actually implements the buffer content locks in
bufmgr.c. This isn't that close to a committable shape yet, but it seemed
useful to get it out there.  The first few patches seem closer, so it'll also
be useful to narrow this down.

0001: A straight-up bugfix in lwlock.c - albeit for a bug that seems currently
	effectively harmless.

0002: Not really required, but seems like an improvement to me

0003: A prerequisite to 0004, pretty boring itself

0004: Use 64bit atomics for BufferDesc.state - at this point nothing uses the
additional bits yet, though.  Some annoying reformatting required to avoid
long lines.

0005: There already was a wait event class for BUFFERPIN. It seems better to
make that more general than to implement them separately.

0006+0007: This is preparatory work for 0008, but also worthwhile on its
own. The private refcount stuff does show up in profiles. The reason it's
related is that without these changes the added information in 0008 makes that
worse.

0008: The main change. Implements buffer content locking independently from
lwlock.c. There's obviously a lot of similarity between lwlock.c code and
this, but I've not found a good way to reduce the duplication without giving
up too much.  This patch does immediately introduce share-exclusive as a new
lock level, mostly because it was too painful to do separately.

0009+0010+0011: Preparatory work for 0012.

0012: One of the main goals of this patchset - use the new share-exclusive
lock level to only allow hint bits to be set while no IO is going on.

0013: Prototype of making UnlockReleaseBuffer() faster and of using it more
widely in nbtree.c

0014: Now that hint bits can't be done while IO is going on, we don't need to
copy pages anymore.  This needs a fair bit more work, as denoted by the FIXMEs
in the code.

I've tried to add detail to the more important commit messages, at least until
0012.


I want to again emphasize that the important commits (i.e. 0008, 0012, 0014)
aren't close to being mergeable. But I think they're in a stage that they
could benefit from "lenient" high-level review.

Greetings,

Andres Freund

  21. Re: Buffer locking is special (hints, checksums, AIO writes)

    Greg Burd <greg@burd.me> — 2025-11-20T19:08:57Z

    On Nov 19 2025, at 9:47 pm, Andres Freund <andres@anarazel.de> wrote:

> Hi,
> 
> On 2025-10-09 17:16:49 -0400, Andres Freund wrote:
>> On 2025-10-09 16:35:44 -0400, Andres Freund wrote:
>> > I pushed a few commits from this patchset after Matthias' review
>> > (thanks!). Unfortunately in 5e899859287 I missed that the valgrind annotations
>> > would not be done anymore for the buffers returned by
>> > StrategyGetBuffer(). Which turned skink red.
>> > 
>> > The attached 0001 patch centralizes the valgrind initialization in
>> > TrackNewBufferPin(), which 5e899859287 had added. The nice side
>> effect of that
>> > is that there are fewer VALGRIND_MAKE_MEM_DEFINED() calls than
>> before. The
>> > naming isn't the perfect match, but it seems fine to me.
>> 
>> Forgot to say: I'll push this patch soon, to get skink back to green. Unless
>> somebody says something.  We can adjust this later, if the comment and/or
>> placement of VALGRIND_MAKE_MEM_DEFINED() isn't to everyones liking.
> 
> I have pushed that fix as well as the subsequent buffer header locking changes
> a while ago.

Hello Andres,

After talking to you about these ideas at PGConf in NYC I've been
anxiously awaiting this patch set. Thanks for dedicating the energy and
time to get it to this stage.

High level feedback after reading the patches/email/commit messages is
that it looks to get you to where you wanted to be, unblocking AIO
writes. I think they'll end up being faster than what's in place now,
even before you get to the AIO piece.  Certainly removing the copy of
each page to do a checksum will help.  Opening the door to a future
where we can have super-pinned/locked pages is also a net win.

Everything before/after 0008 was rather easy to digest and understand
and I found nothing really to call out at this stage

0008 is understandable too, it's just sizable. While it is large, I find
it well laid out and more readable than before.  I gave the locking code
a good look, it seems correct AFAICT.

Keep going, I'll be happy to dedicate time to testing and digging into
the commits as you get this into a final state.  I look forward to
extending/enhancing this code once integrated.

> I want to again emphasize that the important commits (i.e. 0008, 0012, 0014)
> aren't close to being mergeable. But I think they're in a stage that they
> could benefit from "lenient" high-level review.
> 
> Greetings,
> 
> Andres Freund

cheers.

-greg

  22. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2025-11-20T20:51:33Z

    Hi,

On 2025-11-20 14:08:57 -0500, Greg Burd wrote:
> After talking to you about these ideas at PGConf in NYC I've been
> anxiously awaiting this patch set. Thanks for dedicating the energy and
> time to get it to this stage.

Thanks!


> High level feedback after reading the patches/email/commit messages is
> that it looks to get you to where you wanted to be, unblocking AIO
> writes. I think they'll end up being faster than what's in place now,
> even before you get to the AIO piece.  Certainly removing the copy of
> each page to do a checksum will help.  Opening the door to a future
> where we can have super-pinned/locked pages is also a net win.

I actually had planned to write about the performance effects, in particular
of 0012, a bit more:

It's worth pointing out that the new way of setting hint bits is inherently
more expensive than what we did before - upgrading a lock to a different lock
level isn't free, compared to doing, well, nothing.

For paths that set the hint bits of a whole page, like a seqscan, that cost is
more than amortized by the batched approach introduced in 0011. Those get
faster with the patch, both when already hinted and when not.

However, there are paths that aren't easily amenable to that approach, like
e.g. an ordered index scan referencing unhinted tuples. There we only ever
access a single tuple and release the upgraded lock after every tuple. If the
index scan is perfectly correlated with the table and every tuple is unhinted,
that's a decent amount of additional work.

I've spent a lot of time micro-optimizing that workload, to avoid any
significiant regressions. An extreme stress-test started out being about 20%
slower than today, as of my current local version, it's a bit faster (~1%) on
one of my machines and a bit slower (~2%) on another. Partially that was
achieved by optimizing the hint-bit-lock-upgrade code more (e.g. having a fast
path for updating a single hint bit, avoiding redundant reads of the lock
state by having MarkSharedBufferDirtyHint(), ...), partially by optimizing the
locking code.  The latter is a bit of a cheat though - things would be even
faster if we went with the old way of setting hint bits, but with the
independent optimizations applied.

I think that's ok though:

1) the old way of setting hint bits is a pretty dirty hack that causes issues
   in quite a few places.

2) by definition, having to set hint bits is an ephemeral state, once the hint
   bits are set, the difference vanishes

3) no normal workload shows the difference - my stress test does
   SELECT * FROM manyrows_idx ORDER BY i OFFSET 10000000;
   on a perfectly correlated table with very narrow rows, i.e. an index scan
   of the whole table, where none of the scan results are ever used. Once one
   actually uses the resulting rows, the performance difference completely
   vanishes.

4) as part of the index prefetching work, we might get the infrastructure to
   actually batch the hint-bit setting in this case too.


I see some mild performance gains in workloads like pgbench [-S], but nothing
to write home about. Which I think is about what I would expect - there's a
minor efficiency gain due to the private refcount changes and not having state
tracking for two error recovery mechanisms (lwlocks' and private refcount).

Non-all-visible seqscans do see some performance gain due to 0011, whether the
table is hinted or not. But it's again something that's mostly noticeable in
microbenchmarks, as e.g. tuple deforming or qual evaluation has a much bigger
impact.


> Everything before/after 0008 was rather easy to digest and understand
> and I found nothing really to call out at this stage
>
> 0008 is understandable too, it's just sizable. While it is large, I find
> it well laid out and more readable than before.  I gave the locking code
> a good look, it seems correct AFAICT.

I hope so :), the locking logic it's largely the same as lwlock.c, with some
exceptions due to the added lock level and differences in error handling
state.

I don't really see a good way to split 0008 unfortunately...  I previously had
split 0012 into four patches (core changes, heapam changes, the rest of the
adaptions to setting hint bits in various places, adding assertions relying on
the different lock levels), but I found it pretty unwieldly from a "comment
management" perspective, because comments that need to be rewritten are
temporarily wrong, or would need to be modified in yet another path.  But I'm
open to going back to that approach.

I guess I could pull out the addition of UnlockBuffer() and the "redirection"
to it from LockBuffer() into a separate patch.


> Keep going, I'll be happy to dedicate time to testing and digging into
> the commits as you get this into a final state.  I look forward to
> extending/enhancing this code once integrated.

Cool!

I think 0001, 0002, 0003, 0005 and 0009 should be mergeable pretty soon.  I've
some further polishing to do for 0006 and 0007, but I think they could go in
well ahead of the rest.

For 0008, in addition to what's noted in the commit message, I think there
needs to be an additional section in src/backend/storage/buffer/README (or
such), explaining that buffer content locks used to be lwlocks but aren't
anymore for xyz reasons.  I suspect it'd also be good to have a few references
from lwlock.c to bufmgr.c to make sure the code is co-evolved.

For 0012, I think it might make sense to pull out some of the changes to
fsm_vacuum_page() out into a separate commit. Basically changing the code to
acquire a lock on the page - I still can't quite believe that somebody thought
it's sane to update the page without even bothering with a share lock.

0014 needs a lot more polishing, there's references to the hint bit / locking
interactions all over. Pretty hard to find all the references :(.

Greetings,

Andres Freund

  23. Re: Buffer locking is special (hints, checksums, AIO writes)

    Melanie Plageman <melanieplageman@gmail.com> — 2025-11-21T17:52:38Z

    This email is just a review for some (specified below) of the patches 0001-0007

On Wed, Nov 19, 2025 at 9:47 PM Andres Freund <andres@anarazel.de> wrote:
>
> 0002: Not really required, but seems like an improvement to me

The commit message says the point is to get compiler warnings for
switch cases that should be exhaustive, but as soon as I looked for a
switch case like that, I see BufferIsLockedByMeInMode()

        switch (mode)
        {
            case BUFFER_LOCK_EXCLUSIVE:
                lw_mode = LW_EXCLUSIVE;
                break;
            case BUFFER_LOCK_SHARE:
                lw_mode = LW_SHARED;
                break;
            default:
                pg_unreachable();
        }

Which makes it impossible to get such a warning. When I add a lock
mode, it specifically doesn't warn when compiling.
However, I'm a big fan of using enums instead of macros when
appropriate, so I have no issue with this change. I just think the
commit message is a bit confusing.

> 0003: A prerequisite to 0004, pretty boring itself

LGTM.

> 0004: Use 64bit atomics for BufferDesc.state - at this point nothing uses the
> additional bits yet, though.  Some annoying reformatting required to avoid
> long lines.

I noticed that the BUF_STATE_GET_REFCOUNT and BUF_STATE_GET_USAGECOUNT
macros cast the return value to a uint32. We won't use the extra bits
but we did bother to keep the macro result sized to the field width
before so keeping it uint32 is probably more confusing now that state
is 64 bit.

Not related to this patch, but I noticed GetBufferDescriptor() calls
for a uint32 and all the callers pretty much pass a signed int —
wonder why it calls for uint32.

> 0005: There already was a wait event class for BUFFERPIN. It seems better to
> make that more general than to implement them separately.

I reviewed and see no issues with the code, but I don't have an
opinion on this wait event naming so maybe you better _wait_ for some
other review ;)

> 0006+0007: This is preparatory work for 0008, but also worthwhile on its
> own. The private refcount stuff does show up in profiles. The reason it's
> related is that without these changes the added information in 0008 makes that
> worse.

I found it slightly confusing that this commit appears to
unnecessarily add the PrivateRefCountData struct (given that it
doesn't need it to do the new parallel array thing). You could wait
until you need it in 0008, but 0008 is big as it is, so it probably is
fine where it is.

in InitBufferManagerAccess(), why do you have

memset(&PrivateRefCountArrayKeys, 0, sizeof(Buffer));
seems like it should be
memset(PrivateRefCountArrayKeys, 0, sizeof(PrivateRefCountArrayKeys));

I wonder how easy it will be to keep the Buffer in sync between
PrivateRefCountArrayKeys and the PrivateRefCountEntry — would a helper
function help?

ForgetPrivateRefCountEntry doesn’t clear the data member — but maybe
it doesn’t matter...

in ReservePrivateRefCountEntry() there is a superfluous clear

memset(&victim_entry->data, 0, sizeof(victim_entry->data));
victim_entry->data.refcount = 0;

0007
needs a commit message. overall seems fine though.
You should probably capitalize the "c" of "count" in
PrivateRefcountEntryLast to be consistent with the other names.

- Melanie

  24. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2025-11-24T20:57:59Z

    Hi,

On 2025-11-19 21:47:49 -0500, Andres Freund wrote:
> 0001: A straight-up bugfix in lwlock.c - albeit for a bug that seems currently
> 	effectively harmless.

Does anybody have opinions about whether to backpatch this fix? Given that it
has no real consequences I'm mildly inclined not to, but maybe there are cases
where the additional wait list lock cycle matters?

Greetings,

Andres Freund

  25. Re: Buffer locking is special (hints, checksums, AIO writes)

    Melanie Plageman <melanieplageman@gmail.com> — 2025-11-24T21:04:41Z

    On Mon, Nov 24, 2025 at 3:58 PM Andres Freund <andres@anarazel.de> wrote:
>
> Hi,
>
> On 2025-11-19 21:47:49 -0500, Andres Freund wrote:
> > 0001: A straight-up bugfix in lwlock.c - albeit for a bug that seems currently
> >       effectively harmless.
>
> Does anybody have opinions about whether to backpatch this fix? Given that it
> has no real consequences I'm mildly inclined not to, but maybe there are cases
> where the additional wait list lock cycle matters?

Since it is a mistake, I am mildly in favor of backporting to avoid
confusion for future developers. It's pretty weird that LWLockWakeup()
has to be called again to actually unset LW_FLAG_HAS_WAITERS. But
since it's not really harmful, this is a very mild opinion.

- Melanie

  26. Re: Buffer locking is special (hints, checksums, AIO writes)

    Thomas Munro <thomas.munro@gmail.com> — 2025-11-25T00:09:38Z

    On Fri, Nov 21, 2025 at 9:51 AM Andres Freund <andres@anarazel.de> wrote:
> It's worth pointing out that the new way of setting hint bits is inherently
> more expensive than what we did before - upgrading a lock to a different lock
> level isn't free, compared to doing, well, nothing.
>
> For paths that set the hint bits of a whole page, like a seqscan, that cost is
> more than amortized by the batched approach introduced in 0011. Those get
> faster with the patch, both when already hinted and when not.

Nice work!

> However, there are paths that aren't easily amenable to that approach, like
> e.g. an ordered index scan referencing unhinted tuples. There we only ever
> access a single tuple and release the upgraded lock after every tuple. If the
> index scan is perfectly correlated with the table and every tuple is unhinted,
> that's a decent amount of additional work.

Yeah, but it was only faster because it was cheating.  It presumably
doesn't happen when you bulk load and then create index.  It
presumably does happen when you insert a lot of data in order, on
first correlated index scan.  Seems like an inherent limitation of the
current tuple-at-a-time architecture when combined with the *required*
interlocking, and not a blocker for this work.

+ Some filesystems, raid implementations, ... do not tolerate the data being

I was aware of BTRFS (EIO on read) and ZFS 2.4 (EIO on read or write
depending on configuration option), but hadn't thought about RAID.
Ugh, right, non-matching RAID1 mirrors (and I guess also b0rked RAID5
parity bits?).  Fun.

https://bugzilla.kernel.org/show_bug.cgi?id=99171

> I've spent a lot of time micro-optimizing that workload, to avoid any
> significiant regressions. An extreme stress-test started out being about 20%
> slower than today, as of my current local version, it's a bit faster (~1%) on
> one of my machines and a bit slower (~2%) on another. Partially that was
> achieved by optimizing the hint-bit-lock-upgrade code more (e.g. having a fast
> path for updating a single hint bit, avoiding redundant reads of the lock
> state by having MarkSharedBufferDirtyHint(), ...), partially by optimizing the
> locking code.  The latter is a bit of a cheat though - things would be even
> faster if we went with the old way of setting hint bits, but with the
> independent optimizations applied.
>
> I think that's ok though:
>
> 1) the old way of setting hint bits is a pretty dirty hack that causes issues
>    in quite a few places.
>
> 2) by definition, having to set hint bits is an ephemeral state, once the hint
>    bits are set, the difference vanishes
>
> 3) no normal workload shows the difference - my stress test does
>    SELECT * FROM manyrows_idx ORDER BY i OFFSET 10000000;
>    on a perfectly correlated table with very narrow rows, i.e. an index scan
>    of the whole table, where none of the scan results are ever used. Once one
>    actually uses the resulting rows, the performance difference completely
>    vanishes.
>
> 4) as part of the index prefetching work, we might get the infrastructure to
>    actually batch the hint-bit setting in this case too.

Yeah.  Was just thinking the same.  Both the streaming and batching
projects have opportunities to figure out an amortisation scheme.  I
have a few vague ideas about stream-based approaches already, hmm...

+1, I think this is OK for now.

  27. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2025-11-25T00:17:19Z

    Hi,

On 2025-11-24 16:04:41 -0500, Melanie Plageman wrote:
> On Mon, Nov 24, 2025 at 3:58 PM Andres Freund <andres@anarazel.de> wrote:
> > On 2025-11-19 21:47:49 -0500, Andres Freund wrote:
> > > 0001: A straight-up bugfix in lwlock.c - albeit for a bug that seems currently
> > >       effectively harmless.
> >
> > Does anybody have opinions about whether to backpatch this fix? Given that it
> > has no real consequences I'm mildly inclined not to, but maybe there are cases
> > where the additional wait list lock cycle matters?
> 
> Since it is a mistake, I am mildly in favor of backporting to avoid
> confusion for future developers. It's pretty weird that LWLockWakeup()
> has to be called again to actually unset LW_FLAG_HAS_WAITERS. But
> since it's not really harmful, this is a very mild opinion.

Thanks for chiming in, done.

Greetings,

Andres Freund

  28. Re: Buffer locking is special (hints, checksums, AIO writes)

    Melanie Plageman <melanieplageman@gmail.com> — 2025-11-25T15:44:31Z

    On Wed, Nov 19, 2025 at 9:47 PM Andres Freund <andres@anarazel.de> wrote:
>
> 0008: The main change. Implements buffer content locking independently from
> lwlock.c. There's obviously a lot of similarity between lwlock.c code and
> this, but I've not found a good way to reduce the duplication without giving
> up too much.  This patch does immediately introduce share-exclusive as a new
> lock level, mostly because it was too painful to do separately.
>
> 0009+0010+0011: Preparatory work for 0012.
>
> 0012: One of the main goals of this patchset - use the new share-exclusive
> lock level to only allow hint bits to be set while no IO is going on.

Below is a review of 0008-0012
I skipped 0013 and 0014 after seeing "#if 1" in 0013 :)

0008:
-------
> [PATCH v6 08/14] bufmgr: Implement buffer content locks independently
 of lwlocks
...
> This commit unfortunately introduces some code that is very similar to the
> code in lwlock.c, however the code is not equivalent enough to easily merge
> it. The future wins that this commit makes possible seem worth the cost.

It is a truly unfortunate amount of duplication. I tried some
refactoring myself just to convince myself it wasn't a good idea.
However, ISTM there is no reason for the lwlock and buffer locking
implementations to have to stay in sync. So they can diverge as
features are added and perhaps the duplication won't be as conspicuous
in the future.

> As of this commit nothing uses the new share-exclusive lock mode. It will be
>documented and used in a future commit. It seemed too complicated to introduce
> the lock-level in a separate commit.

I would have liked this mentioned earlier in the commit message. Also,
I don't know how I feel about it being "documented" in a future
commit...perhaps just don't say that.

diff --git a/src/include/storage/buf_internals.h
b/src/include/storage/buf_internals.h
index 28519ad2813..0a145d95024 100644
--- a/src/include/storage/buf_internals.h
+++ b/src/include/storage/buf_internals.h
@@ -32,22 +33,29 @@
 /*
  * Buffer state is a single 64-bit variable where following data is combined.
  *
+ * State of the buffer itself:
  * - 18 bits refcount
  * - 4 bits usage count
  * - 10 bits of flags
  *
+ * State of the content lock:
+ * - 1 bit has_waiter
+ * - 1 bit release_ok
+ * - 1 bit lock state locked

Somewhere you should clearly explain the scenarios in which you still
need to take the buffer header lock with LockBufHdr() vs when you can
just use atomic operations/CAS loops.

Separately, you should clearly explain what BM_LOCKED (I presume what
"lock state locked" refers to) protects.

+/*
+ * Definitions related to buffer content locks
+ */
+#define BM_LOCK_HAS_WAITERS         (UINT64CONST(1) << 63)
+#define BM_LOCK_RELEASE_OK          (UINT64CONST(1) << 62)
+
+#define BM_LOCK_VAL_SHARED          (UINT64CONST(1) << 32)
+#define BM_LOCK_VAL_SHARE_EXCLUSIVE (UINT64CONST(1) << (32 +
MAX_BACKENDS_BITS))
+#define BM_LOCK_VAL_EXCLUSIVE       (UINT64CONST(1) << (32 + 1 +
MAX_BACKENDS_BITS))
+
+#define BM_LOCK_MASK                (((uint64)MAX_BACKENDS << 32) |
BM_LOCK_VAL_SHARE_EXCLUSIVE | BM_LOCK_VAL_EXCLUSIVE)

Not the fault of this patch, but I always found it confusing that the
BM flags were for buffer manager. I always think BM stands for buffer
mapping.

On a more actionable note: you probably want to update the comment in
procnumber.h in your earlier patch to take out the part about how the
limitation could be lifted using a 64 bit state -- since you made the
state 64 bit and didn't lift the limitation (see below for the
specific comment).

/*
 * Note: MAX_BACKENDS_BITS is 18 as that is the space available for buffer
 * refcounts in buf_internals.h.  This limitation could be lifted by using a
 * 64bit state; but it's unlikely to be worthwhile as 2^18-1 backends exceed
 * currently realistic configurations. Even if that limitation were removed,
 * we still could not a) exceed 2^23-1 because inval.c stores the ProcNumber
 * as a 3-byte signed integer, b) INT_MAX/4 because some places compute
 * 4*MaxBackends without any overflow check.  We check that the configured
 * number of backends does not exceed MAX_BACKENDS in InitializeMaxBackends().
 */

@@ -268,6 +287,15 @@ BufMappingPartitionLockByIndex(uint32 index)
  * wait_backend_pgprocno and setting flag bit BM_PIN_COUNT_WAITER.  At present,
  * there can be only one such waiter per buffer.
  *
+ * The content of buffers is protected via the buffer content lock,
+ * implemented as part buffer state. Note that the buffer header lock is *not*
+ * used to control access to the data in the buffer! We used to use an LWLock
+ * to implement the content lock, but having a dedicated implementation of
+ * content locks allows to implement some otherwise hard things (e.g.
+ * race-freely checking if AIO is in progress before locking a buffer
+ * exclusively) and makes otherwise impossible optimizations possible
+ * (e.g. unlocking and unpinning a buffer in one atomic operation).
+ *

I don't really like that this includes a description of how it used to
work. It makes the description more confusing when trying to
understand the current state. And comments like that make most sense
when the current state is confusing because of a long annoying
history.

@@ -302,7 +303,24 @@ extern void BufferGetTag(Buffer buffer,
RelFileLocator *rlocator,
 extern void MarkBufferDirtyHint(Buffer buffer, bool buffer_std);
 extern void UnlockBuffers(void);
-extern void LockBuffer(Buffer buffer, BufferLockMode mode);
+extern void UnlockBuffer(Buffer buffer);
+extern void LockBufferInternal(Buffer buffer, BufferLockMode mode);
+
+/*
+ * Handling BUFFER_LOCK_UNLOCK in bufmgr.c leads to sufficiently worse branch
+ * prediction to impact performance. Therefore handle that switch here, were
+ * most of the time `mode` will be a constant and thus can be optimized out by
+ * the compiler.

Typo: were -> where

+ */
+static inline void
+LockBuffer(Buffer buffer, BufferLockMode mode)
+{
+    if (mode == BUFFER_LOCK_UNLOCK)
+        UnlockBuffer(buffer);
+    else
+        LockBufferInternal(buffer, mode);
+}
+

Is there a reason to stick with the LockBuffer(buf,
BUFFER_LOCK_UNLOCK) interface here? It seems like a cgood time to
start using UnlockBuffer() which I thought most people find more
intuitive.

 /*
- * Acquire or release the content_lock for the buffer.
+ * Acquire the buffer content lock in the specified mode
+ *
+ * If the lock is not available, sleep until it is.
+ *
+ * Side effect: cancel/die interrupts are held off until lock release.
+ *
+ * This uses almost the same locking approach as lwlock.c's
+ * LWLockAcquire(). See documentation atop of lwlock.c for a more detailed
+ * discussion.
+ *
+ * The reason that this, and most of the other BufferLock* functions, get both
+ * the Buffer and BufferDesc* as parameters, is that looking up one from the
+ * other repeatedly shows up noticeably in profiles.
+ */
+static inline void
+BufferLockAcquire(Buffer buffer, BufferDesc *buf_hdr, BufferLockMode mode)

Either here or over the lock mode enum, I'd spend a few sentences
describing how the buffer lock modes interact -- what conflicts with
what. You spend more time saying how this is like LWLock than
explaining what it actually is.

+/*
+ * Remove ourselves from the waitlist.
+ *
+ * This is used if we queued ourselves because we thought we needed to sleep
+ * but, after further checking, we discovered that we don't actually need to
+ * do so.
+ */
+static void
+BufferLockDequeueSelf(BufferDesc *buf_hdr)
+{
+    bool        on_waitlist;
+
+    LockBufHdr(buf_hdr);
+
...
+    /* XXX: combine with fetch_and above? */
+    UnlockBufHdr(buf_hdr);

I know you just ported this comment from the LWLock implementation but
I can't see how it could be worth the effort. It won't be in the
hottest path as you must satisfy a few conditions to get here.


+ * Wakeup all the lockers that currently have a chance to acquire the lock.
+ */
+static void
+BufferLockWakeup(BufferDesc *buf_hdr, bool unlocked)

This should have a more explanatory comment. You should especially
document the unlocked parameter. I would expect something somewhere in
or above this function that basically says (maybe less verbosely)

- Before waking anything: allow all
- After waking SHARE:
    wake SHARE only
- After waking SHARE_EXCLUSIVE:
    wake SHARE only
    (no more SHARE_EXCLUSIVE)
- After waking EXCLUSIVE:
    wake none

+{
+    bool        new_release_ok;
+    bool        wake_exclusive = unlocked;
+    bool        wake_share_exclusive = true;
+    proclist_head wakeup;
+    proclist_mutable_iter iter;
+
+    proclist_init(&wakeup);
+
+    new_release_ok = true;
+
+    /* lock wait list while collecting backends to wake up */
+    LockBufHdr(buf_hdr);
+
+    proclist_foreach_modify(iter, &buf_hdr->lock_waiters, lwWaitLink)
+    {
+        PGPROC       *waiter = GetPGProcByNumber(iter.cur);
+
+        if (!wake_exclusive && waiter->lwWaitMode == BUFFER_LOCK_EXCLUSIVE)
+            continue;
+
+        if (!wake_share_exclusive && waiter->lwWaitMode ==
BUFFER_LOCK_SHARE_EXCLUSIVE)
+            continue;
+

It seems there is a difference between LWLockWakeup() and
BufferLockWakeup() if the queue contains a share lock waiter followed
by an exclusive lock waiter.

In LWLockWakeup(), it will wake up the share waiter, set
wokeup_somebody to true, and then not wake up the exclusive lock
waiter because wokeup_somebody is true.

In BufferLockWakeup() when unlocked is true, it will wake up the share
lock waiter and then wake up the exclusive lock waiter because
wake_exclusive and wake_share_exclusive are both still true.

This might be the intended behavior, but it is a difference from
LWLockWakeup(), so I think it is worth documenting why it is okay.


+         * Signal that the process isn't on the wait list anymore. This allows
+         * BufferLockDequeueSelf() to remove itself of the waitlist with a
+         * proclist_delete(), rather than having to check if it has been

I know this comment was ported over, but the "remove itself of the
waitlist" -- the "of" is confusing in the original comment and it is
confusing here.

+ * Compute subtraction from buffer state for a release of a held lock in
+ * `mode`.
+ *
+ * This is separated from BufferLockUnlock() as we want to combine the lock
+ * release with other atomic operations when possible, leading to the lock
+ * release being done in multiple places.
+ */
+static inline uint64
+BufferLockReleaseSub(BufferLockMode mode)

I don't understand why this is a separate function even with your comment.

+ * BufferLockHeldByMe - test whether my process holds the content lock in any
+ * mode
+ *
+ * This is meant as debug support only.
+ */
+static bool
+BufferLockHeldByMe(BufferDesc *buf_hdr)
+{
+    PrivateRefCountEntry *entry =
+        GetPrivateRefCountEntry(BufferDescriptorGetBuffer(buf_hdr), false);
+
+    if (!entry)
+        return false;
+    else
+        return entry->data.lockmode != BUFFER_LOCK_UNLOCK;
+}

Previously, if I called LockBuffer(buf, BUFFER_LOCK_SHARE) again after
calling it once, say in RelationCopyStorageUsingBuffer(), I would trip
an assert when unpinning the buffer about how I needed to have
released the lock.

Now, though, it doesn't trip the assert because we don't track
multiple locks. When I call LockBuffer(UNLOCK), it sets lockmode to
BUFFER_LOCK_UNLOCK and BufferLockHeldByMe() only checks the private
refcount entry. Whereas LWLockHeldByMe() checked held_lwlocks which
had multiple entries and would report that I still held a lock. Now,
if I call LockBuffer(SHARE) twice, I'll only find out later when I
have a hang because someone is trying to get an exclusive lock and the
actual BufferDesc->state still has a lock set.

Maybe you should add an assert to the lock acquisition path that the
prevate ref count entry mode is UNLOCK?

 void
-LockBuffer(Buffer buffer, BufferLockMode mode)
+LockBufferInternal(Buffer buffer, BufferLockMode mode)
 {
-    buf = GetBufferDescriptor(buffer - 1);
+    buf_hdr = GetBufferDescriptor(buffer - 1);

-    if (mode == BUFFER_LOCK_UNLOCK)
-        LWLockRelease(BufferDescriptorGetContentLock(buf));
-    else if (mode == BUFFER_LOCK_SHARE)
-        LWLockAcquire(BufferDescriptorGetContentLock(buf), LW_SHARED);
+    if (mode == BUFFER_LOCK_SHARE)
+        BufferLockAcquire(buffer, buf_hdr, BUFFER_LOCK_SHARE);
+    else if (mode == BUFFER_LOCK_SHARE_EXCLUSIVE)
+        BufferLockAcquire(buffer, buf_hdr, BUFFER_LOCK_SHARE_EXCLUSIVE);
     else if (mode == BUFFER_LOCK_EXCLUSIVE)
-        LWLockAcquire(BufferDescriptorGetContentLock(buf), LW_EXCLUSIVE);
+        BufferLockAcquire(buffer, buf_hdr, BUFFER_LOCK_EXCLUSIVE);
     else
         elog(ERROR, "unrecognized buffer lock mode: %d", mode);

I presume you've stuck with this if statement structure because that
is what LockBuffer() used. Even though now the BufferLockMode passed
through to BufferLockAcquire is the exact thing you are testing. It
caught my eye and I had to check multiple times if the mode being
passed in is the same. Basically I found it a bit distracting.


@@ -6625,7 +7295,25 @@ ResOwnerReleaseBufferPin(Datum res)
     if (BufferIsLocal(buffer))
         UnpinLocalBufferNoOwner(buffer);
     else
+    {
+        PrivateRefCountEntry *ref;
+
+        ref = GetPrivateRefCountEntry(buffer, false);
+
+        /*
+         * If the buffer was locked at the time of the resowner release,
+         * release the lock now. This should only happen after errors.
+         */
+        if (ref->data.lockmode != BUFFER_LOCK_UNLOCK)
+        {
+            BufferDesc *buf = GetBufferDescriptor(buffer - 1);
+
+            HOLD_INTERRUPTS();    /* match the upcoming RESUME_INTERRUPTS */
+            BufferLockUnlock(buffer, buf);
+        }
+
         UnpinBufferNoOwner(GetBufferDescriptor(buffer - 1));
+    }
 }

 Bit confusing that ResOwnerReleaseBufferBin() now releases locks as well.

0009 -- I didn't look closely at 0009

0010:
--------
> [PATCH v6 10/14] heapam: Use exclusive lock on old page in CLUSTER

> To be able to guarantee that we can set the hint bit, acquire an exclusive
> lock on the old buffer. We need the hint bits to be set as otherwise
> reform_and_rewrite_tuple() -> rewrite_heap_tuple() -> heap_freeze_tuple() will
> get confused.

So, this is an active bug? And what exactly do you mean
heap_freeze_tuple() gets confused? I thought somewhere in there we
would check the clog.

0011:
--------
> [PATCH v6 11/14] heapam: Add batch mode mvcc check and use it in page mode

> 2) We would like to stop setting hint bits while pages are being written
> out. The necessary locking becomes visible for page mode scans if done for
> every tuple. With batching the overhead can be amortized to only happen
> once per page.

I don't understand the above point. What does this patch have to do
with not setting hint bits while pages are being written out (that
happens in the next patch)? And I presume you mean don't set hint bits
on a buffer that is being flushed by someone else -- but it sounds
like you mean not to set hint bits as part of flushing a buffer.

diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 4b0c49f4bb0..ddabd1a3ec3 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -504,42 +504,93 @@ page_collect_tuples(HeapScanDesc scan, Snapshot snapshot,
                     BlockNumber block, int lines,
                     bool all_visible, bool check_serializable)
 {
+    Oid            relid = RelationGetRelid(scan->rs_base.rs_rd);
+#ifdef BATCHMVCC_FEWER_ARGS
+    BatchMVCCState batchmvcc;
+    HeapTupleData *tuples = batchmvcc.tuples;
+    bool       *visible = batchmvcc.visible;
+#else
+    HeapTupleData tuples[MaxHeapTuplesPerPage];
+    bool        visible[MaxHeapTuplesPerPage];
+#endif
     int            ntup = 0;

It's pretty confusing when visible is an output vs input parameter and
who fills it in when. (i.e. it's filled in in page_collect_tuples() if
check_serializable and all_visible are true, otherwise it's filled in
in HeapTupleSatisifiesMVCCBatch())

Personally, I think I'd almost prefer an all-visible and
not-all-visible version of page_collect_tuples() (or helper functions
containing the loop) that separate this. (I haven't tried it though)

BATCHMVCC_FEWER_ARGS definitely doesn't make it any easier to read --
which I assume you are removing.

+        /*
+         * If the page is not all-visible or we need to check serializability,
+         * maintain enough state to be able to refind the tuple efficiently,
+         * without again needing to extract it from the page.
+         */
+        if (!all_visible || check_serializable)
+        {

"enough state" is pretty vague here. Maybe mention what it wouldn't be
valid to do with the tuples array?

/*
* If the page is all visible, these fields won'otherwise wont be
* populated in loop below.
*/

Some spelling issues with "won'" and "wont"

+ * visibility is set in batchmvcc->visible[]. In addition, ->vistuples_dense
+ * is set to contain the offsets of visible tuples.
+ *
+ * Returns the number of visible tuples.
+ */
+int
+HeapTupleSatisfiesMVCCBatch(Snapshot snapshot, Buffer buffer,
+                            int ntups,

I don't really get why this (the patch in general) would be
substantially faster.  You still call HeapTupleSatisfiesMVCC() for
each tuple in a loop. The difference is that you've got pointers into
the array of tuples instead of doing PageGetItem(), then calling
HeapTupleSatisfiesMVCC() for each tuple.


0012:
-------

> [PATCH v6 12/14] Require share-exclusive lock to set hint bits
> To address these issue, this commit changes the rules so that modifications to
> pages are not allowed anymore while holding a share lock. Instead the new

In the commit message, you make it sound like you only change the lock
level for setting the hint bits. But that wouldn't solve any problems
if FlushBuffer() could still happen with a share lock. I would try to
make it clear that you change the lock level both for setting the hint
bits and flushing the buffer.

@@ -77,6 +73,16 @@ gistkillitems(IndexScanDesc scan)
      */
     for (i = 0; i < so->numKilled; i++)
     {
+        if (!killedsomething)
+        {
+            /*
+             * Use hint bit infrastructure to be allowed to modify the page
+             * without holding an exclusive lock.
+             */
+            if (!BufferBeginSetHintBits(buffer))
+                goto unlock;
+        }
+

I don't understand why this is in the loop. Clearly you want to call
BufferBeginSetHintBits() once, but why would you do it in the loop?

In the comment, I might also note that the lock level will be upgraded
as needed or something since we only have a share lock, it is
confusing at first

- * SetHintBits()
+ * To be allowed to set hint bits, SetHintBits() needs to call
+ * BufferBeginSetHintBits(). However, that's not free, and some callsites call
+ * SetHintBits() on many tuples in a row. For those it makes sense to amortize
+ * the cost of BufferBeginSetHintBits(). Additionally it's desirable to defer
+ * the cost of BufferBeginSetHintBits() until a hint bit needs to actually be
+ * set. This enum serves as the necessary state space passed to
+ * SetHintbitsExt().
+ */
+typedef enum SetHintBitsState
+{
+   /* not yet checked if hint bits may be set */
+   SHB_INITIAL,
+   /* failed to get permission to set hint bits, don't check again */
+   SHB_DISABLED,
+   /* allowed to set hint bits */
+   SHB_ENABLED,
+} SetHintBitsState;

I dislike the SHB prefix. Perhaps something involving the word hint?
And should the enum name itself (SetHintBitsState) include the word
"batch"? I know that would make it long. At least the comment should
explain that these are needed when batch setting hint bits.

+SetHintBitsExt(HeapTupleHeader tuple, Buffer buffer,
+               uint16 infomask, TransactionId xid, SetHintBitsState *state)
 {
     if (TransactionIdIsValid(xid))
     {
-        /* NB: xid must be known committed here! */
-        XLogRecPtr    commitLSN = TransactionIdGetCommitLSN(xid);
+        if (BufferIsPermanent(buffer))
+        {

I really wish there was a way to better pull apart the batch and
non-batch cases in a way that could allow the below block to be in a
helper do_set_hint() (or whatever) which SetHintBitsExt() and
SetHintBits() called. And then you inlined BufferSetHintBits16().

I presume you didn't do this because HeapTupleSatisifiesMVCC() for
SNAPSHOT_MVCC calls the non-batch version (and, of course,
HeapTupleSatisifiesVisibility() is the much more common case).

if (TransactionIdIsValid(xid))
{
        if (BufferIsPermanent(buffer))
        {
                /* NB: xid must be known committed here! */
                XLogRecPtr    commitLSN = TransactionIdGetCommitLSN(xid);

                if (XLogNeedsFlush(commitLSN) &&
                        BufferGetLSNAtomic(buffer) < commitLSN)
                {
                        /* not flushed and no LSN interlock, so don't
set hint */
                        return; false;
                }
        }
}

Separately, I was thinking, should we assert here about having the
right lock type?

 * It is only safe to set a transaction-committed hint bit if we know the
 * transaction's commit record is guaranteed to be flushed to disk before the
 * buffer, or if the table is temporary or unlogged and will be obliterated by
 * a crash anyway.  We cannot change the LSN of the page here, because we may
 * hold only a share lock on the buffer, so we can only use the LSN to
 * interlock this if the buffer's LSN already is newer than the commit LSN;
 * otherwise we have to just refrain from setting the hint bit until some
 * future re-examination of the tuple.
 *

Should this say "we may hold only a share exclusive lock on the
buffer". Also what is "this" in "only use the LSN to interlock this"?

@@ -1628,6 +1701,9 @@ HeapTupleSatisfiesMVCCBatch(Snapshot snapshot,
Buffer buffer,
+    if (state == SHB_ENABLED)
+        BufferFinishSetHintBits(buffer, true, true);
+
     return nvis;
 }

I wondered if it would be more natural for BufferBeginSetHintBits()
and BufferFinishSetHintBits() to set SHB_INITIAL and SHB_DISABLED
instead of having callers do it. But, I guess you don't do this
because of gist and hash indexes using this for doing their own
modifications. It seems like it also would help make it clear that
BufferBegin and BufferFinish are for batch mode.

I can't help but feel a bit of awkwardness in the whole API between
this and the way you've supported non-batch mode in SetHintBitsExt().
But it's easy for me to criticize without providing concrete ideas of
how to reorganize it.

+static inline bool
+SharedBufferBeginSetHintBits(Buffer buffer, BufferDesc *buf_hdr,
uint64 *locksta>
+{
+   uint64      old_state;
+   PrivateRefCountEntry *ref;
+   BufferLockMode mode;

These functions probably ought to have comments. And I wonder if you
should say anything (or even assert anything) about how IO better not
be in progress (though it is enforced by the locks).

+        /*
+         * Can't upgrade if somebody else holds the lock in exlusive or
+         * share-exclusive mode.
+         */

typo: exlusive -> exclusive

-4. It is considered OK to update tuple commit status bits (ie, OR the
-values HEAP_XMIN_COMMITTED, HEAP_XMIN_INVALID, HEAP_XMAX_COMMITTED, or
+4. Non-critical information on a page ("hint bits") may be modified while
+holding only a share-exclusive lock and pin on the page. To do so in cases
+where only a share lock is already held, use BufferBeginSetHintBits() &
+BufferFinishSetHintBits() (if multiple hint bits are to be set) or
+BufferSetHintBits16() (if a single hit bit is set).

typo: single hit -> single hint
Also, you should probably say that you use BufferBeginSetHintBits() to
actually upgrade the lock.

I also don't see anywhere in the README where flushing a buffer is
mentioned -- and what lock level is needed for that in different
situations. It kind of feels like it is worth mentioning.

-   if ((pg_atomic_read_u64(&bufHdr->state) & (BM_DIRTY | BM_JUST_DIRTIED)) !=
-       (BM_DIRTY | BM_JUST_DIRTIED))
+   if (unlikely((lockstate & (BM_DIRTY | BM_JUST_DIRTIED)) !=
+                (BM_DIRTY | BM_JUST_DIRTIED)))

I don't quite understand why you do the atomic read in the call to
MarkSharedBufferDirtyHint() form MarkBufferDirtyHint() instead of at
the top of MarkSharedBufferDirtyHint() -- and then you wouldn't have
to pass the lockstate as a parameter, right?

--- a/src/backend/storage/freespace/freespace.c
+++ b/src/backend/storage/freespace/freespace.c
@@ -904,14 +904,22 @@ fsm_vacuum_page(Relation rel, FSMAddress addr,
    max_avail = fsm_get_max_avail(page);
    /*
-    * Reset the next slot pointer. This encourages the use of low-numbered
-    * pages, increasing the chances that a later vacuum can truncate the
-    * relation.  We don't bother with a lock here, nor with marking the page
-    * dirty if it wasn't already, since this is just a hint.
+    * Try to reset the next slot pointer. This encourages the use of
+    * low-numbered pages, increasing the chances that a later vacuum can
+    * truncate the relation.  We don't bother with a lock here, nor with
+    * marking the page dirty if it wasn't already, since this is just a hint.
+    *
+    * To be allowed to update the page without an exclusive lock, we have to
+    * use the hint bit infrastructure.
     */

What the heck? This didn't even take a share lock before...

diff --git a/src/backend/storage/freespace/fsmpage.c
b/src/backend/storage/freesp>
index 66a5c80b5a6..a59696b6484 100644
--- a/src/backend/storage/freespace/fsmpage.c
+++ b/src/backend/storage/freespace/fsmpage.c
@@ -298,9 +298,18 @@ restart:
     * lock and get a garbled next pointer every now and then, than take the
     * concurrency hit of an exclusive lock.
     *
+    * Without an exclusive lock, we need to use the hint bit infrastructure
+    * to be allowed to modify the page.
+    *

Is the sentence above this still correct?

/*
* Update the next-target pointer. Note that we do this even if we're only
* holding a shared lock, on the grounds that it's better to use a shared
* lock and get a garbled next pointer every now and then, than take the
* concurrency hit of an exclusive lock.

We appear to avoid the garbling now?

In general on 0012, I didn't spend much time checking if you caught
all the places where we mention our hint bit hackery (only taking the
share lock). But those can always be caught later as we inevitably
encounter them.

- Melanie

  29. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2025-11-25T16:54:00Z

    Hi,

Thanks a lot for that detailed review!  A few questions and comments, before I
try to address the comments in the next version.


On 2025-11-25 10:44:31 -0500, Melanie Plageman wrote:
> On Wed, Nov 19, 2025 at 9:47 PM Andres Freund <andres@anarazel.de> wrote:
> >
> > 0008: The main change. Implements buffer content locking independently from
> > lwlock.c. There's obviously a lot of similarity between lwlock.c code and
> > this, but I've not found a good way to reduce the duplication without giving
> > up too much.  This patch does immediately introduce share-exclusive as a new
> > lock level, mostly because it was too painful to do separately.
> >
> > 0009+0010+0011: Preparatory work for 0012.
> >
> > 0012: One of the main goals of this patchset - use the new share-exclusive
> > lock level to only allow hint bits to be set while no IO is going on.
> 
> Below is a review of 0008-0012
> I skipped 0013 and 0014 after seeing "#if 1" in 0013 :)

I left that in there to make the comparison easier. But clearly that's not to
be committed...


> 0008:
> -------
> > [PATCH v6 08/14] bufmgr: Implement buffer content locks independently
>  of lwlocks
> ...
> > This commit unfortunately introduces some code that is very similar to the
> > code in lwlock.c, however the code is not equivalent enough to easily merge
> > it. The future wins that this commit makes possible seem worth the cost.
> 
> It is a truly unfortunate amount of duplication. I tried some
> refactoring myself just to convince myself it wasn't a good idea.

Without success, I guess?


> However, ISTM there is no reason for the lwlock and buffer locking
> implementations to have to stay in sync. So they can diverge as
> features are added and perhaps the duplication won't be as conspicuous
> in the future.

Right - I'd expect further divergence.


> > As of this commit nothing uses the new share-exclusive lock mode. It will be
> > documented and used in a future commit. It seemed too complicated to introduce
> > the lock-level in a separate commit.
> 
> I would have liked this mentioned earlier in the commit message.

Hm, ok.  I could split share-exclusive out again, but it was somewhat painful,
because it doesn't just lead to adding code, but to changing code.


> Also, I don't know how I feel about it being "documented" in a future
> commit...perhaps just don't say that.

Hm, why?


> diff --git a/src/include/storage/buf_internals.h
> b/src/include/storage/buf_internals.h
> index 28519ad2813..0a145d95024 100644
> --- a/src/include/storage/buf_internals.h
> +++ b/src/include/storage/buf_internals.h
> @@ -32,22 +33,29 @@
>  /*
>   * Buffer state is a single 64-bit variable where following data is combined.
>   *
> + * State of the buffer itself:
>   * - 18 bits refcount
>   * - 4 bits usage count
>   * - 10 bits of flags
>   *
> + * State of the content lock:
> + * - 1 bit has_waiter
> + * - 1 bit release_ok
> + * - 1 bit lock state locked
> 
> Somewhere you should clearly explain the scenarios in which you still
> need to take the buffer header lock with LockBufHdr() vs when you can
> just use atomic operations/CAS loops.

There is an explanation of that, or at least my attempt at it ;). See the
dcoumentation for BufferDesc.


> @@ -268,6 +287,15 @@ BufMappingPartitionLockByIndex(uint32 index)
>   * wait_backend_pgprocno and setting flag bit BM_PIN_COUNT_WAITER.  At present,
>   * there can be only one such waiter per buffer.
>   *
> + * The content of buffers is protected via the buffer content lock,
> + * implemented as part buffer state. Note that the buffer header lock is *not*
> + * used to control access to the data in the buffer! We used to use an LWLock
> + * to implement the content lock, but having a dedicated implementation of
> + * content locks allows to implement some otherwise hard things (e.g.
> + * race-freely checking if AIO is in progress before locking a buffer
> + * exclusively) and makes otherwise impossible optimizations possible
> + * (e.g. unlocking and unpinning a buffer in one atomic operation).
> + *
> 
> I don't really like that this includes a description of how it used to
> work. It makes the description more confusing when trying to
> understand the current state. And comments like that make most sense
> when the current state is confusing because of a long annoying
> history.

I generally agree with that - however, in this case it seemed like folks, in
the future, might actually be wondering why this isn't using lwlocks.


> + */
> +static inline void
> +LockBuffer(Buffer buffer, BufferLockMode mode)
> +{
> +    if (mode == BUFFER_LOCK_UNLOCK)
> +        UnlockBuffer(buffer);
> +    else
> +        LockBufferInternal(buffer, mode);
> +}
> +
> 
> Is there a reason to stick with the LockBuffer(buf,
> BUFFER_LOCK_UNLOCK) interface here? It seems like a cgood time to
> start using UnlockBuffer() which I thought most people find more
> intuitive.

There are like 200 uses of BUFFER_LOCK_UNLOCK|GIN_UNLOCK|GIST_UNLOCK. And
probably a lot of external ones.  I'm not against using UnlockBuffer()
directly in the future, but changing all that code as part of this change
doesn't quite seem to make sense.


>  /*
> - * Acquire or release the content_lock for the buffer.
> + * Acquire the buffer content lock in the specified mode
> + *
> + * If the lock is not available, sleep until it is.
> + *
> + * Side effect: cancel/die interrupts are held off until lock release.
> + *
> + * This uses almost the same locking approach as lwlock.c's
> + * LWLockAcquire(). See documentation atop of lwlock.c for a more detailed
> + * discussion.
> + *
> + * The reason that this, and most of the other BufferLock* functions, get both
> + * the Buffer and BufferDesc* as parameters, is that looking up one from the
> + * other repeatedly shows up noticeably in profiles.
> + */
> +static inline void
> +BufferLockAcquire(Buffer buffer, BufferDesc *buf_hdr, BufferLockMode mode)
> 
> Either here or over the lock mode enum, I'd spend a few sentences
> describing how the buffer lock modes interact -- what conflicts with
> what. You spend more time saying how this is like LWLock than
> explaining what it actually is.

Historically they're just explained in the README. But yea, it makes sense to
add to the enum. I like adding it to the enum better than to the function, as
there are multiple ways to acquire a lock.


> +{
> +    bool        new_release_ok;
> +    bool        wake_exclusive = unlocked;
> +    bool        wake_share_exclusive = true;
> +    proclist_head wakeup;
> +    proclist_mutable_iter iter;
> +
> +    proclist_init(&wakeup);
> +
> +    new_release_ok = true;
> +
> +    /* lock wait list while collecting backends to wake up */
> +    LockBufHdr(buf_hdr);
> +
> +    proclist_foreach_modify(iter, &buf_hdr->lock_waiters, lwWaitLink)
> +    {
> +        PGPROC       *waiter = GetPGProcByNumber(iter.cur);
> +
> +        if (!wake_exclusive && waiter->lwWaitMode == BUFFER_LOCK_EXCLUSIVE)
> +            continue;
> +
> +        if (!wake_share_exclusive && waiter->lwWaitMode ==
> BUFFER_LOCK_SHARE_EXCLUSIVE)
> +            continue;
> +
> 
> It seems there is a difference between LWLockWakeup() and
> BufferLockWakeup() if the queue contains a share lock waiter followed
> by an exclusive lock waiter.
> 
> In LWLockWakeup(), it will wake up the share waiter, set
> wokeup_somebody to true, and then not wake up the exclusive lock
> waiter because wokeup_somebody is true.
> 
> In BufferLockWakeup() when unlocked is true, it will wake up the share
> lock waiter and then wake up the exclusive lock waiter because
> wake_exclusive and wake_share_exclusive are both still true.
> 
> This might be the intended behavior, but it is a difference from
> LWLockWakeup(), so I think it is worth documenting why it is okay.

Yea, that doesn't look quite right. I think I just whacked the code around too
much and somewhere lost that branch.


> 
> +         * Signal that the process isn't on the wait list anymore. This allows
> +         * BufferLockDequeueSelf() to remove itself of the waitlist with a
> +         * proclist_delete(), rather than having to check if it has been
> 
> I know this comment was ported over, but the "remove itself of the
> waitlist" -- the "of" is confusing in the original comment and it is
> confusing here.

As in s/of/from/?


> + * Compute subtraction from buffer state for a release of a held lock in
> + * `mode`.
> + *
> + * This is separated from BufferLockUnlock() as we want to combine the lock
> + * release with other atomic operations when possible, leading to the lock
> + * release being done in multiple places.
> + */
> +static inline uint64
> +BufferLockReleaseSub(BufferLockMode mode)
> 
> I don't understand why this is a separate function even with your comment.

Because there are operations where we want to unlock the buffer as well as do
something else. E.g. in 0013, UnlockReleaseBuffer() we want to unlock the
buffer and decrease the refcount in one atomic operation. For that we need to
know what to subtract from the state variable for the lock portion - hence
BufferLockReleaseSub().
 

> + * BufferLockHeldByMe - test whether my process holds the content lock in any
> + * mode
> + *
> + * This is meant as debug support only.
> + */
> +static bool
> +BufferLockHeldByMe(BufferDesc *buf_hdr)
> +{
> +    PrivateRefCountEntry *entry =
> +        GetPrivateRefCountEntry(BufferDescriptorGetBuffer(buf_hdr), false);
> +
> +    if (!entry)
> +        return false;
> +    else
> +        return entry->data.lockmode != BUFFER_LOCK_UNLOCK;
> +}
> 
> Previously, if I called LockBuffer(buf, BUFFER_LOCK_SHARE) again after
> calling it once, say in RelationCopyStorageUsingBuffer(), I would trip
> an assert when unpinning the buffer about how I needed to have
> released the lock.

Right, that's important to keep that way.


> Now, though, it doesn't trip the assert because we don't track
> multiple locks. When I call LockBuffer(UNLOCK), it sets lockmode to
> BUFFER_LOCK_UNLOCK and BufferLockHeldByMe() only checks the private
> refcount entry.

That part seems right.


> Whereas LWLockHeldByMe() checked held_lwlocks which had multiple entries and
> would report that I still held a lock.

It'd be much better if we had detected that redundant lock acquisition, that's
not legal...


> Now, if I call LockBuffer(SHARE) twice, I'll only find out later when I have
> a hang because someone is trying to get an exclusive lock and the actual
> BufferDesc->state still has a lock set.
> 
> Maybe you should add an assert to the lock acquisition path that the
> prevate ref count entry mode is UNLOCK?

Yes, we should...

It'd be nice if we had a decent way to test things that we except to
crash. Like repeated buffer lock acquisitions...

>  void
> -LockBuffer(Buffer buffer, BufferLockMode mode)
> +LockBufferInternal(Buffer buffer, BufferLockMode mode)
>  {
> -    buf = GetBufferDescriptor(buffer - 1);
> +    buf_hdr = GetBufferDescriptor(buffer - 1);
> 
> -    if (mode == BUFFER_LOCK_UNLOCK)
> -        LWLockRelease(BufferDescriptorGetContentLock(buf));
> -    else if (mode == BUFFER_LOCK_SHARE)
> -        LWLockAcquire(BufferDescriptorGetContentLock(buf), LW_SHARED);
> +    if (mode == BUFFER_LOCK_SHARE)
> +        BufferLockAcquire(buffer, buf_hdr, BUFFER_LOCK_SHARE);
> +    else if (mode == BUFFER_LOCK_SHARE_EXCLUSIVE)
> +        BufferLockAcquire(buffer, buf_hdr, BUFFER_LOCK_SHARE_EXCLUSIVE);
>      else if (mode == BUFFER_LOCK_EXCLUSIVE)
> -        LWLockAcquire(BufferDescriptorGetContentLock(buf), LW_EXCLUSIVE);
> +        BufferLockAcquire(buffer, buf_hdr, BUFFER_LOCK_EXCLUSIVE);
>      else
>          elog(ERROR, "unrecognized buffer lock mode: %d", mode);
> 
> I presume you've stuck with this if statement structure because that
> is what LockBuffer() used.
>
> Even though now the BufferLockMode passed
> through to BufferLockAcquire is the exact thing you are testing.

Ah. Using explicit constants allows the compiler to do constant propagation
(so e.g. BufferLockAttempt() doesn't have runtime branches on mode), which
turns out to generate more efficient code.  I'll add a comment to that effect.


> @@ -6625,7 +7295,25 @@ ResOwnerReleaseBufferPin(Datum res)
>      if (BufferIsLocal(buffer))
>          UnpinLocalBufferNoOwner(buffer);
>      else
> +    {
> +        PrivateRefCountEntry *ref;
> +
> +        ref = GetPrivateRefCountEntry(buffer, false);
> +
> +        /*
> +         * If the buffer was locked at the time of the resowner release,
> +         * release the lock now. This should only happen after errors.
> +         */
> +        if (ref->data.lockmode != BUFFER_LOCK_UNLOCK)
> +        {
> +            BufferDesc *buf = GetBufferDescriptor(buffer - 1);
> +
> +            HOLD_INTERRUPTS();    /* match the upcoming RESUME_INTERRUPTS */
> +            BufferLockUnlock(buffer, buf);
> +        }
> +
>          UnpinBufferNoOwner(GetBufferDescriptor(buffer - 1));
> +    }
>  }
> 
>  Bit confusing that ResOwnerReleaseBufferBin() now releases locks as well.

Do you have a better suggestion? I'll add a comment that makes that explicit,
but other than that I don't have a great idea. Renaming the whole buffer pin
mechanism seems pretty noisy.


> 0009 -- I didn't look closely at 0009

That's hopefully just a boring code move...


> 0010:
> --------
> > [PATCH v6 10/14] heapam: Use exclusive lock on old page in CLUSTER
> 
> > To be able to guarantee that we can set the hint bit, acquire an exclusive
> > lock on the old buffer. We need the hint bits to be set as otherwise
> > reform_and_rewrite_tuple() -> rewrite_heap_tuple() -> heap_freeze_tuple() will
> > get confused.
> 
> So, this is an active bug?

I don't think so - we have an AEL on the relation at that point, so nobody
else can access the buffer, aside from checkpointer writing it out. Which
doesn't currrently block hint bits from being set.


> And what exactly do you mean heap_freeze_tuple() gets confused? I thought
> somewhere in there we would check the clog.

heap_freeze_tuple()->heap_prepare_freeze_tuple() assumes that:
 * It is assumed that the caller has checked the tuple with
 * HeapTupleSatisfiesVacuum() and determined that it is not HEAPTUPLE_DEAD
 * (else we should be removing the tuple, not freezing it).

In the new world, if HTSV were not allowed to set the hint bit, e.g. because
the page is in the process of being written out, there wouldn't be a guarantee
that hints were set.  Which then leads to confusion, because some of the code
assumes that hint bits were already set.  TBH, I don't quite remember the
precise details, it's been a while (this was already part of an earlier
attempt at dealing with the hint bit stuff).  I'll try to reconstruct.


> 0011:
> --------
> > [PATCH v6 11/14] heapam: Add batch mode mvcc check and use it in page mode
> 
> > 2) We would like to stop setting hint bits while pages are being written
> > out. The necessary locking becomes visible for page mode scans if done for
> > every tuple. With batching the overhead can be amortized to only happen
> > once per page.
> 
> I don't understand the above point. What does this patch have to do
> with not setting hint bits while pages are being written out (that
> happens in the next patch)?

It's just an explanation for why we want to use batch mode. Without the
changed locking model the reason for that is a lot less clear.

> And I presume you mean don't set hint bits on a buffer that is being flushed
> by someone else -- but it sounds like you mean not to set hint bits as part
> of flushing a buffer.

Right, I do mean the former.


> diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
> index 4b0c49f4bb0..ddabd1a3ec3 100644
> --- a/src/backend/access/heap/heapam.c
> +++ b/src/backend/access/heap/heapam.c
> @@ -504,42 +504,93 @@ page_collect_tuples(HeapScanDesc scan, Snapshot snapshot,
>                      BlockNumber block, int lines,
>                      bool all_visible, bool check_serializable)
>  {
> +    Oid            relid = RelationGetRelid(scan->rs_base.rs_rd);
> +#ifdef BATCHMVCC_FEWER_ARGS
> +    BatchMVCCState batchmvcc;
> +    HeapTupleData *tuples = batchmvcc.tuples;
> +    bool       *visible = batchmvcc.visible;
> +#else
> +    HeapTupleData tuples[MaxHeapTuplesPerPage];
> +    bool        visible[MaxHeapTuplesPerPage];
> +#endif
>      int            ntup = 0;
> 
> It's pretty confusing when visible is an output vs input parameter and
> who fills it in when. (i.e. it's filled in in page_collect_tuples() if
> check_serializable and all_visible are true, otherwise it's filled in
> in HeapTupleSatisifiesMVCCBatch())

I don't really see a good alternative.


> Personally, I think I'd almost prefer an all-visible and
> not-all-visible version of page_collect_tuples() (or helper functions
> containing the loop) that separate this. (I haven't tried it though)

I have a hard time seeing how that comes out better. Where would you make the
switch between the different functions and how would that lead to easier to
understand code?


> BATCHMVCC_FEWER_ARGS definitely doesn't make it any easier to read --
> which I assume you are removing.

Yea, I only left it in so others perhaps can reproduce the performance effect
of needing BATCHMVCC_FEWER_ARGS.


> +        /*
> +         * If the page is not all-visible or we need to check serializability,
> +         * maintain enough state to be able to refind the tuple efficiently,
> +         * without again needing to extract it from the page.
> +         */
> +        if (!all_visible || check_serializable)
> +        {
> 
> "enough state" is pretty vague here.

I don't really follow. Wouldn't going into more detail just restate the code
in a comment?


> Maybe mention what it wouldn't be valid to do with the tuples array?

Hm? The tuples array *is* that state?



> + * visibility is set in batchmvcc->visible[]. In addition, ->vistuples_dense
> + * is set to contain the offsets of visible tuples.
> + *
> + * Returns the number of visible tuples.
> + */
> +int
> +HeapTupleSatisfiesMVCCBatch(Snapshot snapshot, Buffer buffer,
> +                            int ntups,
> 
> I don't really get why this (the patch in general) would be
> substantially faster.  You still call HeapTupleSatisfiesMVCC() for
> each tuple in a loop. The difference is that you've got pointers into
> the array of tuples instead of doing PageGetItem(), then calling
> HeapTupleSatisfiesMVCC() for each tuple.

There are a few reasons:

1) Most importantly, in the batched world, we only need to check if we can set
   hint bits once. That check is far from free. We also only need to mark the
   buffer dirty once.

2) HeapTupleSatisfiesMVCCBatch() can inline HeapTupleSatisfiesMVCC() and avoid
   some redundant work, e.g. it doesn't have to set up a stack frame each
   time.

3) A loop over the tuples in heapam.c needs an external function call to
   HeapTupleSatisfiesMVCC(), as that's in heapam_visibility.c. That function
   call quickly shows up.


> 
> 0012:
> -------
> 
> > [PATCH v6 12/14] Require share-exclusive lock to set hint bits
> > To address these issue, this commit changes the rules so that modifications to
> > pages are not allowed anymore while holding a share lock. Instead the new
> 
> In the commit message, you make it sound like you only change the lock
> level for setting the hint bits. But that wouldn't solve any problems
> if FlushBuffer() could still happen with a share lock. I would try to
> make it clear that you change the lock level both for setting the hint
> bits and flushing the buffer.

Good point.


> @@ -77,6 +73,16 @@ gistkillitems(IndexScanDesc scan)
>       */
>      for (i = 0; i < so->numKilled; i++)
>      {
> +        if (!killedsomething)
> +        {
> +            /*
> +             * Use hint bit infrastructure to be allowed to modify the page
> +             * without holding an exclusive lock.
> +             */
> +            if (!BufferBeginSetHintBits(buffer))
> +                goto unlock;
> +        }
> +
> 
> I don't understand why this is in the loop. Clearly you want to call
> BufferBeginSetHintBits() once, but why would you do it in the loop?

Why would we want to continue if we can't set hint bits?


> In the comment, I might also note that the lock level will be upgraded
> as needed or something since we only have a share lock, it is
> confusing at first

I don't think copying the way this works into all the callers of
BufferBeginSetHintBits() is a good idea. That just makes it harder to adjust
going down the road.


> - * SetHintBits()
> + * To be allowed to set hint bits, SetHintBits() needs to call
> + * BufferBeginSetHintBits(). However, that's not free, and some callsites call
> + * SetHintBits() on many tuples in a row. For those it makes sense to amortize
> + * the cost of BufferBeginSetHintBits(). Additionally it's desirable to defer
> + * the cost of BufferBeginSetHintBits() until a hint bit needs to actually be
> + * set. This enum serves as the necessary state space passed to
> + * SetHintbitsExt().
> + */
> +typedef enum SetHintBitsState
> +{
> +   /* not yet checked if hint bits may be set */
> +   SHB_INITIAL,
> +   /* failed to get permission to set hint bits, don't check again */
> +   SHB_DISABLED,
> +   /* allowed to set hint bits */
> +   SHB_ENABLED,
> +} SetHintBitsState;
> 
> I dislike the SHB prefix. Perhaps something involving the word hint?

Shrug. It's a very locally used enum, I don't think it matters terribly.


> And should the enum name itself (SetHintBitsState) include the word
> "batch"? I know that would make it

> long. At least the comment should explain that these are needed when batch
> setting hint bits.

Isn't that what the comment does, explaining that we want to amortize the cost
of BufferBeginSetHintBits()?


> +SetHintBitsExt(HeapTupleHeader tuple, Buffer buffer,
> +               uint16 infomask, TransactionId xid, SetHintBitsState *state)
>  {
>      if (TransactionIdIsValid(xid))
>      {
> -        /* NB: xid must be known committed here! */
> -        XLogRecPtr    commitLSN = TransactionIdGetCommitLSN(xid);
> +        if (BufferIsPermanent(buffer))
> +        {
> 
> I really wish there was a way to better pull apart the batch and
> non-batch cases in a way that could allow the below block to be in a
> helper do_set_hint() (or whatever) which SetHintBitsExt() and
> SetHintBits() called.

I couldn't see a way that didn't lead to substantially more code duplication.


> And then you inlined BufferSetHintBits16().

Hm?


> I presume you didn't do this because HeapTupleSatisifiesMVCC() for
> SNAPSHOT_MVCC calls the non-batch version (and, of course,
> HeapTupleSatisifiesVisibility() is the much more common case).
> 
> if (TransactionIdIsValid(xid))
> {
>         if (BufferIsPermanent(buffer))
>         {
>                 /* NB: xid must be known committed here! */
>                 XLogRecPtr    commitLSN = TransactionIdGetCommitLSN(xid);
> 
>                 if (XLogNeedsFlush(commitLSN) &&
>                         BufferGetLSNAtomic(buffer) < commitLSN)
>                 {
>                         /* not flushed and no LSN interlock, so don't
> set hint */
>                         return; false;
>                 }
>         }
> }
> 
> Separately, I was thinking, should we assert here about having the
> right lock type?

Not sure I get what assert of what locktype where?


>  * It is only safe to set a transaction-committed hint bit if we know the
>  * transaction's commit record is guaranteed to be flushed to disk before the
>  * buffer, or if the table is temporary or unlogged and will be obliterated by
>  * a crash anyway.  We cannot change the LSN of the page here, because we may
>  * hold only a share lock on the buffer, so we can only use the LSN to
>  * interlock this if the buffer's LSN already is newer than the commit LSN;
>  * otherwise we have to just refrain from setting the hint bit until some
>  * future re-examination of the tuple.
>  *
> 
> Should this say "we may hold only a share exclusive lock on the
> buffer". Also what is "this" in "only use the LSN to interlock this"?

That's a pre-existing comment, right?


> @@ -1628,6 +1701,9 @@ HeapTupleSatisfiesMVCCBatch(Snapshot snapshot,
> Buffer buffer,
> +    if (state == SHB_ENABLED)
> +        BufferFinishSetHintBits(buffer, true, true);
> +
>      return nvis;
>  }
> 
> I wondered if it would be more natural for BufferBeginSetHintBits()
> and BufferFinishSetHintBits() to set SHB_INITIAL and SHB_DISABLED
> instead of having callers do it. But, I guess you don't do this
> because of gist and hash indexes using this for doing their own
> modifications.

I thought about it. But yea, the different callers seemed to make that not
really useful. It'd also mean that we'd do an external function call for every
tuple, which I think would be prohibitively expensive.



> --- a/src/backend/storage/freespace/freespace.c
> +++ b/src/backend/storage/freespace/freespace.c
> @@ -904,14 +904,22 @@ fsm_vacuum_page(Relation rel, FSMAddress addr,
>     max_avail = fsm_get_max_avail(page);
>     /*
> -    * Reset the next slot pointer. This encourages the use of low-numbered
> -    * pages, increasing the chances that a later vacuum can truncate the
> -    * relation.  We don't bother with a lock here, nor with marking the page
> -    * dirty if it wasn't already, since this is just a hint.
> +    * Try to reset the next slot pointer. This encourages the use of
> +    * low-numbered pages, increasing the chances that a later vacuum can
> +    * truncate the relation.  We don't bother with a lock here, nor with
> +    * marking the page dirty if it wasn't already, since this is just a hint.
> +    *
> +    * To be allowed to update the page without an exclusive lock, we have to
> +    * use the hint bit infrastructure.
>      */
> 
> What the heck? This didn't even take a share lock before...

It is insane. I do not understand how anybody thought this was ok.  I think I
probably should split this out into a separate commit, since it's so insane.


> diff --git a/src/backend/storage/freespace/fsmpage.c
> b/src/backend/storage/freesp>
> index 66a5c80b5a6..a59696b6484 100644
> --- a/src/backend/storage/freespace/fsmpage.c
> +++ b/src/backend/storage/freespace/fsmpage.c
> @@ -298,9 +298,18 @@ restart:
>      * lock and get a garbled next pointer every now and then, than take the
>      * concurrency hit of an exclusive lock.
>      *
> +    * Without an exclusive lock, we need to use the hint bit infrastructure
> +    * to be allowed to modify the page.
> +    *
> 
> Is the sentence above this still correct?

Seems ok enough to me.


> /*
> * Update the next-target pointer. Note that we do this even if we're only
> * holding a shared lock, on the grounds that it's better to use a shared
> * lock and get a garbled next pointer every now and then, than take the
> * concurrency hit of an exclusive lock.
> 
> We appear to avoid the garbling now?

I don't think so. Two backends concurrently can do fsm_search_avail() and
one backend might set a hint to a page that is already used up by the other
one. At least I think so?


> In general on 0012, I didn't spend much time checking if you caught
> all the places where we mention our hint bit hackery (only taking the
> share lock). But those can always be caught later as we inevitably
> encounter them.

There's definitely some more search needed as part of polishing that
commit. But I think it's pretty much inevitable that we'll miss some comment
somewhere :(

Greetings,

Andres Freund

  30. Re: Buffer locking is special (hints, checksums, AIO writes)

    Melanie Plageman <melanieplageman@gmail.com> — 2025-11-25T20:02:02Z

    On Tue, Nov 25, 2025 at 11:54 AM Andres Freund <andres@anarazel.de> wrote:
>
> > I skipped 0013 and 0014 after seeing "#if 1" in 0013 :)
>
> I left that in there to make the comparison easier. But clearly that's not to
> be committed...

Eh, I mostly just ran out of steam.

> > > [PATCH v6 08/14] bufmgr: Implement buffer content locks independently
> >  of lwlocks
> > ...
> > > This commit unfortunately introduces some code that is very similar to the
> > > code in lwlock.c, however the code is not equivalent enough to easily merge
> > > it. The future wins that this commit makes possible seem worth the cost.
> >
> > It is a truly unfortunate amount of duplication. I tried some
> > refactoring myself just to convince myself it wasn't a good idea.
>
> Without success, I guess?

Nothing that seemed better...and not worse :)

> > > As of this commit nothing uses the new share-exclusive lock mode. It will be
> > > documented and used in a future commit. It seemed too complicated to introduce
> > > the lock-level in a separate commit.
> >
> > I would have liked this mentioned earlier in the commit message.
>
> Hm, ok.  I could split share-exclusive out again, but it was somewhat painful,
> because it doesn't just lead to adding code, but to changing code.

I don't think you need to introduce share-exclusive in a separate
commit. I just mean it would be good to mention in the commit message
before you get into so many other details that this commit doesn't use
the share-exclusive level yet.

> > Also, I don't know how I feel about it being "documented" in a future
> > commit...perhaps just don't say that.
>
> Hm, why?

I don't really see you documenting share-exclusive lock semantics in a
later commit. Documentation for what the lock level is should be in
the same commit that introduces it -- which I think you mostly have
done. I feel like it is weird to say you will document that lock level
later when 1) you don't really do that and 2) this commit mostly
(besides some requests I had for elaboration) already does that.

> > diff --git a/src/include/storage/buf_internals.h
> > b/src/include/storage/buf_internals.h
> > index 28519ad2813..0a145d95024 100644
> > --- a/src/include/storage/buf_internals.h
> > +++ b/src/include/storage/buf_internals.h
> > @@ -32,22 +33,29 @@
> >  /*
> >   * Buffer state is a single 64-bit variable where following data is combined.
> >   *
> > + * State of the buffer itself:
> >   * - 18 bits refcount
> >   * - 4 bits usage count
> >   * - 10 bits of flags
> >   *
> > + * State of the content lock:
> > + * - 1 bit has_waiter
> > + * - 1 bit release_ok
> > + * - 1 bit lock state locked
> >
> > Somewhere you should clearly explain the scenarios in which you still
> > need to take the buffer header lock with LockBufHdr() vs when you can
> > just use atomic operations/CAS loops.
>
> There is an explanation of that, or at least my attempt at it ;). See the
> dcoumentation for BufferDesc.

Hmm. I suppose. I was imagining something above the state member since
you have added to it, but okay.

Separately, I'll admit I don't quite understand when I have to use
LockBufHdr() and when I should use a CAS loop to update the
bufferdesc->state.

> >
> > +         * Signal that the process isn't on the wait list anymore. This allows
> > +         * BufferLockDequeueSelf() to remove itself of the waitlist with a
> > +         * proclist_delete(), rather than having to check if it has been
> >
> > I know this comment was ported over, but the "remove itself of the
> > waitlist" -- the "of" is confusing in the original comment and it is
> > confusing here.
>
> As in s/of/from/?

Yes, I wasn't sure if it meant "from" or "off" -- but I believe "from"
is more grammatically correct.

> > +static inline uint64
> > +BufferLockReleaseSub(BufferLockMode mode)
> >
> > I don't understand why this is a separate function even with your comment.
>
> Because there are operations where we want to unlock the buffer as well as do
> something else. E.g. in 0013, UnlockReleaseBuffer() we want to unlock the
> buffer and decrease the refcount in one atomic operation. For that we need to
> know what to subtract from the state variable for the lock portion - hence
> BufferLockReleaseSub().

Hmm. Okay well maybe save it for 0013? I don't care that much, though.

> > Maybe you should add an assert to the lock acquisition path that the
> > prevate ref count entry mode is UNLOCK?
>
> Yes, we should...
>
> It'd be nice if we had a decent way to test things that we except to
> crash. Like repeated buffer lock acquisitions...

Indeed.

> > @@ -6625,7 +7295,25 @@ ResOwnerReleaseBufferPin(Datum res)
> >      if (BufferIsLocal(buffer))
> >          UnpinLocalBufferNoOwner(buffer);
> >      else
> > +    {
> > +        PrivateRefCountEntry *ref;
> > +
> > +        ref = GetPrivateRefCountEntry(buffer, false);
> > +
> > +        /*
> > +         * If the buffer was locked at the time of the resowner release,
> > +         * release the lock now. This should only happen after errors.
> > +         */
> > +        if (ref->data.lockmode != BUFFER_LOCK_UNLOCK)
> > +        {
> > +            BufferDesc *buf = GetBufferDescriptor(buffer - 1);
> > +
> > +            HOLD_INTERRUPTS();    /* match the upcoming RESUME_INTERRUPTS */
> > +            BufferLockUnlock(buffer, buf);
> > +        }
> > +
> >          UnpinBufferNoOwner(GetBufferDescriptor(buffer - 1));
> > +    }
> >  }
> >
> >  Bit confusing that ResOwnerReleaseBufferBin() now releases locks as well.
>
> Do you have a better suggestion? I'll add a comment that makes that explicit,
> but other than that I don't have a great idea. Renaming the whole buffer pin
> mechanism seems pretty noisy.

ResOwnerReleaseBuffer()?

What do you mean renaming the whole buffer pin mechanism?

> > 0011:
> > --------
> > > [PATCH v6 11/14] heapam: Add batch mode mvcc check and use it in page mode
> >
> > > 2) We would like to stop setting hint bits while pages are being written
> > > out. The necessary locking becomes visible for page mode scans if done for
> > > every tuple. With batching the overhead can be amortized to only happen
> > > once per page.
>
> > And I presume you mean don't set hint bits on a buffer that is being flushed
> > by someone else -- but it sounds like you mean not to set hint bits as part
> > of flushing a buffer.
>
> Right, I do mean the former.

I would try and state it more clearly then.

> > +        /*
> > +         * If the page is not all-visible or we need to check serializability,
> > +         * maintain enough state to be able to refind the tuple efficiently,
> > +         * without again needing to extract it from the page.
> > +         */
> > +        if (!all_visible || check_serializable)
> > +        {
> >
> > "enough state" is pretty vague here.
>
> I don't really follow. Wouldn't going into more detail just restate the code
> in a comment?

I guess maybe something like

If the page is not all-visible or we need to check serializability,
keep track of the tuples so we can examine them later without the
overhead of extracting them from the page again.

> > 0012:
> > -------
>
> > @@ -77,6 +73,16 @@ gistkillitems(IndexScanDesc scan)
> >       */
> >      for (i = 0; i < so->numKilled; i++)
> >      {
> > +        if (!killedsomething)
> > +        {
> > +            /*
> > +             * Use hint bit infrastructure to be allowed to modify the page
> > +             * without holding an exclusive lock.
> > +             */
> > +            if (!BufferBeginSetHintBits(buffer))
> > +                goto unlock;
> > +        }
> > +
> >
> > I don't understand why this is in the loop. Clearly you want to call
> > BufferBeginSetHintBits() once, but why would you do it in the loop?
>
> Why would we want to continue if we can't set hint bits?

No, I'm suggesting that you move BufferBeginSetHintBits() outside the
loop so it is more obvious that it is happening once at the beginning.
Like this:

    if (so->numKilled > 0 && !BufferBeginSetHintBits(buffer))
        goto unlock;

    for (i = 0; i < so->numKilled; i++)
    {
        offnum = so->killedItems[i];
        iid = PageGetItemId(page, offnum);
        ItemIdMarkDead(iid);
        killedsomething = true;
    }

    if (killedsomething)
    {
        GistMarkPageHasGarbage(page);
        BufferFinishSetHintBits(buffer, true, true);
    }

unlock:
    UnlockReleaseBuffer(buffer);

> > And should the enum name itself (SetHintBitsState) include the word
> > "batch"? I know that would make it
> > long. At least the comment should explain that these are needed when batch
> > setting hint bits.
>
> Isn't that what the comment does, explaining that we want to amortize the cost
> of BufferBeginSetHintBits()?

well, amortize is a pretty fancy word and you don't say "batch" anywhere.

> > And then you inlined BufferSetHintBits16().

This was part of my wishlist for separating the batch and non-batch versions.

> > I presume you didn't do this because HeapTupleSatisifiesMVCC() for
> > SNAPSHOT_MVCC calls the non-batch version (and, of course,
> > HeapTupleSatisifiesVisibility() is the much more common case).
> >
> > if (TransactionIdIsValid(xid))
> > {
> >         if (BufferIsPermanent(buffer))
> >         {
> >                 /* NB: xid must be known committed here! */
> >                 XLogRecPtr    commitLSN = TransactionIdGetCommitLSN(xid);
> >
> >                 if (XLogNeedsFlush(commitLSN) &&
> >                         BufferGetLSNAtomic(buffer) < commitLSN)
> >                 {
> >                         /* not flushed and no LSN interlock, so don't
> > set hint */
> >                         return; false;
> >                 }
> >         }
> > }
> >
> > Separately, I was thinking, should we assert here about having the
> > right lock type?
>
> Not sure I get what assert of what locktype where?

In SetHintBitsExt() that we have share-exclusive or above. Or are
there still callers with only a share lock?

> >  * It is only safe to set a transaction-committed hint bit if we know the
> >  * transaction's commit record is guaranteed to be flushed to disk before the
> >  * buffer, or if the table is temporary or unlogged and will be obliterated by
> >  * a crash anyway.  We cannot change the LSN of the page here, because we may
> >  * hold only a share lock on the buffer, so we can only use the LSN to
> >  * interlock this if the buffer's LSN already is newer than the commit LSN;
> >  * otherwise we have to just refrain from setting the hint bit until some
> >  * future re-examination of the tuple.
> >  *
> >
> > Should this say "we may hold only a share exclusive lock on the
> > buffer". Also what is "this" in "only use the LSN to interlock this"?
>
> That's a pre-existing comment, right?

Right, but are there callers that will only have a share lock after your change?

> > @@ -1628,6 +1701,9 @@ HeapTupleSatisfiesMVCCBatch(Snapshot snapshot,
> > Buffer buffer,
> > +    if (state == SHB_ENABLED)
> > +        BufferFinishSetHintBits(buffer, true, true);
> > +
> >      return nvis;
> >  }
> >
> > I wondered if it would be more natural for BufferBeginSetHintBits()
> > and BufferFinishSetHintBits() to set SHB_INITIAL and SHB_DISABLED
> > instead of having callers do it. But, I guess you don't do this
> > because of gist and hash indexes using this for doing their own
> > modifications.
>
> I thought about it. But yea, the different callers seemed to make that not
> really useful. It'd also mean that we'd do an external function call for every
> tuple, which I think would be prohibitively expensive.

I'm confused why this would mean more external function calls.

> > --- a/src/backend/storage/freespace/freespace.c
> > +++ b/src/backend/storage/freespace/freespace.c
> > @@ -904,14 +904,22 @@ fsm_vacuum_page(Relation rel, FSMAddress addr,
> >     max_avail = fsm_get_max_avail(page);
> >     /*
> > -    * Reset the next slot pointer. This encourages the use of low-numbered
> > -    * pages, increasing the chances that a later vacuum can truncate the
> > -    * relation.  We don't bother with a lock here, nor with marking the page
> > -    * dirty if it wasn't already, since this is just a hint.
> > +    * Try to reset the next slot pointer. This encourages the use of
> > +    * low-numbered pages, increasing the chances that a later vacuum can
> > +    * truncate the relation.  We don't bother with a lock here, nor with
> > +    * marking the page dirty if it wasn't already, since this is just a hint.
> > +    *
> > +    * To be allowed to update the page without an exclusive lock, we have to
> > +    * use the hint bit infrastructure.
> >      */
> >
> > What the heck? This didn't even take a share lock before...
>
> It is insane. I do not understand how anybody thought this was ok.  I think I
> probably should split this out into a separate commit, since it's so insane.

Aren't you going to backport having it take a lock? Then it can be
separate (e.g. have it take a share lock in the "fix" commit and then
this commit bumps it to share-exclusive).

> > diff --git a/src/backend/storage/freespace/fsmpage.c
> > b/src/backend/storage/freesp>
> > /*
> > * Update the next-target pointer. Note that we do this even if we're only
> > * holding a shared lock, on the grounds that it's better to use a shared
> > * lock and get a garbled next pointer every now and then, than take the
> > * concurrency hit of an exclusive lock.
> >
> > We appear to avoid the garbling now?
>
> I don't think so. Two backends concurrently can do fsm_search_avail() and
> one backend might set a hint to a page that is already used up by the other
> one. At least I think so?

Maybe I don't know what it meant by garbled, but I thought it was
talking about two backends each trying to set fp_next_slot. If they
now have to have a share-exclusive lock and they can't both have a
share-exclusive lock at the same time, then it seems like that
wouldn't be a problem. It sounds like you may be talking about a
backend taking up the freespace of a page that is referred to by the
fp_next_slot?

- Melanie

  31. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2025-11-25T20:46:49Z

    Hi,

On 2025-11-25 15:02:02 -0500, Melanie Plageman wrote:
> On Tue, Nov 25, 2025 at 11:54 AM Andres Freund <andres@anarazel.de> wrote:
> > > > As of this commit nothing uses the new share-exclusive lock mode. It will be
> > > > documented and used in a future commit. It seemed too complicated to introduce
> > > > the lock-level in a separate commit.
> > >
> > > I would have liked this mentioned earlier in the commit message.
> >
> > Hm, ok.  I could split share-exclusive out again, but it was somewhat painful,
> > because it doesn't just lead to adding code, but to changing code.
> 
> I don't think you need to introduce share-exclusive in a separate
> commit. I just mean it would be good to mention in the commit message
> before you get into so many other details that this commit doesn't use
> the share-exclusive level yet.
> 
> > > Also, I don't know how I feel about it being "documented" in a future
> > > commit...perhaps just don't say that.
> >
> > Hm, why?
> 
> I don't really see you documenting share-exclusive lock semantics in a
> later commit. Documentation for what the lock level is should be in
> the same commit that introduces it -- which I think you mostly have
> done. I feel like it is weird to say you will document that lock level
> later when 1) you don't really do that and 2) this commit mostly
> (besides some requests I had for elaboration) already does that.

The problem I faced was that the explanation for the lock level depends on
later changes - how do you document why share-exclusive is useful without
explaining the hint bit thing, which isn't yet implemented as of that commit.


> > > diff --git a/src/include/storage/buf_internals.h
> > > b/src/include/storage/buf_internals.h
> > > index 28519ad2813..0a145d95024 100644
> > > --- a/src/include/storage/buf_internals.h
> > > +++ b/src/include/storage/buf_internals.h
> > > @@ -32,22 +33,29 @@
> > >  /*
> > >   * Buffer state is a single 64-bit variable where following data is combined.
> > >   *
> > > + * State of the buffer itself:
> > >   * - 18 bits refcount
> > >   * - 4 bits usage count
> > >   * - 10 bits of flags
> > >   *
> > > + * State of the content lock:
> > > + * - 1 bit has_waiter
> > > + * - 1 bit release_ok
> > > + * - 1 bit lock state locked
> > >
> > > Somewhere you should clearly explain the scenarios in which you still
> > > need to take the buffer header lock with LockBufHdr() vs when you can
> > > just use atomic operations/CAS loops.
> >
> > There is an explanation of that, or at least my attempt at it ;). See the
> > dcoumentation for BufferDesc.
> 
> Hmm. I suppose. I was imagining something above the state member since
> you have added to it, but okay.
> 
> Separately, I'll admit I don't quite understand when I have to use
> LockBufHdr() and when I should use a CAS loop to update the
> bufferdesc->state.

It's definitely subtle :(. Not sure what to do about that, other than to work
on eventually just making everything doable with just a CAS. But that's a fair
bit of future work away.


> > > @@ -6625,7 +7295,25 @@ ResOwnerReleaseBufferPin(Datum res)
> > >      if (BufferIsLocal(buffer))
> > >          UnpinLocalBufferNoOwner(buffer);
> > >      else
> > > +    {
> > > +        PrivateRefCountEntry *ref;
> > > +
> > > +        ref = GetPrivateRefCountEntry(buffer, false);
> > > +
> > > +        /*
> > > +         * If the buffer was locked at the time of the resowner release,
> > > +         * release the lock now. This should only happen after errors.
> > > +         */
> > > +        if (ref->data.lockmode != BUFFER_LOCK_UNLOCK)
> > > +        {
> > > +            BufferDesc *buf = GetBufferDescriptor(buffer - 1);
> > > +
> > > +            HOLD_INTERRUPTS();    /* match the upcoming RESUME_INTERRUPTS */
> > > +            BufferLockUnlock(buffer, buf);
> > > +        }
> > > +
> > >          UnpinBufferNoOwner(GetBufferDescriptor(buffer - 1));
> > > +    }
> > >  }
> > >
> > >  Bit confusing that ResOwnerReleaseBufferBin() now releases locks as well.
> >
> > Do you have a better suggestion? I'll add a comment that makes that explicit,
> > but other than that I don't have a great idea. Renaming the whole buffer pin
> > mechanism seems pretty noisy.
> 
> ResOwnerReleaseBuffer()?
> 
> What do you mean renaming the whole buffer pin mechanism?

All the *PrivateRefCount* stuff arguably aught to be renamed...



> > > I presume you didn't do this because HeapTupleSatisifiesMVCC() for
> > > SNAPSHOT_MVCC calls the non-batch version (and, of course,
> > > HeapTupleSatisifiesVisibility() is the much more common case).
> > >
> > > if (TransactionIdIsValid(xid))
> > > {
> > >         if (BufferIsPermanent(buffer))
> > >         {
> > >                 /* NB: xid must be known committed here! */
> > >                 XLogRecPtr    commitLSN = TransactionIdGetCommitLSN(xid);
> > >
> > >                 if (XLogNeedsFlush(commitLSN) &&
> > >                         BufferGetLSNAtomic(buffer) < commitLSN)
> > >                 {
> > >                         /* not flushed and no LSN interlock, so don't
> > > set hint */
> > >                         return; false;
> > >                 }
> > >         }
> > > }
> > >
> > > Separately, I was thinking, should we assert here about having the
> > > right lock type?
> >
> > Not sure I get what assert of what locktype where?
> 
> In SetHintBitsExt() that we have share-exclusive or above.

We *don't* necessarily hold that though, it'll just be acquired by
BufferBeginSetHintBits().  Or do you mean in the SHB_ENABLED case?


> Or are there still callers with only a share lock?

Almost all of them, I think? We can't just call this with an unconditional
share-exclusive lock, since that'd destroy concurrency. Instead we just try to
upgrade to share-exclusive to set the hint bit. If we can't get
share-exclusive, we don't need to block, we just haven't set the hint bit.


> > > @@ -1628,6 +1701,9 @@ HeapTupleSatisfiesMVCCBatch(Snapshot snapshot,
> > > Buffer buffer,
> > > +    if (state == SHB_ENABLED)
> > > +        BufferFinishSetHintBits(buffer, true, true);
> > > +
> > >      return nvis;
> > >  }
> > >
> > > I wondered if it would be more natural for BufferBeginSetHintBits()
> > > and BufferFinishSetHintBits() to set SHB_INITIAL and SHB_DISABLED
> > > instead of having callers do it. But, I guess you don't do this
> > > because of gist and hash indexes using this for doing their own
> > > modifications.
> >
> > I thought about it. But yea, the different callers seemed to make that not
> > really useful. It'd also mean that we'd do an external function call for every
> > tuple, which I think would be prohibitively expensive.
> 
> I'm confused why this would mean more external function calls.

The visibility logic is in heapam_visibility.c, the locking for the buffer in
bufmgr.c? If the knowledge about SetHintBitsState lives in
BufferBeginSetHintBits(), we need to call it to do that check.


> > > --- a/src/backend/storage/freespace/freespace.c
> > > +++ b/src/backend/storage/freespace/freespace.c
> > > @@ -904,14 +904,22 @@ fsm_vacuum_page(Relation rel, FSMAddress addr,
> > >     max_avail = fsm_get_max_avail(page);
> > >     /*
> > > -    * Reset the next slot pointer. This encourages the use of low-numbered
> > > -    * pages, increasing the chances that a later vacuum can truncate the
> > > -    * relation.  We don't bother with a lock here, nor with marking the page
> > > -    * dirty if it wasn't already, since this is just a hint.
> > > +    * Try to reset the next slot pointer. This encourages the use of
> > > +    * low-numbered pages, increasing the chances that a later vacuum can
> > > +    * truncate the relation.  We don't bother with a lock here, nor with
> > > +    * marking the page dirty if it wasn't already, since this is just a hint.
> > > +    *
> > > +    * To be allowed to update the page without an exclusive lock, we have to
> > > +    * use the hint bit infrastructure.
> > >      */
> > >
> > > What the heck? This didn't even take a share lock before...
> >
> > It is insane. I do not understand how anybody thought this was ok.  I think I
> > probably should split this out into a separate commit, since it's so insane.
> 
> Aren't you going to backport having it take a lock? Then it can be
> separate (e.g. have it take a share lock in the "fix" commit and then
> this commit bumps it to share-exclusive).

I wasn't thinking we should backport that. It's been this way for ages,
without having caused known issues....


> > > diff --git a/src/backend/storage/freespace/fsmpage.c
> > > b/src/backend/storage/freesp>
> > > /*
> > > * Update the next-target pointer. Note that we do this even if we're only
> > > * holding a shared lock, on the grounds that it's better to use a shared
> > > * lock and get a garbled next pointer every now and then, than take the
> > > * concurrency hit of an exclusive lock.
> > >
> > > We appear to avoid the garbling now?
> >
> > I don't think so. Two backends concurrently can do fsm_search_avail() and
> > one backend might set a hint to a page that is already used up by the other
> > one. At least I think so?
> 
> Maybe I don't know what it meant by garbled, but I thought it was
> talking about two backends each trying to set fp_next_slot. If they
> now have to have a share-exclusive lock and they can't both have a
> share-exclusive lock at the same time, then it seems like that
> wouldn't be a problem. It sounds like you may be talking about a
> backend taking up the freespace of a page that is referred to by the
> fp_next_slot?

Yes, a version of the latter. The value that fp_next_slot will be set to can
be outdated by the time we actually set it, unless we do all of
fsm_search_avail() under some form of exclusive lock - clearly not something
desirable.

Greetings,

Andres Freund

  32. Re: Buffer locking is special (hints, checksums, AIO writes)

    Melanie Plageman <melanieplageman@gmail.com> — 2025-11-25T21:23:04Z

    On Tue, Nov 25, 2025 at 3:46 PM Andres Freund <andres@anarazel.de> wrote:
>
>
> > > > I presume you didn't do this because HeapTupleSatisifiesMVCC() for
> > > > SNAPSHOT_MVCC calls the non-batch version (and, of course,
> > > > HeapTupleSatisifiesVisibility() is the much more common case).
> > > >
> > > > if (TransactionIdIsValid(xid))
> > > > {
> > > >         if (BufferIsPermanent(buffer))
> > > >         {
> > > >                 /* NB: xid must be known committed here! */
> > > >                 XLogRecPtr    commitLSN = TransactionIdGetCommitLSN(xid);
> > > >
> > > >                 if (XLogNeedsFlush(commitLSN) &&
> > > >                         BufferGetLSNAtomic(buffer) < commitLSN)
> > > >                 {
> > > >                         /* not flushed and no LSN interlock, so don't
> > > > set hint */
> > > >                         return; false;
> > > >                 }
> > > >         }
> > > > }
> > > >
> > > > Separately, I was thinking, should we assert here about having the
> > > > right lock type?
> > >
> > > Not sure I get what assert of what locktype where?
> >
> > In SetHintBitsExt() that we have share-exclusive or above.
>
> We *don't* necessarily hold that though, it'll just be acquired by
> BufferBeginSetHintBits().  Or do you mean in the SHB_ENABLED case?

Yea, in the enabled case. Also, can't we skip the whole

    if (TransactionIdIsValid(xid))
    {
        if (BufferIsPermanent(buffer))
        {
            /* NB: xid must be known committed here! */
            XLogRecPtr    commitLSN = TransactionIdGetCommitLSN(xid);

            if (XLogNeedsFlush(commitLSN) &&
                BufferGetLSNAtomic(buffer) < commitLSN)
            {
                /* not flushed and no LSN interlock, so don't set hint */
                return;
            }
        }
    }

part if state is SHB_DISABLED?

- Melanie

  33. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2025-12-01T20:28:09Z

    Hi,

On 2025-11-21 12:52:38 -0500, Melanie Plageman wrote:
> > 0004: Use 64bit atomics for BufferDesc.state - at this point nothing uses the
> > additional bits yet, though.  Some annoying reformatting required to avoid
> > long lines.
>
> I noticed that the BUF_STATE_GET_REFCOUNT and BUF_STATE_GET_USAGECOUNT
> macros cast the return value to a uint32. We won't use the extra bits
> but we did bother to keep the macro result sized to the field width
> before so keeping it uint32 is probably more confusing now that state
> is 64 bit.

I can't really follow - why would we want to return a 64bit value if none of
the values ever can get anywhere near that big?


> Not related to this patch, but I noticed GetBufferDescriptor() calls
> for a uint32 and all the callers pretty much pass a signed int —
> wonder why it calls for uint32.

It's not strictly required - no Buffers exist bet INT32_MAX and
UINT32_MAX. However, GetBufferDescriptor() cannot be used for local buffers
(which would have a negative buffer id), therefore a uint32 is fine. Many of
the callers have dedicated branches to deal with local buffers and therefore
couldn't use a uint32.


> > 0006+0007: This is preparatory work for 0008, but also worthwhile on its
> > own. The private refcount stuff does show up in profiles. The reason it's
> > related is that without these changes the added information in 0008 makes that
> > worse.
>
> I found it slightly confusing that this commit appears to
> unnecessarily add the PrivateRefCountData struct (given that it
> doesn't need it to do the new parallel array thing). You could wait
> until you need it in 0008, but 0008 is big as it is, so it probably is
> fine where it is.

It seemed too annoying to whack the code around multiple times...


> in InitBufferManagerAccess(), why do you have
>
> memset(&PrivateRefCountArrayKeys, 0, sizeof(Buffer));
> seems like it should be
> memset(PrivateRefCountArrayKeys, 0, sizeof(PrivateRefCountArrayKeys));

Ugh, indeed.


> I wonder how easy it will be to keep the Buffer in sync between
> PrivateRefCountArrayKeys and the PrivateRefCountEntry — would a helper
> function help?

I don't think we really need it, the existing helper functions are where it
should be manipulated.


> ForgetPrivateRefCountEntry doesn’t clear the data member — but maybe
> it doesn’t matter...

It asserts that the fields are reset, which seems to suffice.


> in ReservePrivateRefCountEntry() there is a superfluous clear
>
> memset(&victim_entry->data, 0, sizeof(victim_entry->data));
> victim_entry->data.refcount = 0;

It's indeed superfluous today. I guess I put it in as a belt and suspenders
approach to future members... The compiler is easily be able to optimize that
redundancy away.


> 0007
> needs a commit message. overall seems fine though.

This is what I've since written:

    bufmgr: Add one-entry cache for private refcount

    The private refcount entry for a buffer is often looked up repeatedly for the
    same buffer, e.g. to pin and then unpin a buffer. Benchmarking shows that it's
    worthwhile to have a one-entry cache for that case. With that cache in place,
    it's worth splitting GetPrivateRefCountEntry() into a small inline
    portion (for the cache hit case) and an out-of-line helper for the rest.

    This is helpful for some workloads today, but becomes more important in an
    upcoming patch that will utilize the private refcount infrastructure to also
    store whether the buffer is currently locked, as that increases the rate of
    lookups substantially.



> You should probably capitalize the "c" of "count" in
> PrivateRefcountEntryLast to be consistent with the other names.

Ooops, yes.

Greetings,

Andres Freund

  34. Re: Buffer locking is special (hints, checksums, AIO writes)

    Melanie Plageman <melanieplageman@gmail.com> — 2025-12-01T20:41:07Z

    On Mon, Dec 1, 2025 at 3:28 PM Andres Freund <andres@anarazel.de> wrote:
>
> > I noticed that the BUF_STATE_GET_REFCOUNT and BUF_STATE_GET_USAGECOUNT
> > macros cast the return value to a uint32. We won't use the extra bits
> > but we did bother to keep the macro result sized to the field width
> > before so keeping it uint32 is probably more confusing now that state
> > is 64 bit.
>
> I can't really follow - why would we want to return a 64bit value if none of
> the values ever can get anywhere near that big?

Well, usagecount could never have reached anything close to a uint32
either, so I thought that this cast was meant to match the datatype. I
found it rather confusing otherwise.

> > 0007
> > needs a commit message. overall seems fine though.
>
> This is what I've since written:
>
>     bufmgr: Add one-entry cache for private refcount
>
>     The private refcount entry for a buffer is often looked up repeatedly for the
>     same buffer, e.g. to pin and then unpin a buffer. Benchmarking shows that it's
>     worthwhile to have a one-entry cache for that case. With that cache in place,
>     it's worth splitting GetPrivateRefCountEntry() into a small inline
>     portion (for the cache hit case) and an out-of-line helper for the rest.
>
>     This is helpful for some workloads today, but becomes more important in an
>     upcoming patch that will utilize the private refcount infrastructure to also
>     store whether the buffer is currently locked, as that increases the rate of
>     lookups substantially.

Sounds good based on what I recall of the patch.

- Melanie

  35. Re: Buffer locking is special (hints, checksums, AIO writes)

    Heikki Linnakangas <hlinnaka@iki.fi> — 2025-12-02T08:01:06Z

    On 25/11/2025 22:46, Andres Freund wrote:
> On 2025-11-25 15:02:02 -0500, Melanie Plageman wrote:
>> On Tue, Nov 25, 2025 at 11:54 AM Andres Freund <andres@anarazel.de> wrote:
>>>> --- a/src/backend/storage/freespace/freespace.c
>>>> +++ b/src/backend/storage/freespace/freespace.c
>>>> @@ -904,14 +904,22 @@ fsm_vacuum_page(Relation rel, FSMAddress addr,
>>>>      max_avail = fsm_get_max_avail(page);
>>>>      /*
>>>> -    * Reset the next slot pointer. This encourages the use of low-numbered
>>>> -    * pages, increasing the chances that a later vacuum can truncate the
>>>> -    * relation.  We don't bother with a lock here, nor with marking the page
>>>> -    * dirty if it wasn't already, since this is just a hint.
>>>> +    * Try to reset the next slot pointer. This encourages the use of
>>>> +    * low-numbered pages, increasing the chances that a later vacuum can
>>>> +    * truncate the relation.  We don't bother with a lock here, nor with
>>>> +    * marking the page dirty if it wasn't already, since this is just a hint.
>>>> +    *
>>>> +    * To be allowed to update the page without an exclusive lock, we have to
>>>> +    * use the hint bit infrastructure.
>>>>       */
>>>>
>>>> What the heck? This didn't even take a share lock before...
>>>
>>> It is insane. I do not understand how anybody thought this was ok.  I think I
>>> probably should split this out into a separate commit, since it's so insane.
>>
>> Aren't you going to backport having it take a lock? Then it can be
>> separate (e.g. have it take a share lock in the "fix" commit and then
>> this commit bumps it to share-exclusive).
> 
> I wasn't thinking we should backport that. It's been this way for ages,
> without having caused known issues....

I wrote that originally :-). The FSM code always treats the FSM page 
contents as potentially corrupted garbage. FSM page updates are not 
WAL-logged, for starters. And as the comment says, the next slot pointer 
is just a hint for where within the page to start looking for free space.

Page checksums were added later, and now we know about the problems of 
modifying a page a write() is in progress, on some filesystems with 
filesystem-level checksums. So it's a good idea to tighten it up, but it 
was fine back then.

>>>> diff --git a/src/backend/storage/freespace/fsmpage.c
>>>> b/src/backend/storage/freesp>
>>>> /*
>>>> * Update the next-target pointer. Note that we do this even if we're only
>>>> * holding a shared lock, on the grounds that it's better to use a shared
>>>> * lock and get a garbled next pointer every now and then, than take the
>>>> * concurrency hit of an exclusive lock.
>>>>
>>>> We appear to avoid the garbling now?
>>>
>>> I don't think so. Two backends concurrently can do fsm_search_avail() and
>>> one backend might set a hint to a page that is already used up by the other
>>> one. At least I think so?
>>
>> Maybe I don't know what it meant by garbled, but I thought it was
>> talking about two backends each trying to set fp_next_slot. If they
>> now have to have a share-exclusive lock and they can't both have a
>> share-exclusive lock at the same time, then it seems like that
>> wouldn't be a problem. It sounds like you may be talking about a
>> backend taking up the freespace of a page that is referred to by the
>> fp_next_slot?
> 
> Yes, a version of the latter. The value that fp_next_slot will be set to can
> be outdated by the time we actually set it, unless we do all of
> fsm_search_avail() under some form of exclusive lock - clearly not something
> desirable.

I'm pretty sure the "garbled" in the comment means the former, not the 
latter. I.e. it means that the pointer itself might become garbage. 
Would be good to update the comment if that's no longer possible.

But speaking of that: why do we not allow two processes to concurrently 
set hint bits on a page anymore?

- Heikki

  36. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2025-12-02T13:20:03Z

    Hi,

On 2025-12-02 10:01:06 +0200, Heikki Linnakangas wrote:
> On 25/11/2025 22:46, Andres Freund wrote:
> > > > > diff --git a/src/backend/storage/freespace/fsmpage.c
> > > > > b/src/backend/storage/freesp>
> > > > > /*
> > > > > * Update the next-target pointer. Note that we do this even if we're only
> > > > > * holding a shared lock, on the grounds that it's better to use a shared
> > > > > * lock and get a garbled next pointer every now and then, than take the
> > > > > * concurrency hit of an exclusive lock.
> > > > > 
> > > > > We appear to avoid the garbling now?
> > > > 
> > > > I don't think so. Two backends concurrently can do fsm_search_avail() and
> > > > one backend might set a hint to a page that is already used up by the other
> > > > one. At least I think so?
> > > 
> > > Maybe I don't know what it meant by garbled, but I thought it was
> > > talking about two backends each trying to set fp_next_slot. If they
> > > now have to have a share-exclusive lock and they can't both have a
> > > share-exclusive lock at the same time, then it seems like that
> > > wouldn't be a problem. It sounds like you may be talking about a
> > > backend taking up the freespace of a page that is referred to by the
> > > fp_next_slot?
> > 
> > Yes, a version of the latter. The value that fp_next_slot will be set to can
> > be outdated by the time we actually set it, unless we do all of
> > fsm_search_avail() under some form of exclusive lock - clearly not something
> > desirable.
> 
> I'm pretty sure the "garbled" in the comment means the former, not the
> latter. I.e. it means that the pointer itself might become garbage. Would be
> good to update the comment if that's no longer possible.

Hm. I thought we had always assumed that 4byte values can be read/written
tear-free. Hence thinking that garbled couldn't refer to reading entire
garbage due to a concurrent write.


> But speaking of that: why do we not allow two processes to concurrently set
> hint bits on a page anymore?

It'd make the locking a lot more complicated without much of a benefit.

The new share-exclusive lock mode only requires one additional bit of lock
state, for the single allowed holder. If we wanted a new lockmode that
prevented the page from being written out concurrently, but could be held
multiple times, we'd need at least MAX_BACKENDS bits for the lock level
allowing hint bits to be set and another lock level to acquire while writing
out the buffer.

At the same time, there seems to be little benefit in setting hint bits on a
page concurrently. A very common case is that the same hint bit(s) would be
set by multiple backends, we don't gain anything from that. And in the cases
where hint bits were intended to be set for different tuples, the window in
which that is not allowed is very narrow, and the cost of not setting right in
that moment is pretty small and the cost of not setting the hint bit right
then and there isn't high.

Makes sense?

Greetings,

Andres Freund

  37. Re: Buffer locking is special (hints, checksums, AIO writes)

    Heikki Linnakangas <hlinnaka@iki.fi> — 2025-12-02T13:38:34Z

    On 02/12/2025 15:20, Andres Freund wrote:
> On 2025-12-02 10:01:06 +0200, Heikki Linnakangas wrote:
>> But speaking of that: why do we not allow two processes to concurrently set
>> hint bits on a page anymore?
> 
> It'd make the locking a lot more complicated without much of a benefit.
> 
> The new share-exclusive lock mode only requires one additional bit of lock
> state, for the single allowed holder. If we wanted a new lockmode that
> prevented the page from being written out concurrently, but could be held
> multiple times, we'd need at least MAX_BACKENDS bits for the lock level
> allowing hint bits to be set and another lock level to acquire while writing
> out the buffer.
> 
> At the same time, there seems to be little benefit in setting hint bits on a
> page concurrently. A very common case is that the same hint bit(s) would be
> set by multiple backends, we don't gain anything from that. And in the cases
> where hint bits were intended to be set for different tuples, the window in
> which that is not allowed is very narrow, and the cost of not setting right in
> that moment is pretty small and the cost of not setting the hint bit right
> then and there isn't high.
> 
> Makes sense?

Yep, makes sense. Would be good to put that in a comment somewhere, and 
in the commit message. "We could allow multiple backends to set hint 
bits concurrently, but it'd make the lock implementation more complicated"

- Heikki

  38. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2025-12-03T00:47:35Z

    Hi,

On 2025-11-25 11:54:00 -0500, Andres Freund wrote:
> Thanks a lot for that detailed review!  A few questions and comments, before I
> try to address the comments in the next version.

Here's that new new version, with the following changes

- Some more micro-optimizations, most importantly adding a commit that doesn't
  initialize the delay in LockBufHdr() unless needed. With those I don't see a
  consistent slowdown anymore (slight speedup on one workstation, slight
  slowdown on another, in an absurdly adverse workload)

- Tried to address Melanie's feedback, with some exceptions (some noted below,
  but I also need to make another pass through the reviews)

- re-implemented AssertNotCatalogBufferLock() in the new world

- Substantially expanded comments around setting hint bits (in buffer/README,
  heapam_visibility.c and bufmgr.c)

- split out the change to fsm_vacuum_page() to start to lock the page into is
  own commit

- reordered patch series so that smaller changes are before the 64bit-state
  and "Implement buffer content locks independently of" commits, so they can
  be committed while we finish cleaning the later changes

- I didn't invest much in cleaning up the later patches ("Don't copy pages
  while writing out" and "Make UnlockReleaseBuffer() more efficient") yet,
  wanted to focus on the earlier patches first


Todo:

- still need to rename ResOwnerReleaseBufferPin(). Wondering about what to
  rename ResourceOwnerDesc.name to. "buffer ownership" maybe? Not great...

- gistkillitems() complaint by Melanie

- amortize vs batch vs SetHintBits comment + SHB_* names

- for the next version I'll remove the BATCHMVCC_FEWER_ARGS conditionals from
  0010. I don't love needing BatchMVCCState but I don't really see an
  alternative, the performance difference is pretty persistent.


Questions:
- ForEachLWLockHeldByMe() and LWLockDisown() aren't used anymore, should we
  remove them?


Greetings,

Andres Freund

  39. Re: Buffer locking is special (hints, checksums, AIO writes)

    Peter Geoghegan <pg@bowt.ie> — 2025-12-03T01:12:14Z

    On Tue, Dec 2, 2025 at 7:47 PM Andres Freund <andres@anarazel.de> wrote:
> On 2025-11-25 11:54:00 -0500, Andres Freund wrote:
> > Thanks a lot for that detailed review!  A few questions and comments, before I
> > try to address the comments in the next version.
>
> Here's that new new version, with the following changes

_bt_check_unique will hold an exclusive buffer lock on the page being
LP_DEAD-set in the vast majority of cases. Should we expect your
changes to have no effect at all in that common case?

The BTP_HAS_GARBAGE flag is deprecated these days; we basically don't
use it anymore. How much value might there be in avoiding setting
BTP_HAS_GARBAGE as a way of being able to use BufferSetHintBits16 more
often in nbtree?

-- 
Peter Geoghegan

  40. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2025-12-03T01:18:38Z

    Hi,

On 2025-12-02 20:12:14 -0500, Peter Geoghegan wrote:
> On Tue, Dec 2, 2025 at 7:47 PM Andres Freund <andres@anarazel.de> wrote:
> > On 2025-11-25 11:54:00 -0500, Andres Freund wrote:
> > > Thanks a lot for that detailed review!  A few questions and comments, before I
> > > try to address the comments in the next version.
> >
> > Here's that new new version, with the following changes
> 
> _bt_check_unique will hold an exclusive buffer lock on the page being
> LP_DEAD-set in the vast majority of cases. Should we expect your
> changes to have no effect at all in that common case?

If we already have an exclusive lock, BufferBeginSetHintBits() will quickly
return true and won't ever return false.


> The BTP_HAS_GARBAGE flag is deprecated these days; we basically don't
> use it anymore. How much value might there be in avoiding setting
> BTP_HAS_GARBAGE as a way of being able to use BufferSetHintBits16 more
> often in nbtree?

None of the MarkBufferDirtyHint() cases in nbtree that had to be modified
looked like they would benefit from BufferSetHintBits16(), since they will
typically modify the page multiple times.  But maybe I'm just misunderstanding
what you mean?

Greetings,

Andres Freund

  41. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2025-12-03T16:03:40Z

    Hi,

On 2025-12-02 19:47:35 -0500, Andres Freund wrote:
> On 2025-11-25 11:54:00 -0500, Andres Freund wrote:
> > Thanks a lot for that detailed review!  A few questions and comments, before I
> > try to address the comments in the next version.
> 
> Here's that new new version, with the following changes
> 
> - Some more micro-optimizations, most importantly adding a commit that doesn't
>   initialize the delay in LockBufHdr() unless needed. With those I don't see a
>   consistent slowdown anymore (slight speedup on one workstation, slight
>   slowdown on another, in an absurdly adverse workload)
> 
> - Tried to address Melanie's feedback, with some exceptions (some noted below,
>   but I also need to make another pass through the reviews)
> 
> - re-implemented AssertNotCatalogBufferLock() in the new world
> 
> - Substantially expanded comments around setting hint bits (in buffer/README,
>   heapam_visibility.c and bufmgr.c)
> 
> - split out the change to fsm_vacuum_page() to start to lock the page into is
>   own commit
> 
> - reordered patch series so that smaller changes are before the 64bit-state
>   and "Implement buffer content locks independently of" commits, so they can
>   be committed while we finish cleaning the later changes
> 
> - I didn't invest much in cleaning up the later patches ("Don't copy pages
>   while writing out" and "Make UnlockReleaseBuffer() more efficient") yet,
>   wanted to focus on the earlier patches first
> 
> 
> Todo:
> 
> - still need to rename ResOwnerReleaseBufferPin(). Wondering about what to
>   rename ResourceOwnerDesc.name to. "buffer ownership" maybe? Not great...
> 
> - gistkillitems() complaint by Melanie
> 
> - amortize vs batch vs SetHintBits comment + SHB_* names
> 
> - for the next version I'll remove the BATCHMVCC_FEWER_ARGS conditionals from
>   0010. I don't love needing BatchMVCCState but I don't really see an
>   alternative, the performance difference is pretty persistent.
> 
> 
> Questions:
> - ForEachLWLockHeldByMe() and LWLockDisown() aren't used anymore, should we
>   remove them?

I'm planning to work on committing 0001, 0002, 0003, 0008 soon-ish, unless
somebody sees a reason to hold off on that.  After that I think 0005, 0006
would be next.  I think 0004 is a clear improvement, but nobody has looked at
it yet...

For 0007, I wished ConditionalLockBuffer() accepted the lock level, there's no
point in waiting for the lock in the use case. I'm on the fence about whether
it's worth changing the ~12 users of ConditionalLockBuffer()...

Greetings,

Andres Freund

  42. Re: Buffer locking is special (hints, checksums, AIO writes)

    Heikki Linnakangas <hlinnaka@iki.fi> — 2025-12-17T09:25:50Z

    On 03/12/2025 02:47, Andres Freund wrote:
> On 2025-11-25 11:54:00 -0500, Andres Freund wrote:
>> Thanks a lot for that detailed review!  A few questions and comments, before I
>> try to address the comments in the next version.
> 
> Here's that new new version, with the following changes
> 
> - Some more micro-optimizations, most importantly adding a commit that doesn't
>    initialize the delay in LockBufHdr() unless needed. With those I don't see a
>    consistent slowdown anymore (slight speedup on one workstation, slight
>    slowdown on another, in an absurdly adverse workload)

+1

I'm comparing the patched LockBufHdr() with LWLockWaitListLock(), which 
does pretty much the same thing, and LWLockWaitListLock() already did 
the initialization of the delay that way. But there are some small 
differences:

- LockBufHdr() uses unlikely() in the initial attempt, 
LWLockWaitListLock() does not
- LWLockWaitListLock() uses pg_atomic_read_u32() after spinning, 
LockBufHdr() retries directly with pg_atomic_fetch_or_u32().

Are there reasons for the differences, or is it just that they were 
developed separately and ended up looking slightly different?

- Heikki

  43. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2025-12-17T14:54:32Z

    Hi,

On 2025-12-17 11:25:50 +0200, Heikki Linnakangas wrote:
> On 03/12/2025 02:47, Andres Freund wrote:
> > On 2025-11-25 11:54:00 -0500, Andres Freund wrote:
> > > Thanks a lot for that detailed review!  A few questions and comments, before I
> > > try to address the comments in the next version.
> > 
> > Here's that new new version, with the following changes
> > 
> > - Some more micro-optimizations, most importantly adding a commit that doesn't
> >    initialize the delay in LockBufHdr() unless needed. With those I don't see a
> >    consistent slowdown anymore (slight speedup on one workstation, slight
> >    slowdown on another, in an absurdly adverse workload)
> 
> +1
> 
> I'm comparing the patched LockBufHdr() with LWLockWaitListLock(), which does
> pretty much the same thing, and LWLockWaitListLock() already did the
> initialization of the delay that way. But there are some small differences:
> 
> - LockBufHdr() uses unlikely() in the initial attempt, LWLockWaitListLock()
> does not

I think we probably ought to do that in LWLockWaitListLock() too.


> - LWLockWaitListLock() uses pg_atomic_read_u32() after spinning,
> LockBufHdr() retries directly with pg_atomic_fetch_or_u32().

I think here LWLockWaitListLock() is likely right - but it seems like a change
to LockBufHdr() that I would probably make in a separate commit?


> Are there reasons for the differences, or is it just that they were
> developed separately and ended up looking slightly different?

I think it's just the latter...


Thanks for reviewing,

Andres Freund

  44. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2025-12-18T17:03:36Z

    Hi,

On 2025-12-17 09:54:32 -0500, Andres Freund wrote:
> On 2025-12-17 11:25:50 +0200, Heikki Linnakangas wrote:
> > - LWLockWaitListLock() uses pg_atomic_read_u32() after spinning,
> > LockBufHdr() retries directly with pg_atomic_fetch_or_u32().
> 
> I think here LWLockWaitListLock() is likely right - but it seems like a change
> to LockBufHdr() that I would probably make in a separate commit?

FWIW, I couldn't come up with a scenario where it makes a performance
difference - exclusive content locks just aren't *that* frequent. And because
of that the wait list lock doesn't have similar contention as some non-content
lwlocks (like XidGenLock). The most extreme workload I could think of was
pgbench hammering a single sequence across many sessions. While the exclusive
locks show up in wait events, the buffer header spinlock itself doesn't..

So I'm inclined to not change anything about this for now.

Greetings,

Andres Freund

  45. Re: Buffer locking is special (hints, checksums, AIO writes)

    Heikki Linnakangas <hlinnaka@iki.fi> — 2025-12-18T17:20:38Z

    On 18/12/2025 19:03, Andres Freund wrote:
> Hi,
> 
> On 2025-12-17 09:54:32 -0500, Andres Freund wrote:
>> On 2025-12-17 11:25:50 +0200, Heikki Linnakangas wrote:
>>> - LWLockWaitListLock() uses pg_atomic_read_u32() after spinning,
>>> LockBufHdr() retries directly with pg_atomic_fetch_or_u32().
>>
>> I think here LWLockWaitListLock() is likely right - but it seems like a change
>> to LockBufHdr() that I would probably make in a separate commit?
> 
> FWIW, I couldn't come up with a scenario where it makes a performance
> difference - exclusive content locks just aren't *that* frequent. And because
> of that the wait list lock doesn't have similar contention as some non-content
> lwlocks (like XidGenLock). The most extreme workload I could think of was
> pgbench hammering a single sequence across many sessions. While the exclusive
> locks show up in wait events, the buffer header spinlock itself doesn't..
> 
> So I'm inclined to not change anything about this for now.

Ok. My thinking was just that LockBufHdr() and LWLockWaitListLock() 
should be consistent with each other. Otherwise anyone reading the code 
will ask the question "why are they different?". They're the only two 
things using the spin delay mechanism in our codebase, in addition to 
actual spinlocks.

BTW, I wonder if it would be worthwhile to have an inlineable fast-path 
of LockBufHdr() for the common case that the lock is free? I see that 
UnlockBufHdr() is already a static inline function.

- Heikki

  46. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2025-12-18T22:06:43Z

    Hi,

On 2025-12-18 19:20:38 +0200, Heikki Linnakangas wrote:
> On 18/12/2025 19:03, Andres Freund wrote:
> > On 2025-12-17 09:54:32 -0500, Andres Freund wrote:
> > > On 2025-12-17 11:25:50 +0200, Heikki Linnakangas wrote:
> > > > - LWLockWaitListLock() uses pg_atomic_read_u32() after spinning,
> > > > LockBufHdr() retries directly with pg_atomic_fetch_or_u32().
> > >
> > > I think here LWLockWaitListLock() is likely right - but it seems like a change
> > > to LockBufHdr() that I would probably make in a separate commit?
> >
> > FWIW, I couldn't come up with a scenario where it makes a performance
> > difference - exclusive content locks just aren't *that* frequent. And because
> > of that the wait list lock doesn't have similar contention as some non-content
> > lwlocks (like XidGenLock). The most extreme workload I could think of was
> > pgbench hammering a single sequence across many sessions. While the exclusive
> > locks show up in wait events, the buffer header spinlock itself doesn't..
> >
> > So I'm inclined to not change anything about this for now.
>
> Ok. My thinking was just that LockBufHdr() and LWLockWaitListLock() should
> be consistent with each other. Otherwise anyone reading the code will ask
> the question "why are they different?". They're the only two things using
> the spin delay mechanism in our codebase, in addition to actual spinlocks.

I guess for me it didn't really seem like this patch's job to fix
that. Regardless of that, here's a version that tries to make them more
similar.

I did check, adding a likely() to LWLockWaitListLock()'s break does improve
code generation (verified by looking at the generated code) and seems to
improve performance in some very extreme workloads (e.g. [1]) a bit.

I'll try to come up with a combined patch that applies the optimizations in
LWLockWaitListLock() and LockBufHdr() to each other.


> BTW, I wonder if it would be worthwhile to have an inlineable fast-path of
> LockBufHdr() for the common case that the lock is free? I see that
> UnlockBufHdr() is already a static inline function.

I tried that a while ago and couldn't see any improvement, I think because all
the performance relevant callers are in bufmgr.c and thus can already inline
[parts of] the implementation.  I guess you could make the generated code a
bit smaller if you use pg_noinline on the slowpath, but that seems like a
separate project / effort to me.

Greetings,

Andres Freund


[1] Many connections doing
DO $do$
    BEGIN
        FOR i IN 1 .. 1000 LOOP
            PERFORM txid_current();
	    COMMIT;
	END LOOP;
     END;
$do$;

  47. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2025-12-18T23:39:00Z

    Hi,

On 2025-12-18 17:06:43 -0500, Andres Freund wrote:
> I'll try to come up with a combined patch that applies the optimizations in
> LWLockWaitListLock() and LockBufHdr() to each other.

Attached is a rebased version of this patch series, with the above patch as
0001.

The later patches have a lot, but not yet all, of Melanie's feedback
addressed. I mainly am sending them because cfbot wanted a rebase anyway.

I'm mainly not yet happy with how 0007 changes buf_internals.h, some more
comment work is also needed. 0009 & 0010 still are more for POC than ready to
be in-depth reviewed.

Greetings,

Andres Freund

  48. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-01-09T00:29:35Z

    Hi,

I pushed what was 0001, 0002 in v8. Attached is an updated set of patches for
the rest.

Main changes:

- the explanation in "heapam: Use exclusive lock on old page in CLUSTER" as to
  why it's problematic to not set hint bits wasn't quite right. I've updated
  it:

    heapam: Use exclusive lock on old page in CLUSTER

    To be able to guarantee that we can set the hint bit, acquire an exclusive
    lock on the old buffer. This is required as a future commit will only allow
    hint bits to be set with a new lock level, which is acquired as-needed in a
    non-blocking fashion.

    We need the hint bits, set in heapam_relation_copy_for_cluster() ->
    HeapTupleSatisfiesVacuum(), to be set, as otherwise reform_and_rewrite_tuple()
    -> rewrite_heap_tuple() will get confused. Specifically, rewrite_heap_tuple()
    checks for HEAP_XMAX_INVALID in the old tuple to determine whether to check
    the old-to-new mapping hash table.

- I added a patch that inverts the meaning of LW_FLAG_RELEASE_OK, to make the
  equivalent code for content locks easier. For buffer content locks we reset
  the flags when invalidating, and otherwise we'd either need to not have the
  equivalent of LW_FLAG_RELEASE_OK in the flag mask or explicitly add it after
  making the buffer valid.

  I think it's also nicer this way round, because we e.g. can assert that
  there are no pending wakeups when invalidating a buffer.

- I added a patch to reorganize some of the flags stuff in buf_internals.h, to
  make the later patches cleaner. In particular flags are now defined with a
  macro so that changing at which offset flag bits are doesn't require
  touching every single flag value.

- For the main commit, the reorganized flag stuff removed one of the remaining
  FIXMEs.

- I removed the performance instrumentation stuff from the batch visibility
  commit.


I think 0001, 0002, 0003 can be committed. 0004, 0005 are new and probably
could use a sanity check.  0006 hasn't changed much and is imo pretty much
ready, but should be pushed together with 0007.  0007 is getting close, I
think.  0008-0010 need a bit more work, but I think that can wait until 0007
has been pushed.

For 0008, it'd be nice if somebody could look at the way buf_internals.h now
looks.

The only remaining FIXME in 0008 is about the the reuse of
PGPROC->{lwWaiting,lwWaitMode,lwWaitLink}. I think reusing them for content
locks isn't pretty, but it's probably not worth duplicating them. Thoughts?

Greetings,

Andres Freund

  49. Re: Buffer locking is special (hints, checksums, AIO writes)

    Kirill Reshke <reshkekirill@gmail.com> — 2026-01-09T08:08:43Z

    Hi!

On Fri, 9 Jan 2026 at 05:29, Andres Freund <andres@anarazel.de> wrote:
>
> I think 0001, 0002, 0003 can be committed. 0004, 0005 are new and probably

0001 LGTM.

I also did look at 0002, looks sane.

Other patches are out of my comprehension for now, I did not review them .



-- 
Best regards,
Kirill Reshke

  50. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-01-12T17:45:03Z

    Hi,

On 2026-01-09 13:08:43 +0500, Kirill Reshke wrote:
> On Fri, 9 Jan 2026 at 05:29, Andres Freund <andres@anarazel.de> wrote:
> >
> > I think 0001, 0002, 0003 can be committed. 0004, 0005 are new and probably
> 
> 0001 LGTM.
> 
> I also did look at 0002, looks sane.

Thanks for looking!

I've pushed 0001/0002 now.  I fixed a typo or two since the last published
version.

I'm doing another pass through 0003 and will push that if I don't find
anything significant.

Also working on doing comment polishing of the later patches, found a few
things, but not quite enough to be worth reposting yet.

Greetings,

Andres Freund

  51. Re: Buffer locking is special (hints, checksums, AIO writes)

    Melanie Plageman <melanieplageman@gmail.com> — 2026-01-12T22:27:30Z

    On Mon, Jan 12, 2026 at 12:45 PM Andres Freund <andres@anarazel.de> wrote:
>
> Also working on doing comment polishing of the later patches, found a few
> things, but not quite enough to be worth reposting yet.

I looked at 0004 and 0005 and re-looked at 0006 - 0007.

For 0004, I think you should clarify the commit message a bit. I had
trouble understanding when WAKE_IN_PROGRESS is set. So, before
RELEASE_OK was set all the time except when a process woke up and
hadn't run yet. Now, WAKE_IN_PROGRESS is only set when a process is
woken up but hasn't run yet. Personally, I just needed a bit more
specificity (maybe even a bit more formality and grammatically
correctness) from the commit message to get it.

I agree that separating 0005 is helpful.

0007 looks basically fine to me. I'd comb through it with an AI tool
to catch a few nits I saw like an outdated reference to RELEASE_OK and
a missing word in the commit message, etc.

Otherwise, I mostly looked to see if the wakeup semantics seemed right
and if anything jumped out at me while skimming (i.e. I didn't go
through every line with a fine-toothed comb).

The two things I came up with were:

I wondered why this was needed (i.e. why it wasn't needed before)

@@ -6688,7 +7428,25 @@ ResOwnerReleaseBufferPin(Datum res)
     if (BufferIsLocal(buffer))
         UnpinLocalBufferNoOwner(buffer);
     else
+    {
+        PrivateRefCountEntry *ref;
+
+        ref = GetPrivateRefCountEntry(buffer, false);
+
+        /*
+         * If the buffer was locked at the time of the resowner release,
+         * release the lock now. This should only happen after errors.
+         */
+        if (ref->data.lockmode != BUFFER_LOCK_UNLOCK)
+        {
+            BufferDesc *buf = GetBufferDescriptor(buffer - 1);
+
+            HOLD_INTERRUPTS();    /* match the upcoming RESUME_INTERRUPTS */
+            BufferLockUnlock(buffer, buf);
+        }
+
         UnpinBufferNoOwner(GetBufferDescriptor(buffer - 1));
+    }
 }

 is it related to your comment in the commit message

2) Error recovery for content locks is implemented as part of the
already existing private-refcount tracking mechanism in combination
with resowners?

As for your FIXMEs,
+    /*
+     * FIXME: This is reusing the lwlock fields. That's not a correctness
+     * issue, a backend can't wait for both an lwlock and a buffer content
+     * lock at the same time. However, it seems pretty ugly, particularly
+     * given that the field names have an lw* prefix. But duplicating the
+     * fields also seems somewhat superfluous.
+     */

personally I can live with reusing the lwlock fields now that it's
fairly well documented.

+    /* XXX: combine with fetch_and above? */
+    UnlockBufHdr(buf_hdr);

Are you thinking about adding a helper that stops waiting and unlocks?

> Perhaps move the locking code into a buffer_locking.h or such? Needs to be inline functions for efficiency unfortunately.

So you mean put all of the static buffer locking functions you added
to bufmgr.c inline into a header file?

bufmgr.c is super long anyway, so it's not like making it separate
makes the file manageable. On the other hand, it's probably better to
not keep making it worse. For example, I find it really annoying that
the helper function prototypes for res owner and ref count related
functions are grouped before their implementations and then below that
there is another seemingly arbitrary group of prototypes and then
their implementations. Like, what is the logic there?

- Melanie

  52. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-01-12T23:22:37Z

    Hi,

On 2026-01-12 17:27:30 -0500, Melanie Plageman wrote:
> On Mon, Jan 12, 2026 at 12:45 PM Andres Freund <andres@anarazel.de> wrote:
> >
> > Also working on doing comment polishing of the later patches, found a few
> > things, but not quite enough to be worth reposting yet.
>
> I looked at 0004 and 0005 and re-looked at 0006 - 0007.
>
> For 0004, I think you should clarify the commit message a bit. I had
> trouble understanding when WAKE_IN_PROGRESS is set. So, before
> RELEASE_OK was set all the time except when a process woke up and
> hadn't run yet. Now, WAKE_IN_PROGRESS is only set when a process is
> woken up but hasn't run yet. Personally, I just needed a bit more
> specificity (maybe even a bit more formality and grammatically
> correctness) from the commit message to get it.

Is this better?
    lwlock: Invert meaning of LW_FLAG_RELEASE_OK

    Previously, a flag was set to indicate that a lock release should wake up
    waiters. Since waking waiters is the default behavior in the majority of
    cases, this logic has been inverted. The new LW_FLAG_WAKE_IN_PROGRESS flag is
    now set iff wakeups are explicitly inhibited.

    The motivation for this change is that in an upcoming commit, content locks
    will be implemented independently of lwlocks, with the lock state stored as
    part of BufferDesc.state. As all of a buffer's flags are cleared when the
    buffer is invalidated, without this change we would have to re-add the
    RELEASE_OK flag after clearing the flags; otherwise, the next lock release
    would not wake waiters.

    It seems good to keep the implementation of lwlocks and buffer content locks
    as similar as reasonably possible.

    Discussion: https://postgr.es/m/4csodkvvfbfloxxjlkgsnl2lgfv2mtzdl7phqzd4jxjadxm4o5@usw7feyb5bzf



> I agree that separating 0005 is helpful.

Kewl.


> 0007 looks basically fine to me. I'd comb through it with an AI tool
> to catch a few nits I saw like an outdated reference to RELEASE_OK and
> a missing word in the commit message, etc.

Found a few that way and with some manual searching.


> Otherwise, I mostly looked to see if the wakeup semantics seemed right
> and if anything jumped out at me while skimming (i.e. I didn't go
> through every line with a fine-toothed comb).
>
> The two things I came up with were:
>
> I wondered why this was needed (i.e. why it wasn't needed before)

> @@ -6688,7 +7428,25 @@ ResOwnerReleaseBufferPin(Datum res)
>      if (BufferIsLocal(buffer))
>          UnpinLocalBufferNoOwner(buffer);
>      else
> +    {
> +        PrivateRefCountEntry *ref;
> +
> +        ref = GetPrivateRefCountEntry(buffer, false);
> +
> +        /*
> +         * If the buffer was locked at the time of the resowner release,
> +         * release the lock now. This should only happen after errors.
> +         */
> +        if (ref->data.lockmode != BUFFER_LOCK_UNLOCK)
> +        {
> +            BufferDesc *buf = GetBufferDescriptor(buffer - 1);
> +
> +            HOLD_INTERRUPTS();    /* match the upcoming RESUME_INTERRUPTS */
> +            BufferLockUnlock(buffer, buf);
> +        }
> +
>          UnpinBufferNoOwner(GetBufferDescriptor(buffer - 1));
> +    }
>  }

It's needed because previously content locks were released as part of the
LWLockReleaseAll() that are sprinkled across various error recovery paths. Now
that content locks aren't implemented via lwlocks anymore, something new is needed.


>  is it related to your comment in the commit message

> 2) Error recovery for content locks is implemented as part of the
> already existing private-refcount tracking mechanism in combination
> with resowners?

Yes.


> As for your FIXMEs,
> +    /*
> +     * FIXME: This is reusing the lwlock fields. That's not a correctness
> +     * issue, a backend can't wait for both an lwlock and a buffer content
> +     * lock at the same time. However, it seems pretty ugly, particularly
> +     * given that the field names have an lw* prefix. But duplicating the
> +     * fields also seems somewhat superfluous.
> +     */
>
> personally I can live with reusing the lwlock fields now that it's
> fairly well documented.

Cool. That's the conclusion I also came to. So unless somebody pipes up soon,
I'll remove the FIXME from the commit message and code.


> +    /* XXX: combine with fetch_and above? */
> +    UnlockBufHdr(buf_hdr);
>
> Are you thinking about adding a helper that stops waiting and unlocks?

I'm not sure what you mean by that? Just whether I plan to implement the
FIXME?



> > Perhaps move the locking code into a buffer_locking.h or such? Needs to be inline functions for efficiency unfortunately.
>
> So you mean put all of the static buffer locking functions you added
> to bufmgr.c inline into a header file?

Yes, that's what I was wondering about.


> bufmgr.c is super long anyway, so it's not like making it separate
> makes the file manageable. On the other hand, it's probably better to
> not keep making it worse.

Yea. OTOH I don't know if a header that's just included by one file is really
an improvement :/


> For example, I find it really annoying that the helper function prototypes
> for res owner and ref count related functions are grouped before their
> implementations and then below that there is another seemingly arbitrary
> group of prototypes and then their implementations. Like, what is the logic
> there?

I agree it's pretty awful this way. I don't know how the hell that happened,
despite probably being the party to blame (4b4b680c3d6d). Nobody in that
thread commented upon it, it was that way starting in the first version. Odd.
I guess I should propose fixing that :/

Greetings,

Andres Freund

  53. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-01-13T00:33:56Z

    Hi,

On 2026-01-12 12:45:03 -0500, Andres Freund wrote:
> I'm doing another pass through 0003 and will push that if I don't find
> anything significant.

Done, after adjust two comments in minor ways.


> Also working on doing comment polishing of the later patches, found a few
> things, but not quite enough to be worth reposting yet.

Here are the remaining commits, with a bit of polish:

- fixed references to old names in some places (lwlocks, release_ok)

- Aded an assert that we don't already hold a lock in BufferLockConditional()

- typo and grammar fixes

- updated the commit message of the LW_FLAG_RELEASE_OK, as "requested" by
  Melanie. I hope this explains the situation better.

- added a commit that renames ResOwnerReleaseBufferPin to
  ResOwnerReleaseBuffer (et al), as it now also releases content locks if held

  I kept this separate as I'm not yet sure about the new name, partially due
  to there also being a "buffer io" resowner.  I tried "buffer ownership" for
  the resowner that tracks pins and locks, but that was long and not clearly
  better.

Greetings,

Andres Freund

  54. Re: Buffer locking is special (hints, checksums, AIO writes)

    Melanie Plageman <melanieplageman@gmail.com> — 2026-01-13T14:59:12Z

    On Mon, Jan 12, 2026 at 6:22 PM Andres Freund <andres@anarazel.de> wrote:
>
> Is this better?
>     lwlock: Invert meaning of LW_FLAG_RELEASE_OK
>
>     Previously, a flag was set to indicate that a lock release should wake up
>     waiters. Since waking waiters is the default behavior in the majority of
>     cases, this logic has been inverted. The new LW_FLAG_WAKE_IN_PROGRESS flag is
>     now set iff wakeups are explicitly inhibited.

I think what you have would work for most people. The key thing for me
is that the wakeups are inhibited _because_ someone else is already
awake. So, you don't have to wake anyone up when you release the lock
because there is already someone awake. Having you explain that
off-list was necessary for me to bridge the gap between RELEASE_NOT_OK
and WAKE_IN_PROGRESS. And I do agree that WAKE_IN_PROGRESS is more
descriptive of when the flag is actually set. RELEASE_NOT_OK doesn't
explain the state or who/when it should be set.

> > I wondered why this was needed (i.e. why it wasn't needed before)
>
> > @@ -6688,7 +7428,25 @@ ResOwnerReleaseBufferPin(Datum res)
> >      if (BufferIsLocal(buffer))
> >          UnpinLocalBufferNoOwner(buffer);
> >      else
> > +    {
> > +        PrivateRefCountEntry *ref;
> > +
> > +        ref = GetPrivateRefCountEntry(buffer, false);
> > +
> > +        /*
> > +         * If the buffer was locked at the time of the resowner release,
> > +         * release the lock now. This should only happen after errors.
> > +         */
> > +        if (ref->data.lockmode != BUFFER_LOCK_UNLOCK)
> > +        {
> > +            BufferDesc *buf = GetBufferDescriptor(buffer - 1);
> > +
> > +            HOLD_INTERRUPTS();    /* match the upcoming RESUME_INTERRUPTS */
> > +            BufferLockUnlock(buffer, buf);
> > +        }
> > +
> >          UnpinBufferNoOwner(GetBufferDescriptor(buffer - 1));
> > +    }
> >  }
>
> It's needed because previously content locks were released as part of the
> LWLockReleaseAll() that are sprinkled across various error recovery paths. Now
> that content locks aren't implemented via lwlocks anymore, something new is needed.

And all those LWLockReleaseAll()s are still needed because we might
hold other LWLocks even though we won't hold them for buffer content
access?

> > +    /* XXX: combine with fetch_and above? */
> > +    UnlockBufHdr(buf_hdr);
> >
> > Are you thinking about adding a helper that stops waiting and unlocks?
>
> I'm not sure what you mean by that? Just whether I plan to implement the
> FIXME?

I was trying to figure out why you left it as a FIXME and didn't just
do it or not do it. I thought maybe it was because you weren't sure if
you wanted to add another helper in addition to UnlockBufHdr().

> > bufmgr.c is super long anyway, so it's not like making it separate
> > makes the file manageable. On the other hand, it's probably better to
> > not keep making it worse.
>
> Yea. OTOH I don't know if a header that's just included by one file is really
> an improvement :/

Yea, I suppose that is a bit odd. Though it could be a pattern you
start for organizing gigantic files. I'm overall a +0.7 unless you
explain some other downsides than oddity.

- Melanie

  55. Re: Buffer locking is special (hints, checksums, AIO writes)

    Melanie Plageman <melanieplageman@gmail.com> — 2026-01-13T15:05:02Z

    On Mon, Jan 12, 2026 at 7:33 PM Andres Freund <andres@anarazel.de> wrote:
>
> - added a commit that renames ResOwnerReleaseBufferPin to
>   ResOwnerReleaseBuffer (et al), as it now also releases content locks if held
>
>   I kept this separate as I'm not yet sure about the new name, partially due
>   to there also being a "buffer io" resowner.  I tried "buffer ownership" for
>   the resowner that tracks pins and locks, but that was long and not clearly
>   better.

I didn't look at the patch but I strongly agree that
ResOwnerReleaseBufferPin() should not also release locks, so it should
have a new name. Ironic that ResOwnerReleaseBufferIO() releases pins
and not locks.

What about ResOwnerReleaseBufferClaim() or
ResOwnerReleaseBufferAccess() or ResOwnerReleaseBufferHold()?

- Melanie

  56. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-01-14T00:49:00Z

    Hi,

On 2026-01-13 10:05:02 -0500, Melanie Plageman wrote:
> On Mon, Jan 12, 2026 at 7:33 PM Andres Freund <andres@anarazel.de> wrote:
> >
> > - added a commit that renames ResOwnerReleaseBufferPin to
> >   ResOwnerReleaseBuffer (et al), as it now also releases content locks if held
> >
> >   I kept this separate as I'm not yet sure about the new name, partially due
> >   to there also being a "buffer io" resowner.  I tried "buffer ownership" for
> >   the resowner that tracks pins and locks, but that was long and not clearly
> >   better.
> 
> I didn't look at the patch but I strongly agree that
> ResOwnerReleaseBufferPin() should not also release locks, so it should
> have a new name.

OK.

> Ironic that ResOwnerReleaseBufferIO() releases pins and not locks.

Not sure I follow? I don't think it releases pins? And why should it release
locks?


> What about ResOwnerReleaseBufferClaim() or
> ResOwnerReleaseBufferAccess() or ResOwnerReleaseBufferHold()?

I'm inclined to go with just ResOwnerReleaseBuffer() at the moment. Buffer IO
kind of is a subsidiary thing, and it requires holding a pin as well, so it
doesn't feel too wrong.

I also wonder if we could merge BufferIO into the private refcount
infrastructure, similar to how the patches store the lockmode in the private
refcount.  The separate resowner acquisition does show up in profiles when
reading from the kernel page cache, so that'd be a nice (but small)
improvement.

Greetings,

Andres Freund

  57. Re: Buffer locking is special (hints, checksums, AIO writes)

    Chao Li <li.evan.chao@gmail.com> — 2026-01-14T02:26:07Z

    
> On Jan 13, 2026, at 08:33, Andres Freund <andres@anarazel.de> wrote:
> 
> Hi,
> 
> On 2026-01-12 12:45:03 -0500, Andres Freund wrote:
>> I'm doing another pass through 0003 and will push that if I don't find
>> anything significant.
> 
> Done, after adjust two comments in minor ways.
> 
> 
>> Also working on doing comment polishing of the later patches, found a few
>> things, but not quite enough to be worth reposting yet.
> 
> Here are the remaining commits, with a bit of polish:
> 
> - fixed references to old names in some places (lwlocks, release_ok)
> 
> - Aded an assert that we don't already hold a lock in BufferLockConditional()
> 
> - typo and grammar fixes
> 
> - updated the commit message of the LW_FLAG_RELEASE_OK, as "requested" by
>  Melanie. I hope this explains the situation better.
> 
> - added a commit that renames ResOwnerReleaseBufferPin to
>  ResOwnerReleaseBuffer (et al), as it now also releases content locks if held
> 
>  I kept this separate as I'm not yet sure about the new name, partially due
>  to there also being a "buffer io" resowner.  I tried "buffer ownership" for
>  the resowner that tracks pins and locks, but that was long and not clearly
>  better.
> 
> Greetings,
> 
> Andres Freund
> <v10-0001-lwlock-Invert-meaning-of-LW_FLAG_RELEASE_OK.patch><v10-0002-bufmgr-Make-definitions-related-to-buffer-descri.patch><v10-0003-bufmgr-Change-BufferDesc.state-to-be-a-64-bit-at.patch><v10-0004-bufmgr-Implement-buffer-content-locks-independen.patch><v10-0005-Require-share-exclusive-lock-to-set-hint-bits-an.patch><v10-0006-WIP-Make-UnlockReleaseBuffer-more-efficient.patch><v10-0007-WIP-bufmgr-Don-t-copy-pages-while-writing-out.patch><v10-0008-WIP-bufmgr-Rename-ResOwnerReleaseBufferPin.patch>

Hi Andres,

So far I’ve only reviewed 0001 and 0002. I’m not very familiar with this area, so the review has been a bit slow.

Overall, 0001 looks good to me. It renames LW_FLAG_RELEASE_OK to LW_FLAG_WAKE_IN_PROGRESS and inverts the meaning, which makes sense. I only have a small nit on naming: the local variable “new_release_in_progress". I see that it’s inherited from the old name and was updated from “_ok" to “_in_progress", but now that the flag itself is renamed, would it make sense to rename the variable as well? Something like “wake_in_progress" or “new_wake_in_progress" might better reflect the new flag name.

In 0002, a bunch of new macros are introduced. My initial impression wasn’t great, mostly due to the amount of line wrapping. Looking a bit closer, I also noticed some duplication, for example, "BUF_REFCOUNT_BITS + BUF_USAGECOUNT_BITS" appears more than once; and a small inconsistency between BUF_STATE_GET_REFCOUNT and BUF_STATE_GET_USAGECOUNT (even though the former doesn’t actually need a shift).

I tried a small refactor of the macro definitions in the attached diff to see if things could be made a bit more regular. It introduces a helper macro MASK() and a BUF_REFCOUNT_SHIFT constant, and removes a bit of duplication. If you like it, feel free to take it; otherwise, please just ignore it. Note that, the diff is based on 0002.

(I actually hesitated to attach a diff, because if you’ve already created a CF entry, the attached diff could break the CI tests. If that happens, sorry about that.)

Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/

  58. Re: Buffer locking is special (hints, checksums, AIO writes)

    Chao Li <li.evan.chao@gmail.com> — 2026-01-14T03:41:19Z

    
> On Jan 13, 2026, at 08:33, Andres Freund <andres@anarazel.de> wrote:
> 
> Hi,
> 
> On 2026-01-12 12:45:03 -0500, Andres Freund wrote:
>> I'm doing another pass through 0003 and will push that if I don't find
>> anything significant.
> 
> Done, after adjust two comments in minor ways.
> 
> 
>> Also working on doing comment polishing of the later patches, found a few
>> things, but not quite enough to be worth reposting yet.
> 
> Here are the remaining commits, with a bit of polish:
> 
> - fixed references to old names in some places (lwlocks, release_ok)
> 
> - Aded an assert that we don't already hold a lock in BufferLockConditional()
> 
> - typo and grammar fixes
> 
> - updated the commit message of the LW_FLAG_RELEASE_OK, as "requested" by
>  Melanie. I hope this explains the situation better.
> 
> - added a commit that renames ResOwnerReleaseBufferPin to
>  ResOwnerReleaseBuffer (et al), as it now also releases content locks if held
> 
>  I kept this separate as I'm not yet sure about the new name, partially due
>  to there also being a "buffer io" resowner.  I tried "buffer ownership" for
>  the resowner that tracks pins and locks, but that was long and not clearly
>  better.
> 
> Greetings,
> 
> Andres Freund
> <v10-0001-lwlock-Invert-meaning-of-LW_FLAG_RELEASE_OK.patch><v10-0002-bufmgr-Make-definitions-related-to-buffer-descri.patch><v10-0003-bufmgr-Change-BufferDesc.state-to-be-a-64-bit-at.patch><v10-0004-bufmgr-Implement-buffer-content-locks-independen.patch><v10-0005-Require-share-exclusive-lock-to-set-hint-bits-an.patch><v10-0006-WIP-Make-UnlockReleaseBuffer-more-efficient.patch><v10-0007-WIP-bufmgr-Don-t-copy-pages-while-writing-out.patch><v10-0008-WIP-bufmgr-Rename-ResOwnerReleaseBufferPin.patch>

A couple of comments on v10-0003, I just noticed 0001 and 0002 have been pushed.

Basically, code changes in 0003 is straightforward, just a couple of small comments:

1
```
- * refcounts in buf_internals.h.  This limitation could be lifted by using a
- * 64bit state; but it's unlikely to be worthwhile as 2^18-1 backends exceed
- * currently realistic configurations. Even if that limitation were removed,
- * we still could not a) exceed 2^23-1 because inval.c stores the ProcNumber
- * as a 3-byte signed integer, b) INT_MAX/4 because some places compute
- * 4*MaxBackends without any overflow check.  We check that the configured
- * number of backends does not exceed MAX_BACKENDS in InitializeMaxBackends().
+ * refcounts in buf_internals.h.  This limitation could be lifted, but it's
```

Before this patch, there was room for lifting the limitation. With this patch, state is 64bit already, but the significant 32bit will be used for buffer locking as stated in buf_internals.h, in other words, there is no room for lifting the limitation now. If that’s true, then I think we can remove the statements about lifting limitation.

2. By searching for “LockBufHdr”, I found one place missed to update in contrib/pg_prewarm/autoprewarm.c at line 706:
```
	for (num_blocks = 0, i = 0; i < NBuffers; i++)
	{
		uint32		buf_state; <=== line 706, should change to uint64

		CHECK_FOR_INTERRUPTS();

		bufHdr = GetBufferDescriptor(i);

		/* Lock each buffer header before inspecting. */
		buf_state = LockBufHdr(bufHdr);
```

I will continue reviewing 0004 tomorrow.


Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/

  59. Re: Buffer locking is special (hints, checksums, AIO writes)

    Melanie Plageman <melanieplageman@gmail.com> — 2026-01-14T14:17:22Z

    On Tue, Jan 13, 2026 at 7:49 PM Andres Freund <andres@anarazel.de> wrote:
>
> On 2026-01-13 10:05:02 -0500, Melanie Plageman wrote:
>
> > Ironic that ResOwnerReleaseBufferIO() releases pins and not locks.
>
> Not sure I follow? I don't think it releases pins? And why should it release
> locks?

Ah, I must not have actually read it or read the wrong thing.

> I also wonder if we could merge BufferIO into the private refcount
> infrastructure, similar to how the patches store the lockmode in the private
> refcount.  The separate resowner acquisition does show up in profiles when
> reading from the kernel page cache, so that'd be a nice (but small)
> improvement.

When you say "BufferIO", do you mean io_wref in the BufferDesc?

- Melanie

  60. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-01-14T15:20:02Z

    Hi,

On 2026-01-14 09:17:22 -0500, Melanie Plageman wrote:
> On Tue, Jan 13, 2026 at 7:49 PM Andres Freund <andres@anarazel.de> wrote:
> > I also wonder if we could merge BufferIO into the private refcount
> > infrastructure, similar to how the patches store the lockmode in the private
> > refcount.  The separate resowner acquisition does show up in profiles when
> > reading from the kernel page cache, so that'd be a nice (but small)
> > improvement.
> 
> When you say "BufferIO", do you mean io_wref in the BufferDesc?

I was trying to refer to ResourceOwnerRememberBufferIO(),
ResourceOwnerForgetBufferIO(), ResOwnerReleaseBufferIO(), etc. That's
basically used to unset BM_IO_IN_PROGRESS when an error occurs while trying to
perform IO.

Greetings,

Andres Freund

  61. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-01-14T16:23:05Z

    Hi,

On 2026-01-14 10:26:07 +0800, Chao Li wrote:
> So far I’ve only reviewed 0001 and 0002. I’m not very familiar with this area, so the review has been a bit slow.
> 
> Overall, 0001 looks good to me. It renames LW_FLAG_RELEASE_OK to
> LW_FLAG_WAKE_IN_PROGRESS and inverts the meaning, which makes sense. I only
> have a small nit on naming: the local variable “new_release_in_progress". I
> see that it’s inherited from the old name and was updated from “_ok" to
> “_in_progress", but now that the flag itself is renamed, would it make sense
> to rename the variable as well? Something like “wake_in_progress" or
> “new_wake_in_progress" might better reflect the new flag name.

Agreed that is better. Updated that way.



> In 0002, a bunch of new macros are introduced. My initial impression wasn’t
> great, mostly due to the amount of line wrapping.

I think the previous formatting made it hard to actually write useful comments
and caused line-length problems in the subsequent commits. Lines are cheap.


> Looking a bit closer, I also noticed some duplication, for example,
> "BUF_REFCOUNT_BITS + BUF_USAGECOUNT_BITS" appears more than once

Yea, that's probably better to avoid. I'll add a fix to that in the commit
changing it to 64bits, I think.


> ; and a small inconsistency between BUF_STATE_GET_REFCOUNT and
> BUF_STATE_GET_USAGECOUNT (even though the former doesn’t actually need a
> shift).

I don't see the point, if we later want to move refcounts elsewhere, we can do
it at that time.


> I tried a small refactor of the macro definitions in the attached diff to
> see if things could be made a bit more regular. It introduces a helper macro
> MASK() and a BUF_REFCOUNT_SHIFT constant, and removes a bit of
> duplication. If you like it, feel free to take it; otherwise, please just
> ignore it. Note that, the diff is based on 0002.

I don't think the MASK thing is an improvement.


> (I actually hesitated to attach a diff, because if you’ve already created a
> CF entry, the attached diff could break the CI tests. If that happens, sorry
> about that.)

FWIW, there's a trick to avoid that: Rename your patch to end in .txt or such.


Greetings,

Andres Freund

  62. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-01-14T16:30:43Z

    Hi,

On 2026-01-14 11:41:19 +0800, Chao Li wrote:
> Basically, code changes in 0003 is straightforward, just a couple of small comments:
> 
> 1
> ```
> - * refcounts in buf_internals.h.  This limitation could be lifted by using a
> - * 64bit state; but it's unlikely to be worthwhile as 2^18-1 backends exceed
> - * currently realistic configurations. Even if that limitation were removed,
> - * we still could not a) exceed 2^23-1 because inval.c stores the ProcNumber
> - * as a 3-byte signed integer, b) INT_MAX/4 because some places compute
> - * 4*MaxBackends without any overflow check.  We check that the configured
> - * number of backends does not exceed MAX_BACKENDS in InitializeMaxBackends().
> + * refcounts in buf_internals.h.  This limitation could be lifted, but it's
> ```
> 
> Before this patch, there was room for lifting the limitation. With this
> patch, state is 64bit already, but the significant 32bit will be used for
> buffer locking as stated in buf_internals.h, in other words, there is no
> room for lifting the limitation now. If that’s true, then I think we can
> remove the statements about lifting limitation.

I'm not following - there's plenty space for more bits if we need that:

 * State of the buffer itself (in order):
 * - 18 bits refcount
 * - 4 bits usage count
 * - 12 bits of flags
 * - 18 bits share-lock count
 * - 1 bit share-exclusive locked
 * - 1 bit exclusive locked

That's 54 bits in total. Which part is in the lower and which in the upper
32bit isn't relevant for anything afaict?


> 2. By searching for “LockBufHdr”, I found one place missed to update in contrib/pg_prewarm/autoprewarm.c at line 706:
> ```
> 	for (num_blocks = 0, i = 0; i < NBuffers; i++)
> 	{
> 		uint32		buf_state; <=== line 706, should change to uint64
> 
> 		CHECK_FOR_INTERRUPTS();
> 
> 		bufHdr = GetBufferDescriptor(i);
> 
> 		/* Lock each buffer header before inspecting. */
> 		buf_state = LockBufHdr(bufHdr);
> ```

Good catch!  I didn't find any other similar omissions...


> I will continue reviewing 0004 tomorrow.

Cool.

I'd like to push

  bufmgr: Change BufferDesc.state to be a 64-bit atomic
  bufmgr: Implement buffer content locks independently of lwlocks

pretty soon, so that we then can concentrate on

  Require share-exclusive lock to set hint bits and to flush

and then subsequently on

  WIP: bufmgr: Don't copy pages while writing out

as there are other patches that have this work as a dependency...

Greetings,

Andres Freund

  63. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-01-14T21:20:58Z

    Hi,

On 2026-01-12 19:33:56 -0500, Andres Freund wrote:
> Here are the remaining commits, with a bit of polish:

I pushed 0001, 0002.

Attached is an updated version of the remaining changes:

- I updated the definition in BUF_DEFINE_FLAG to have a redundant copy of
  BUF_FLAG_SHIFT's "contents", as suggested by Chao

- I realized I had forgotten to remove the BufferContent lwlock tranche

- Updated the FIXME comment about using PGPROC->lw* to not be a fixme anymore,
  it seems nobody is pushing back against that being ugly-but-reasonable for now

- I renamed ResOwnerReleaseBufferPin etc to ResOwnerReleaseBuffer, as I
  suggested nearby.

- Added a commit removing ForEachLWLockHeldByMe, now that it's not used
  anymore. I checked in with Noah, who added it, and he's on-board with that
  plan.

- Added a commit removing LWLockDisown(), LWLockReleaseDisowned(). They were
  added for AIO and AIO doesn't need them anymore, as that's implemented
  purely in bufmgr.c now. I don't see a reason to keep them...

- I reflowed the comments / README in "Require share-exclusive lock to set hint bits and to flush"
  and removed the FIXME about that

- Removed "FIXME: The start of the comment above needs updating." from the
  above commit, I already had rewritten the comment, just hadn't removed the
  FIXME yet


I tried putting the new code in a header, as we had discussed, but that turns
out to not work easily: The locking code needs access to the private-refcount
infrastructure and we can't put the private refcount infrastructure into a
header without making PrivateRef* non-static, which in turn causes slightly
worse code generation.


I'm now working on cleaning up the last two commits. The most crucial bit is
to simplify what happens in MarkSharedBufferDirtyHint(), we afaict can delete
the use of DELAY_CHKPT_START etc and just go to marking the buffer dirty first
and then do the WAL logging, just like normal WAL logging. The previous order
was only required because we were dirtying the page while holding only a
shared lock, which did not conflict with the lock held by SyncBuffers() etc.

There are some comments that arguably should be updated in 0005, but will only
be updated in 0006. I don't really see how to address that without squashing
the two commits though - which I think wouldn't be good, as the necessary
changes are decidedly nontrivial.

Greetings,

Andres Freund

  64. Re: Buffer locking is special (hints, checksums, AIO writes)

    Chao Li <li.evan.chao@gmail.com> — 2026-01-14T23:20:27Z

    
> On Jan 15, 2026, at 00:30, Andres Freund <andres@anarazel.de> wrote:
> 
> Hi,
> 
> On 2026-01-14 11:41:19 +0800, Chao Li wrote:
>> Basically, code changes in 0003 is straightforward, just a couple of small comments:
>> 
>> 1
>> ```
>> - * refcounts in buf_internals.h.  This limitation could be lifted by using a
>> - * 64bit state; but it's unlikely to be worthwhile as 2^18-1 backends exceed
>> - * currently realistic configurations. Even if that limitation were removed,
>> - * we still could not a) exceed 2^23-1 because inval.c stores the ProcNumber
>> - * as a 3-byte signed integer, b) INT_MAX/4 because some places compute
>> - * 4*MaxBackends without any overflow check.  We check that the configured
>> - * number of backends does not exceed MAX_BACKENDS in InitializeMaxBackends().
>> + * refcounts in buf_internals.h.  This limitation could be lifted, but it's
>> ```
>> 
>> Before this patch, there was room for lifting the limitation. With this
>> patch, state is 64bit already, but the significant 32bit will be used for
>> buffer locking as stated in buf_internals.h, in other words, there is no
>> room for lifting the limitation now. If that’s true, then I think we can
>> remove the statements about lifting limitation.
> 
> I'm not following - there's plenty space for more bits if we need that:
> 
> * State of the buffer itself (in order):
> * - 18 bits refcount
> * - 4 bits usage count
> * - 12 bits of flags
> * - 18 bits share-lock count
> * - 1 bit share-exclusive locked
> * - 1 bit exclusive locked
> 
> That's 54 bits in total. Which part is in the lower and which in the upper
> 32bit isn't relevant for anything afaict?

Because I saw the comment in buf_internals.h:
```
 * NB: A future commit will use a significant portion of the remaining bits to
* implement buffer locking as part of the state variable.
```
That seems to indicate all the significant 32 bits will be used for buffer locking. Also, there is an assert that concretes the impression:
```
StaticAssertDecl(BUF_REFCOUNT_BITS + BUF_USAGECOUNT_BITS + BUF_FLAG_BITS == 32,
       "parts of buffer state space need to equal 32");
```

So, I thought we can explain 18bit refcount is good enough without mentioning “lifting” that potentially adds confusion to readers. But anyway, this is not a strong opinion. I won’t insist on this comment.

> 
> 
>> 2. By searching for “LockBufHdr”, I found one place missed to update in contrib/pg_prewarm/autoprewarm.c at line 706:
>> ```
>> for (num_blocks = 0, i = 0; i < NBuffers; i++)
>> {
>> uint32 buf_state; <=== line 706, should change to uint64
>> 
>> CHECK_FOR_INTERRUPTS();
>> 
>> bufHdr = GetBufferDescriptor(i);
>> 
>> /* Lock each buffer header before inspecting. */
>> buf_state = LockBufHdr(bufHdr);
>> ```
> 
> Good catch!  I didn't find any other similar omissions...

I saw you have added this occurrence to v11.

> 
> 
>> I will continue reviewing 0004 tomorrow.
> 
> Cool.
> 
> I'd like to push
> 
>  bufmgr: Change BufferDesc.state to be a 64-bit atomic
>  bufmgr: Implement buffer content locks independently of lwlocks
> 
> pretty soon, so that we then can concentrate on

Other than the “lifting” comment, v11 LGTM. But that’s not a strong opinion. I explained more above, if you consider that’s not a problem, I am totally fine.

Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/

  65. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-01-14T23:37:54Z

    Hi,

On 2026-01-15 07:20:27 +0800, Chao Li wrote:
> > On Jan 15, 2026, at 00:30, Andres Freund <andres@anarazel.de> wrote:
> > On 2026-01-14 11:41:19 +0800, Chao Li wrote:
> >> Basically, code changes in 0003 is straightforward, just a couple of small comments:
> >> 
> >> 1
> >> ```
> >> - * refcounts in buf_internals.h.  This limitation could be lifted by using a
> >> - * 64bit state; but it's unlikely to be worthwhile as 2^18-1 backends exceed
> >> - * currently realistic configurations. Even if that limitation were removed,
> >> - * we still could not a) exceed 2^23-1 because inval.c stores the ProcNumber
> >> - * as a 3-byte signed integer, b) INT_MAX/4 because some places compute
> >> - * 4*MaxBackends without any overflow check.  We check that the configured
> >> - * number of backends does not exceed MAX_BACKENDS in InitializeMaxBackends().
> >> + * refcounts in buf_internals.h.  This limitation could be lifted, but it's
> >> ```
> >> 
> >> Before this patch, there was room for lifting the limitation. With this
> >> patch, state is 64bit already, but the significant 32bit will be used for
> >> buffer locking as stated in buf_internals.h, in other words, there is no
> >> room for lifting the limitation now. If that’s true, then I think we can
> >> remove the statements about lifting limitation.
> > 
> > I'm not following - there's plenty space for more bits if we need that:
> > 
> > * State of the buffer itself (in order):
> > * - 18 bits refcount
> > * - 4 bits usage count
> > * - 12 bits of flags
> > * - 18 bits share-lock count
> > * - 1 bit share-exclusive locked
> > * - 1 bit exclusive locked
> > 
> > That's 54 bits in total. Which part is in the lower and which in the upper
> > 32bit isn't relevant for anything afaict?
> 
> Because I saw the comment in buf_internals.h:
> ```
>  * NB: A future commit will use a significant portion of the remaining bits to
> * implement buffer locking as part of the state variable.
> ```
> That seems to indicate all the significant 32 bits will be used for buffer locking.

A significant portion != all. As the above excerpt from the comment shows, the
locking uses 20 bits. We could increase max backends by 5 bits without running
out of bits (we'd need space both in the refcount bitspace as well as the
share-lock bitspace).


> Also, there is an assert that concretes the impression:
> ```
> StaticAssertDecl(BUF_REFCOUNT_BITS + BUF_USAGECOUNT_BITS + BUF_FLAG_BITS == 32,
>        "parts of buffer state space need to equal 32");
> ```

You can see that being relaxed in the subsequent commit, when we start to use
more bits.


Greetings,

Andres Freund

  66. Re: Buffer locking is special (hints, checksums, AIO writes)

    Chao Li <li.evan.chao@gmail.com> — 2026-01-15T00:04:21Z

    
> On Jan 15, 2026, at 07:37, Andres Freund <andres@anarazel.de> wrote:
> 
> Hi,
> 
> On 2026-01-15 07:20:27 +0800, Chao Li wrote:
>>> On Jan 15, 2026, at 00:30, Andres Freund <andres@anarazel.de> wrote:
>>> On 2026-01-14 11:41:19 +0800, Chao Li wrote:
>>>> Basically, code changes in 0003 is straightforward, just a couple of small comments:
>>>> 
>>>> 1
>>>> ```
>>>> - * refcounts in buf_internals.h.  This limitation could be lifted by using a
>>>> - * 64bit state; but it's unlikely to be worthwhile as 2^18-1 backends exceed
>>>> - * currently realistic configurations. Even if that limitation were removed,
>>>> - * we still could not a) exceed 2^23-1 because inval.c stores the ProcNumber
>>>> - * as a 3-byte signed integer, b) INT_MAX/4 because some places compute
>>>> - * 4*MaxBackends without any overflow check.  We check that the configured
>>>> - * number of backends does not exceed MAX_BACKENDS in InitializeMaxBackends().
>>>> + * refcounts in buf_internals.h.  This limitation could be lifted, but it's
>>>> ```
>>>> 
>>>> Before this patch, there was room for lifting the limitation. With this
>>>> patch, state is 64bit already, but the significant 32bit will be used for
>>>> buffer locking as stated in buf_internals.h, in other words, there is no
>>>> room for lifting the limitation now. If that’s true, then I think we can
>>>> remove the statements about lifting limitation.
>>> 
>>> I'm not following - there's plenty space for more bits if we need that:
>>> 
>>> * State of the buffer itself (in order):
>>> * - 18 bits refcount
>>> * - 4 bits usage count
>>> * - 12 bits of flags
>>> * - 18 bits share-lock count
>>> * - 1 bit share-exclusive locked
>>> * - 1 bit exclusive locked
>>> 
>>> That's 54 bits in total. Which part is in the lower and which in the upper
>>> 32bit isn't relevant for anything afaict?
>> 
>> Because I saw the comment in buf_internals.h:
>> ```
>> * NB: A future commit will use a significant portion of the remaining bits to
>> * implement buffer locking as part of the state variable.
>> ```
>> That seems to indicate all the significant 32 bits will be used for buffer locking.
> 
> A significant portion != all. As the above excerpt from the comment shows, the
> locking uses 20 bits. We could increase max backends by 5 bits without running
> out of bits (we'd need space both in the refcount bitspace as well as the
> share-lock bitspace).

Make sense. I think I misread the comment.

> 
> 
>> Also, there is an assert that concretes the impression:
>> ```
>> StaticAssertDecl(BUF_REFCOUNT_BITS + BUF_USAGECOUNT_BITS + BUF_FLAG_BITS == 32,
>>       "parts of buffer state space need to equal 32");
>> ```
> 
> You can see that being relaxed in the subsequent commit, when we start to use
> more bits.
> 

Sure. I plan to review 0003-0005 today. I believe I will get better understanding.

So, 0001 and 0002 LGTM now.

Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/

  67. Re: Buffer locking is special (hints, checksums, AIO writes)

    Chao Li <li.evan.chao@gmail.com> — 2026-01-15T06:22:09Z

    
> On Jan 15, 2026, at 08:04, Chao Li <li.evan.chao@gmail.com> wrote:
> 
> 
> 
>> On Jan 15, 2026, at 07:37, Andres Freund <andres@anarazel.de> wrote:
>> 
>> Hi,
>> 
>> On 2026-01-15 07:20:27 +0800, Chao Li wrote:
>>>> On Jan 15, 2026, at 00:30, Andres Freund <andres@anarazel.de> wrote:
>>>> On 2026-01-14 11:41:19 +0800, Chao Li wrote:
>>>>> Basically, code changes in 0003 is straightforward, just a couple of small comments:
>>>>> 
>>>>> 1
>>>>> ```
>>>>> - * refcounts in buf_internals.h.  This limitation could be lifted by using a
>>>>> - * 64bit state; but it's unlikely to be worthwhile as 2^18-1 backends exceed
>>>>> - * currently realistic configurations. Even if that limitation were removed,
>>>>> - * we still could not a) exceed 2^23-1 because inval.c stores the ProcNumber
>>>>> - * as a 3-byte signed integer, b) INT_MAX/4 because some places compute
>>>>> - * 4*MaxBackends without any overflow check.  We check that the configured
>>>>> - * number of backends does not exceed MAX_BACKENDS in InitializeMaxBackends().
>>>>> + * refcounts in buf_internals.h.  This limitation could be lifted, but it's
>>>>> ```
>>>>> 
>>>>> Before this patch, there was room for lifting the limitation. With this
>>>>> patch, state is 64bit already, but the significant 32bit will be used for
>>>>> buffer locking as stated in buf_internals.h, in other words, there is no
>>>>> room for lifting the limitation now. If that’s true, then I think we can
>>>>> remove the statements about lifting limitation.
>>>> 
>>>> I'm not following - there's plenty space for more bits if we need that:
>>>> 
>>>> * State of the buffer itself (in order):
>>>> * - 18 bits refcount
>>>> * - 4 bits usage count
>>>> * - 12 bits of flags
>>>> * - 18 bits share-lock count
>>>> * - 1 bit share-exclusive locked
>>>> * - 1 bit exclusive locked
>>>> 
>>>> That's 54 bits in total. Which part is in the lower and which in the upper
>>>> 32bit isn't relevant for anything afaict?
>>> 
>>> Because I saw the comment in buf_internals.h:
>>> ```
>>> * NB: A future commit will use a significant portion of the remaining bits to
>>> * implement buffer locking as part of the state variable.
>>> ```
>>> That seems to indicate all the significant 32 bits will be used for buffer locking.
>> 
>> A significant portion != all. As the above excerpt from the comment shows, the
>> locking uses 20 bits. We could increase max backends by 5 bits without running
>> out of bits (we'd need space both in the refcount bitspace as well as the
>> share-lock bitspace).
> 
> Make sense. I think I misread the comment.
> 
>> 
>> 
>>> Also, there is an assert that concretes the impression:
>>> ```
>>> StaticAssertDecl(BUF_REFCOUNT_BITS + BUF_USAGECOUNT_BITS + BUF_FLAG_BITS == 32,
>>>      "parts of buffer state space need to equal 32");
>>> ```
>> 
>> You can see that being relaxed in the subsequent commit, when we start to use
>> more bits.
>> 
> 
> Sure. I plan to review 0003-0005 today. I believe I will get better understanding.

I have finished reviewing 0003-0005. Basically, 0003 and 0004 just delete some unused functions. I only searched over the source tree to ensure no usages of them. I spent time on 0005. The code logic are solid already, I didn't find any correctness issue and only got some small comments:

1 - 0004 - commit message
```
subsequent commits fixing a typo an a parameter name.
```

Typo: a typo an a parameter name => a typo in a parameter name

2 - 0005 - bufmgr.c
```
-	 * Pin it, share-lock it, write it.  (FlushBuffer will do nothing if the
-	 * buffer is clean by the time we've locked it.)
+	 * Pin it, share-exclusive-lock it, write it.  (FlushBuffer will do
+	 * nothing if the buffer is clean by the time we've locked it.)
 	 */
 	PinBuffer_Locked(bufHdr);
```

I think we don’t need to mention lock type in this comment, because the logic belongs to FlushUnlockedBuffer(). Also, FlushBuffer is misleading here, because we call FlushUnlockedBuffer() here, and FlushUnlockedBuffer() in turn calls FlushBuffer().

So, I think we can simplify the comment as something like “Pin it and flush the buffer"

3 - 0005 - bufmgr.c
```
+inline void
+MarkBufferDirtyHint(Buffer buffer, bool buffer_std)
```

It’s quite uncommon to extern an inline function. I think usually if we want to make an inline function accessible from external, we define it “static inline” in a header file. So, I guess “inline” is a typo here.

4 - 0005 - bufmgr.c
```
/*
* MarkBufferDirtyHint
*
* Mark a buffer dirty for non-critical changes.
*
* This is essentially the same as MarkBufferDirty, except:
*
* 1. The caller does not write WAL; so if checksums are enabled, we may need
* to write an XLOG_FPI_FOR_HINT WAL record to protect against torn pages.
* 2. The caller might have only a share-exclusive-lock instead of an
* exclusive-lock on the buffer's content lock.
```

For point 2, do we need to mention “For shared buffers”. Because this function also handles local buffer that doesn’t require a lock.

5 - 0005 - bufmgr.c
```
+/*
+ * Helper for BufferBeginSetHintBits() and BufferSetHintBits16().

+static inline bool
+SharedBufferBeginSetHintBits(Buffer buffer, BufferDesc *buf_hdr, uint64 *lockstate)
```

In the previous function comment:
```
- * MarkBufferDirtyHint
+ * Shared-buffer only helper for MarkBufferDirtyHint() and
+ * BufferSetHintBits16().
```

It mentions “Shared-buffer only helper”. I think SharedBufferBeginSetHintBits is also only for shared buffer, maybe also add “Shared-buffer only” before “helper” in the comment.

6 - 0005 - bufmgr.c
```
+	}
+
+}
```

Nit: In function SharedBufferBeginSetHintBits, the last empty line is not needed.

7 - 0005 - bufmgr.c
```
+ * This checks if the current lock mode already suffices to allow hint bits
+ * being set and, if not, whether the current lock can be upgraded.
+ */
+static inline bool
+SharedBufferBeginSetHintBits(Buffer buffer, BufferDesc *buf_hdr, uint64 *lockstate)
```

Nit: "if not, whether the current lock can be upgraded” might be read as “lock can be upgraded, so the caller still need to take some action to upgrade the lock”, but the function has upgraded the lock when returning true. So, how about explicitly state something like: "if not, it attempts to atomically upgrade it to share-exclusive. Returns true if hint bits may be set (with or without an upgrade), false otherwise."

8 - 0005 - fsmpage.c
```
 * needs to be updated. exclusive_lock_held should be set to true if the
* caller is already holding an exclusive lock, to avoid extra work.
```

The function comment of fsm_search_avail() needs to be updated. exclusive_lock_held should be set to true if the caller is already holding a **share-exclusive or** exclusive lock.

Maybe the parameter name “exclusive_lock_held” can be enhanced as well.

9 - 0005 - fsmpage.c
```
+		if (!exclusive_lock_held)
+			BufferFinishSetHintBits(buf, false, true);
```

Nit: just curious why set the third parameter as “true”? When the second (mark_dirty) is false, the third parameter is not used at all.

10 - 0005 - freespace.c
```
-	 * Reset the next slot pointer. This encourages the use of low-numbered
-	 * pages, increasing the chances that a later vacuum can truncate the
-	 * relation. We don't bother with marking the page dirty if it wasn't
-	 * already, since this is just a hint.
+	 * Try to reset the next slot pointer. This encourages the use of
+	 * low-numbered pages, increasing the chances that a later vacuum can
+	 * truncate the relation. We don't bother with marking the page dirty if
+	 * it wasn't already, since this is just a hint.
 	 */
 	LockBuffer(buf, BUFFER_LOCK_SHARE);
-	((FSMPage) PageGetContents(page))->fp_next_slot = 0;
+	if (BufferBeginSetHintBits(buf))
+	{
+		((FSMPage) PageGetContents(page))->fp_next_slot = 0;
+		BufferFinishSetHintBits(buf, false, false);
+	}
```

Before this patch, we unconditionally set fp_next_slot, now the setting might be skipped. You have add “Try to” in the comment that has explained the possibility of skipping setting fp_next_slot. Would it be better to add a brief statement for what will result in when skipping setting fp_next_slot? Something like “Skipping the update only affects reuse, not correctness”.

Lately, I saw you have done this in nbtinsert.c:
```
* mark the index entry killed. It's ok if we're not
* allowed to, this isn't required for correctness.
```
which, I think, confirmed my comment.

11 - 0005 Maybe not a problem. In nbtree.h:
```
/*
* We need to be able to tell the difference between read and write
* requests for pages, in order to do locking correctly.
*/
#define BT_READ BUFFER_LOCK_SHARE
#define BT_WRITE BUFFER_LOCK_EXCLUSIVE
```

Can the new lock type BUFFER_LOCK_SHARE_EXCLUSIVE be used by nbt? Maybe implicitly upgrading from BT_READ to BUFFER_LOCK_SHARE_EXCLUSIVE is good enough?

12 - 0005 - heapam_visibility.c

After this commit, tuple hint bits may remain unset if we can’t obtain share-exclusive permission. That’s fine because hint bits are advisory and optional, but this is a behavior change. Would it make sense to mention this explicitly and briefly in the SetHintBitsExt() comment?

13 - 0005 - nbtutils.c
```
+				/*
+				 * If we're not able to set hint bits, there's no point
+				 * continuing.
+				 */
+				if (!killedsomething &&
+					!BufferBeginSetHintBits(buf))
+					goto unlock_page;
```
I like this code, because it ensures to only call BufferBeginSetHintBits once. But lately, I saw the same logic in gistget.c:
```
+		if (!killedsomething)
+		{
+			/*
+			 * Use hint bit infrastructure to be allowed to modify the page
+			 * without holding an exclusive lock.
+			 */
+			if (!BufferBeginSetHintBits(buffer))
+				goto unlock;
+		}
```
I just feel the gist version is easier to read, so maybe change the nbt one to be the same as the gist one. But I think the nbt one’s comment is better.

Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/

  68. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-01-15T16:43:14Z

    Hi,

On 2026-01-15 14:22:09 +0800, Chao Li wrote:
> 3 - 0005 - bufmgr.c
> ```
> +inline void
> +MarkBufferDirtyHint(Buffer buffer, bool buffer_std)
> ```
> 
> It’s quite uncommon to extern an inline function. I think usually if we want
> to make an inline function accessible from external, we define it “static
> inline” in a header file. So, I guess “inline” is a typo here.

It works just fine to put an inline into the function definition, even if it's
an external function. That hints the compiler to inline it inside the
translation unit. Which is useful here, because it leads to
MarkBufferDirtyHint() being inlined into BufferFinishSetHintBits().



> 4 - 0005 - bufmgr.c
> ```
> /*
> * MarkBufferDirtyHint
> *
> * Mark a buffer dirty for non-critical changes.
> *
> * This is essentially the same as MarkBufferDirty, except:
> *
> * 1. The caller does not write WAL; so if checksums are enabled, we may need
> * to write an XLOG_FPI_FOR_HINT WAL record to protect against torn pages.
> * 2. The caller might have only a share-exclusive-lock instead of an
> * exclusive-lock on the buffer's content lock.
> ```
> 
> For point 2, do we need to mention “For shared buffers”. Because this function also handles local buffer that doesn’t require a lock.

That's an old comment, just slightly rephrased. And I don't think it matters
that temp buffers don't need to be locked.



> 8 - 0005 - fsmpage.c
> ```
>  * needs to be updated. exclusive_lock_held should be set to true if the
> * caller is already holding an exclusive lock, to avoid extra work.
> ```
> 
> The function comment of fsm_search_avail() needs to be
> updated. exclusive_lock_held should be set to true if the caller is already
> holding a **share-exclusive or** exclusive lock.

Why?


> 9 - 0005 - fsmpage.c
> ```
> +		if (!exclusive_lock_held)
> +			BufferFinishSetHintBits(buf, false, true);
> ```
> 
> Nit: just curious why set the third parameter as “true”? When the second (mark_dirty) is false, the third parameter is not used at all.

That's wrong, indeed. Not because of the mark_dirty, but because they aren't
standard pages.


> 10 - 0005 - freespace.c
> ```
> -	 * Reset the next slot pointer. This encourages the use of low-numbered
> -	 * pages, increasing the chances that a later vacuum can truncate the
> -	 * relation. We don't bother with marking the page dirty if it wasn't
> -	 * already, since this is just a hint.
> +	 * Try to reset the next slot pointer. This encourages the use of
> +	 * low-numbered pages, increasing the chances that a later vacuum can
> +	 * truncate the relation. We don't bother with marking the page dirty if
> +	 * it wasn't already, since this is just a hint.
>  	 */
>  	LockBuffer(buf, BUFFER_LOCK_SHARE);
> -	((FSMPage) PageGetContents(page))->fp_next_slot = 0;
> +	if (BufferBeginSetHintBits(buf))
> +	{
> +		((FSMPage) PageGetContents(page))->fp_next_slot = 0;
> +		BufferFinishSetHintBits(buf, false, false);
> +	}
> ```
> 
> Before this patch, we unconditionally set fp_next_slot, now the setting
> might be skipped. You have add “Try to” in the comment that has explained
> the possibility of skipping setting fp_next_slot. Would it be better to add
> a brief statement for what will result in when skipping setting
> fp_next_slot? Something like “Skipping the update only affects reuse, not
> correctness”.

I don't see the point. The whole paragraph is about how this isn't crucial.


> 11 - 0005 Maybe not a problem. In nbtree.h:
> ```
> /*
> * We need to be able to tell the difference between read and write
> * requests for pages, in order to do locking correctly.
> */
> #define BT_READ BUFFER_LOCK_SHARE
> #define BT_WRITE BUFFER_LOCK_EXCLUSIVE
> ```
> 
> Can the new lock type BUFFER_LOCK_SHARE_EXCLUSIVE be used by nbt?

I don't think at the moment. If we introduce a user, we can add the
define. Orthogonally, I think stuff like BT_READ/BT_WRITE should eventually be
removed, the only thing it does is to make it harder to search the code.


> Maybe implicitly upgrading from BT_READ to BUFFER_LOCK_SHARE_EXCLUSIVE is good enough?

I don't see where that would trivially be safe.


> 12 - 0005 - heapam_visibility.c
> 
> After this commit, tuple hint bits may remain unset if we can’t obtain
> share-exclusive permission. That’s fine because hint bits are advisory and
> optional, but this is a behavior change. Would it make sense to mention this
> explicitly and briefly in the SetHintBitsExt() comment?

That's not new though, we already could skip setting hint bits due to async
commits.

Greetings,

Andres Freund

  69. Re: Buffer locking is special (hints, checksums, AIO writes)

    Tom Lane <tgl@sss.pgh.pa.us> — 2026-01-15T23:02:34Z

    Various buildfarm animals are complaining about fcb9c977a,
similarly to this from calliphoridae [1]:

[969/2355] ccache gcc -Isrc/backend/postgres_lib.a.p -Isrc/include -I../pgsql/src/include -I/usr/include/libxml2 -fdiagnostics-color=never -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -O2 -g -fno-strict-aliasing -fwrapv -fexcess-precision=standard -D_GNU_SOURCE -Wmissing-prototypes -Wpointer-arith -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wshadow=compatible-local -Wformat-security -Wdeclaration-after-statement -Wmissing-variable-declarations -Wno-format-truncation -Wno-stringop-truncation -O1 -ggdb -g3 -fno-omit-frame-pointer -Wall -Wextra -Wno-unused-parameter -Wno-sign-compare -Wno-missing-field-initializers -DCOPY_PARSE_PLAN_TREES -DRAW_EXPRESSION_COVERAGE_TEST -fPIC -isystem /usr/include/mit-krb5 -pthread -DBUILDING_DLL -MD -MQ src/backend/postgres_lib.a.p/storage_buffer_bufmgr.c.o -MF src/backend/postgres_lib.a.p/storage_buffer_bufmgr.c.o.d -o src/backend/postgres_lib.a.p/storage_buffer_bufmgr.c.o -c ../pgsql/src/backend/storage/buffer/bufmgr.c
In file included from ../pgsql/src/include/pgstat.h:24,
                 from ../pgsql/src/backend/storage/buffer/bufmgr.c:52:
In function \342\200\230pgstat_report_wait_start\342\200\231,
    inlined from \342\200\230BufferLockAcquire\342\200\231 at ../pgsql/src/backend/storage/buffer/bufmgr.c:5833:3:
../pgsql/src/include/utils/wait_event.h:75:49: warning: \342\200\230wait_event\342\200\231 may be used uninitialized [-Wmaybe-uninitialized]
   75 |         *(volatile uint32 *) my_wait_event_info = wait_event_info;
      |         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~
../pgsql/src/backend/storage/buffer/bufmgr.c: In function \342\200\230BufferLockAcquire\342\200\231:
../pgsql/src/backend/storage/buffer/bufmgr.c:5781:33: note: \342\200\230wait_event\342\200\231 was declared here
 5781 |                 uint32          wait_event;
      |                                 ^~~~~~~~~~

Apparently they do not find the coding in this switch persuasive:

		switch (mode)
		{
			case BUFFER_LOCK_EXCLUSIVE:
				wait_event = WAIT_EVENT_BUFFER_EXCLUSIVE;
				break;
			case BUFFER_LOCK_SHARE_EXCLUSIVE:
				wait_event = WAIT_EVENT_BUFFER_SHARE_EXCLUSIVE;
				break;
			case BUFFER_LOCK_SHARE:
				wait_event = WAIT_EVENT_BUFFER_SHARED;
				break;
			case BUFFER_LOCK_UNLOCK:
				pg_unreachable();

		}

It's not clear to me whether that's more about not believing
pg_unreachable() or more about the lack of a default: case.
I see that this is a modification of code that existed before
fcb9c977a and wasn't being complained of, which makes it even
stranger.

			regards, tom lane

[1] https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=calliphoridae&dt=2026-01-15%2019%3A35%3A00&stg=build

  70. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-01-15T23:16:07Z

    Hi,

On 2026-01-15 18:02:34 -0500, Tom Lane wrote:
> In file included from ../pgsql/src/include/pgstat.h:24,
>                  from ../pgsql/src/backend/storage/buffer/bufmgr.c:52:
> In function \342\200\230pgstat_report_wait_start\342\200\231,
>     inlined from \342\200\230BufferLockAcquire\342\200\231 at ../pgsql/src/backend/storage/buffer/bufmgr.c:5833:3:
> ../pgsql/src/include/utils/wait_event.h:75:49: warning: \342\200\230wait_event\342\200\231 may be used uninitialized [-Wmaybe-uninitialized]
>    75 |         *(volatile uint32 *) my_wait_event_info = wait_event_info;
> [...]
> Apparently they do not find the coding in this switch persuasive:
> 
> 		switch (mode)
> 		{
> 			case BUFFER_LOCK_EXCLUSIVE:
> 				wait_event = WAIT_EVENT_BUFFER_EXCLUSIVE;
> 				break;
> 			case BUFFER_LOCK_SHARE_EXCLUSIVE:
> 				wait_event = WAIT_EVENT_BUFFER_SHARE_EXCLUSIVE;
> 				break;
> 			case BUFFER_LOCK_SHARE:
> 				wait_event = WAIT_EVENT_BUFFER_SHARED;
> 				break;
> 			case BUFFER_LOCK_UNLOCK:
> 				pg_unreachable();
> 
> 		}
> 
> It's not clear to me whether that's more about not believing
> pg_unreachable() or more about the lack of a default: case.

> I see that this is a modification of code that existed before
> fcb9c977a and wasn't being complained of, which makes it even
> stranger.

The code before did have a default:, so I guess that could be the
difference...

I can reproduce it here if I build with -Og or -O1, but not with -O2, so I
guess it's "just" the optimizer falling short somehow.

Huh, interesting, the optimization level dependency made me curious. Turns out
the warning goes away if I actually force the inlining of BufferLockAcquire()
with pg_attribute_always_inline. Presumably because then the compiler realizes
that the mode argument is always a constant that it can evaluate.

Can't quite decide between just using always_inline - after all the buffer
locking really is a rather crucial code path - and just initializing
wait_event to 0. Either seems better than using a default:.

Greetings,

Andres Freund

  71. Re: Buffer locking is special (hints, checksums, AIO writes)

    Tom Lane <tgl@sss.pgh.pa.us> — 2026-01-15T23:19:41Z

    Andres Freund <andres@anarazel.de> writes:
> Can't quite decide between just using always_inline - after all the buffer
> locking really is a rather crucial code path - and just initializing
> wait_event to 0. Either seems better than using a default:.

Yeah, I agree with not using a default: in case we ever extend
the mode enum.  I'd be inclined to fix it like this:

 			case BUFFER_LOCK_UNLOCK:
 				pg_unreachable();
+				/* silence possible compiler warning: */ 
+				wait_event = 0;
  		}

			regards, tom lane

  72. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-01-15T23:26:18Z

    On 2026-01-15 18:19:41 -0500, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > Can't quite decide between just using always_inline - after all the buffer
> > locking really is a rather crucial code path - and just initializing
> > wait_event to 0. Either seems better than using a default:.
> 
> Yeah, I agree with not using a default: in case we ever extend
> the mode enum.  I'd be inclined to fix it like this:
> 
>  			case BUFFER_LOCK_UNLOCK:
>  				pg_unreachable();
> +				/* silence possible compiler warning: */ 
> +				wait_event = 0;
>   		}

Unfortunately that doesn't seem to do the trick, at least not locally (gcc
14). I think gcc is considering the case where mode has a value outside of the
known values of the enum.  Hence needing initialize wait_event to zero in the
declaration to avoid the warning.

Greetings,

Andres Freund

  73. Re: Buffer locking is special (hints, checksums, AIO writes)

    Chao Li <li.evan.chao@gmail.com> — 2026-01-16T02:36:55Z

    
> On Jan 16, 2026, at 00:43, Andres Freund <andres@anarazel.de> wrote:
> 
> Hi,
> 
> On 2026-01-15 14:22:09 +0800, Chao Li wrote:
>> 3 - 0005 - bufmgr.c
>> ```
>> +inline void
>> +MarkBufferDirtyHint(Buffer buffer, bool buffer_std)
>> ```
>> 
>> It’s quite uncommon to extern an inline function. I think usually if we want
>> to make an inline function accessible from external, we define it “static
>> inline” in a header file. So, I guess “inline” is a typo here.
> 
> It works just fine to put an inline into the function definition, even if it's
> an external function. That hints the compiler to inline it inside the
> translation unit. Which is useful here, because it leads to
> MarkBufferDirtyHint() being inlined into BufferFinishSetHintBits().
> 

Technically, that for sure works. I raised the comment basically because it’s uncommon in the source tree. I just did a search for “inline” across all .c files, only found two non-static inline functions: ReadBufferExtended() and TrackNewBufferPin(), and they are all in bufmgr.c. So, adding a new one should be fine. I should have done the search yesterday while reviewing.

>> 8 - 0005 - fsmpage.c
>> ```
>> * needs to be updated. exclusive_lock_held should be set to true if the
>> * caller is already holding an exclusive lock, to avoid extra work.
>> ```
>> 
>> The function comment of fsm_search_avail() needs to be
>> updated. exclusive_lock_held should be set to true if the caller is already
>> holding a **share-exclusive or** exclusive lock.
> 
> Why?

Ah, I see. We will never use a share-exclusive lock for fsm_search_avail? Currently, there are two callers of fsm_search_avail, one use a share lock, the other uses an exclusive lock. My bad.

>> 10 - 0005 - freespace.c
>> ```
>> - * Reset the next slot pointer. This encourages the use of low-numbered
>> - * pages, increasing the chances that a later vacuum can truncate the
>> - * relation. We don't bother with marking the page dirty if it wasn't
>> - * already, since this is just a hint.
>> + * Try to reset the next slot pointer. This encourages the use of
>> + * low-numbered pages, increasing the chances that a later vacuum can
>> + * truncate the relation. We don't bother with marking the page dirty if
>> + * it wasn't already, since this is just a hint.
>> */
>> LockBuffer(buf, BUFFER_LOCK_SHARE);
>> - ((FSMPage) PageGetContents(page))->fp_next_slot = 0;
>> + if (BufferBeginSetHintBits(buf))
>> + {
>> + ((FSMPage) PageGetContents(page))->fp_next_slot = 0;
>> + BufferFinishSetHintBits(buf, false, false);
>> + }
>> ```
>> 
>> Before this patch, we unconditionally set fp_next_slot, now the setting
>> might be skipped. You have add “Try to” in the comment that has explained
>> the possibility of skipping setting fp_next_slot. Would it be better to add
>> a brief statement for what will result in when skipping setting
>> fp_next_slot? Something like “Skipping the update only affects reuse, not
>> correctness”.
> 
> I don't see the point. The whole paragraph is about how this isn't crucial.
> 

Yeah, I could understand the comment expresses that isn’t crucial, but just felt that’s not “explicit”. I am fine with the current comment. 

Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/

  74. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-01-16T12:02:46Z

    Hi,

On 2026-01-15 18:26:18 -0500, Andres Freund wrote:
> On 2026-01-15 18:19:41 -0500, Tom Lane wrote:
> > Andres Freund <andres@anarazel.de> writes:
> > > Can't quite decide between just using always_inline - after all the buffer
> > > locking really is a rather crucial code path - and just initializing
> > > wait_event to 0. Either seems better than using a default:.
> > 
> > Yeah, I agree with not using a default: in case we ever extend
> > the mode enum.  I'd be inclined to fix it like this:
> > 
> >  			case BUFFER_LOCK_UNLOCK:
> >  				pg_unreachable();
> > +				/* silence possible compiler warning: */ 
> > +				wait_event = 0;
> >   		}
> 
> Unfortunately that doesn't seem to do the trick, at least not locally (gcc
> 14). I think gcc is considering the case where mode has a value outside of the
> known values of the enum.  Hence needing initialize wait_event to zero in the
> declaration to avoid the warning.

I've now pushed the initialization of wait_event, as I'll be only semi-online
until late Monday.

Greetings,

Andres Freund

  75. Re: Buffer locking is special (hints, checksums, AIO writes)

    Alexander Lakhin <exclusion@gmail.com> — 2026-01-24T19:00:00Z

    Hello Andres,

16.01.2026 01:02, Tom Lane wrote:
> Various buildfarm animals are complaining about fcb9c977a,
> similarly to this from calliphoridae [1]:

I've discovered another anomaly introduced with fcb9c977a, this time
run-time:
for i in `seq 300`; do
echo "iteration $i"

echo "
create table t(f1 text);
create index on t using spgist(f1);
insert into t select 'a' from generate_series(1, 9000) g(i);
vacuum analyze t;
insert into t select 'b' from generate_series(1, 1000) g(i);
drop table t;
" | psql >/dev/null -v ON_ERROR_STOP=1 || break;
done

fails for me as below:
...
iteration 39
server closed the connection unexpectedly

Core was generated by `postgres: law regression [local] INSERT                 '.
Program terminated with signal SIGABRT, Aborted.
#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:44

warning: 44     ./nptl/pthread_kill.c: Нет такого файла или каталога
(gdb) bt
#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:44
#1  __pthread_kill_internal (signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:78
#2  __GI___pthread_kill (threadid=<optimized out>, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
#3  0x000073625aa4527e in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4  0x000073625aa288ff in __GI_abort () at ./stdlib/abort.c:79
#5  0x00005a479e2f685f in ExceptionalCondition (conditionName=conditionName@entry=0x5a479e403cb8 "entry->data.lockmode 
== BUFFER_LOCK_UNLOCK", fileName=fileName@entry=0x5a479e37a84f "bufmgr.c", lineNumber=lineNumber@entry=5908) at assert.c:65
#6  0x00005a479e14d87c in BufferLockConditional (buffer=<optimized out>, buf_hdr=0x73624e882b40, 
mode=mode@entry=BUFFER_LOCK_EXCLUSIVE) at bufmgr.c:5908
#7  0x00005a479e14f4d5 in ConditionalLockBuffer (buffer=buffer@entry=13500) at bufmgr.c:6474
#8  0x00005a479de44da9 in SpGistNewBuffer (index=index@entry=0x73625b1f6ce8) at spgutils.c:420
#9  0x00005a479de45397 in allocNewBuffer (index=index@entry=0x73625b1f6ce8, flags=flags@entry=3) at spgutils.c:528
#10 0x00005a479de456a0 in SpGistGetBuffer (index=index@entry=0x73625b1f6ce8, flags=flags@entry=3, needSpace=<optimized 
out>, needSpace@entry=4088, isNew=isNew@entry=0x7ffd8fa1b2a7) at spgutils.c:663
#11 0x00005a479de3c913 in doPickSplit (index=index@entry=0x73625b1f6ce8, state=state@entry=0x7ffd8fa1b740, 
current=current@entry=0x7ffd8fa1b510, parent=parent@entry=0x7ffd8fa1b530, 
newLeafTuple=newLeafTuple@entry=0x5a47cbd7adc8, level=level@entry=4, isNulls=false, isNew=false) at spgdoinsert.c:1046
#12 0x00005a479de3e542 in spgdoinsert (index=index@entry=0x73625b1f6ce8, state=state@entry=0x7ffd8fa1b740, 
heapPtr=heapPtr@entry=0x5a47cbe0b248, datums=datums@entry=0x7ffd8fa1b8d0, isnulls=isnulls@entry=0x7ffd8fa1b8b0) at 
spgdoinsert.c:2134
#13 0x00005a479de40137 in spginsert (index=0x73625b1f6ce8, values=0x7ffd8fa1b8d0, isnull=0x7ffd8fa1b8b0, 
ht_ctid=0x5a47cbe0b248, heapRel=<optimized out>, checkUnique=<optimized out>, indexUnchanged=false, 
indexInfo=0x5a47cbe0b0b8) at spginsert.c:206
#14 0x00005a479df981d8 in ExecInsertIndexTuples (resultRelInfo=resultRelInfo@entry=0x5a47cbe07120, 
slot=slot@entry=0x5a47cbe0b218, estate=estate@entry=0x5a47cbe06c10, update=update@entry=false, 
noDupErr=noDupErr@entry=false, specConflict=specConflict@entry=0x0, arbiterIndexes=0x0, onlySummarizing=false) at 
execIndexing.c:449
#15 0x00005a479dfcc6a9 in ExecInsert (context=context@entry=0x7ffd8fa1bb70, 
resultRelInfo=resultRelInfo@entry=0x5a47cbe07120, slot=slot@entry=0x5a47cbe0b218, canSetTag=<optimized out>, 
inserted_tuple=inserted_tuple@entry=0x0, insert_destrel=insert_destrel@entry=0x0) at nodeModifyTable.c:1240
#16 0x00005a479dfcdf67 in ExecModifyTable (pstate=0x5a47cbe06f10) at nodeModifyTable.c:4485
...

Best regards,
Alexander

  76. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-01-24T20:31:14Z

    Hi,

On 2026-01-24 21:00:00 +0200, Alexander Lakhin wrote:
> 16.01.2026 01:02, Tom Lane wrote:
> > Various buildfarm animals are complaining about fcb9c977a,
> > similarly to this from calliphoridae [1]:
>
> I've discovered another anomaly introduced with fcb9c977a, this time
> run-time:

With the other anomaly, you mean the spurious compiler warning or something
else?


> for i in `seq 300`; do
> echo "iteration $i"
>
> echo "
> create table t(f1 text);
> create index on t using spgist(f1);
> insert into t select 'a' from generate_series(1, 9000) g(i);
> vacuum analyze t;
> insert into t select 'b' from generate_series(1, 1000) g(i);
> drop table t;
> " | psql >/dev/null -v ON_ERROR_STOP=1 || break;
> done
>
> fails for me as below:
> ...
> iteration 39
> server closed the connection unexpectedly


Thanks for this report, particularly with the easy reproducer!  How did you
find this?


I think this is more likely to be a spgist bug, not a bug in the patch.  From
what I can tell, spgist tries to conditionally lock a buffer that it itself
already has locked exclusively - that's why the assertion is failing.

I reproduced this locally, and could see in a bt full stack that the buffer
that spgist is trying to lock conditionally, is also referenced as
newInnerBuffer in doPickSplit(). So it's not an issue of bufmgr.c loosing
track of which buffers are locked with what mode.

I haven't yet figured out why spgist ends up with a buffer it already is
using.

We could of course just accept this case and have the conditional lock
acquisition fail, but I think trying to conditionally lock a buffer that you
already lock is indicative of something having gone wrong.  But I'm open to
going there anyway, just to avoid causing problems with previously "working"
code.


Greetings,

Andres Freund

  77. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-01-24T21:03:04Z

    Hi,

On 2026-01-24 15:31:14 -0500, Andres Freund wrote:
> I think this is more likely to be a spgist bug, not a bug in the patch.  From
> what I can tell, spgist tries to conditionally lock a buffer that it itself
> already has locked exclusively - that's why the assertion is failing.
> 
> I reproduced this locally, and could see in a bt full stack that the buffer
> that spgist is trying to lock conditionally, is also referenced as
> newInnerBuffer in doPickSplit(). So it's not an issue of bufmgr.c loosing
> track of which buffers are locked with what mode.
> 
> I haven't yet figured out why spgist ends up with a buffer it already is
> using.
> 
> We could of course just accept this case and have the conditional lock
> acquisition fail, but I think trying to conditionally lock a buffer that you
> already lock is indicative of something having gone wrong.  But I'm open to
> going there anyway, just to avoid causing problems with previously "working"
> code.

Looking at the spgist code, and the README, I think we may need to accept the
uglines of silently failing when a backend tries to conditionally lock a
buffer that it itself has already locked.  Even though I still don't
understand how it happens in this this specific case, that doesn't even have
concurrency.

Pretty ... not great ... that spgist does stuff like extending a relation
while holding an exclusive buffer lock.

Greetings,

Andres Freund

  78. Re: Buffer locking is special (hints, checksums, AIO writes)

    Tom Lane <tgl@sss.pgh.pa.us> — 2026-01-24T21:11:47Z

    Andres Freund <andres@anarazel.de> writes:
> I think this is more likely to be a spgist bug, not a bug in the patch.  From
> what I can tell, spgist tries to conditionally lock a buffer that it itself
> already has locked exclusively - that's why the assertion is failing.

I dunno.  It looks to me like the previous LWLock-based implementation
of ConditionalLockBuffer() had no such restriction as

	/*
	 * We better not already hold a lock on the buffer.
	 */
	Assert(entry->data.lockmode == BUFFER_LOCK_UNLOCK);

Maybe I'm missing something, but it looks like the old code would
return false if the buffer was already locked, whether that lock
was held by our process or another one.  SPGist evidently has an
assumption in edge cases that that is the behavior, and I'm not
convinced that it's a good idea to change it.  There may be other
such edge cases we've not tripped over yet.

> We could of course just accept this case and have the conditional lock
> acquisition fail, but I think trying to conditionally lock a buffer that you
> already lock is indicative of something having gone wrong.

I don't really buy this argument.  Yes, within a single function it'd
be silly to lock a buffer and immediately try to lock it again, but
when you consider cases like recursive modifications of index state,
it's *far* from obvious that some lower recursion level might not try
to lock a buffer that some outer level already locked.  In the case at
hand I think it is probably driven by two recursion levels trying to
acquire free space out of the same buffer.  SPGist is expecting the
lower level to fail to get the lock and then go find some free space
elsewhere.  Yeah, we could probably re-code it to get that outcome
in another way, but why?

			regards, tom lane

  79. Re: Buffer locking is special (hints, checksums, AIO writes)

    Peter Geoghegan <pg@bowt.ie> — 2026-01-24T21:21:48Z

    On Sat, Jan 24, 2026 at 4:12 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Andres Freund <andres@anarazel.de> writes:
> > We could of course just accept this case and have the conditional lock
> > acquisition fail, but I think trying to conditionally lock a buffer that you
> > already lock is indicative of something having gone wrong.
>
> I don't really buy this argument.  Yes, within a single function it'd
> be silly to lock a buffer and immediately try to lock it again, but
> when you consider cases like recursive modifications of index state,
> it's *far* from obvious that some lower recursion level might not try
> to lock a buffer that some outer level already locked.  In the case at
> hand I think it is probably driven by two recursion levels trying to
> acquire free space out of the same buffer.

Even nbtree has to deal with this. Also in the context of free space
management. See the comments in _bt_allocbuf, particularly the ones
where we explicitly describe a common scenario where we conditionally
lock a buffer that our own caller/backend already has a lock on.

I actually agree with Andres' general sentiment about this kind of
coding pattern; it also seems sloppy to me. But it's hard to see how
we could do better in places such as _bt_allocbuf. At least within the
confines of the current FSM design.

-- 
Peter Geoghegan

  80. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-01-24T23:03:00Z

    Hi,

On 2026-01-24 16:11:47 -0500, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > I think this is more likely to be a spgist bug, not a bug in the patch.  From
> > what I can tell, spgist tries to conditionally lock a buffer that it itself
> > already has locked exclusively - that's why the assertion is failing.
> 
> I dunno.  It looks to me like the previous LWLock-based implementation
> of ConditionalLockBuffer() had no such restriction as
> 
> 	/*
> 	 * We better not already hold a lock on the buffer.
> 	 */
> 	Assert(entry->data.lockmode == BUFFER_LOCK_UNLOCK);
> 
> Maybe I'm missing something, but it looks like the old code would
> return false if the buffer was already locked, whether that lock
> was held by our process or another one.

Yea. I added that assert when (I think) Melanie complained that the new code
wouldn't detect repeated acquisitions or release of the same content lock as
nicely as before.  I guess I went a bit overboard and also added the assertion
to ConditionalLockBuffer().

I'll go and move it into the branch where we actually got the lock, that seems
worth continuing to do.


> > We could of course just accept this case and have the conditional lock
> > acquisition fail, but I think trying to conditionally lock a buffer that you
> > already lock is indicative of something having gone wrong.
> 
> I don't really buy this argument.  Yes, within a single function it'd
> be silly to lock a buffer and immediately try to lock it again, but
> when you consider cases like recursive modifications of index state,
> it's *far* from obvious that some lower recursion level might not try
> to lock a buffer that some outer level already locked.

Yea, there's probably something too that. But I'm not entirely convinced -
consider what happens with an index on a temporary table: A higher level think
it got a buffer with space, but then a lower level uses up that space...


> In the case at hand I think it is probably driven by two recursion levels
> trying to acquire free space out of the same buffer.  SPGist is expecting
> the lower level to fail to get the lock and then go find some free space
> elsewhere.  Yeah, we could probably re-code it to get that outcome in
> another way, but why?

Regardless of the assertion, it still feels like there may be something off
here. Why is the page marked as empty in the FSM, despite actually not being
empty?


Greetings,

Andres Freund

  81. Re: Buffer locking is special (hints, checksums, AIO writes)

    Tom Lane <tgl@sss.pgh.pa.us> — 2026-01-25T00:54:36Z

    Andres Freund <andres@anarazel.de> writes:
> On 2026-01-24 16:11:47 -0500, Tom Lane wrote:
>> In the case at hand I think it is probably driven by two recursion levels
>> trying to acquire free space out of the same buffer.  SPGist is expecting
>> the lower level to fail to get the lock and then go find some free space
>> elsewhere.  Yeah, we could probably re-code it to get that outcome in
>> another way, but why?

> Regardless of the assertion, it still feels like there may be something off
> here. Why is the page marked as empty in the FSM, despite actually not being
> empty?

Who said anything about its being empty?  There's X amount of space
used on the page now, the outer level is preparing to add a tuple
of size Y, the inner level is looking for a place to put a tuple
of size Z.  As long as X+Y+Z <= 8K, there's no free-space-related
reason the page wouldn't work for this.  We could actually try to
make it work, but that seems fragile to me: the outer level would
have to be prepared for the possibility that the page changes
under it despite holding exclusive lock.  I think it's reasonable
on complexity grounds to insist that the inner recursion level
go play someplace else.

			regards, tom lane

  82. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-01-29T17:27:10Z

    Hi,

On 2026-01-24 19:54:36 -0500, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > On 2026-01-24 16:11:47 -0500, Tom Lane wrote:
> >> In the case at hand I think it is probably driven by two recursion levels
> >> trying to acquire free space out of the same buffer.  SPGist is expecting
> >> the lower level to fail to get the lock and then go find some free space
> >> elsewhere.  Yeah, we could probably re-code it to get that outcome in
> >> another way, but why?
> 
> > Regardless of the assertion, it still feels like there may be something off
> > here. Why is the page marked as empty in the FSM, despite actually not being
> > empty?
> 
> Who said anything about its being empty?

In the crashes due to Alexander's repro that I have looked at, the page is
returned by SpGistNewBuffer()->GetFreeIndexPage(). Which afaict should only
contain empty pages. It's easy to see how that could happen with concurrency,
but there's none in the test.

I was just trying to repro this again while writing this message, and
interestingly I got the same issue in nbtree this time. Which a) confirms
Peter's statement that the "conditionally locking a buffer we already locked"
issue exists for nbtree b) makes me suspect something odd is happening around
indexfsm.


Anyway, independent of that, the behavior clearly needs to be allowed. Here's
a proposed patch.

At first I was thinking of just removing the assertion without anything else
in place - but I think that's not quite right: We could e.g. be trying to
acquire a share or share-exclusive lock when holding a share lock (or the
reverse), but we can't currently don't keep track of two different lock modes
for the same lock.  Therefore it seems safer to just define it so that
acquiring a conditional lock on a buffer that is already locked by us will
always fail, regardless of what existing lock mode we already hold.  I think
all current callers good with that.

Does that sound reasonable?

We could add support for locking the same buffer multiple times, but I don't
think it'd be worth the complexity and (small) overhead that would bring with
it?  It also seems like allowing that would make it more likely for a backend
to trample over its own state higher up in the call tree.

Greetings,

Andres Freund

  83. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-01-29T17:42:41Z

    Hi,

On 2026-01-29 12:27:10 -0500, Andres Freund wrote:
> In the crashes due to Alexander's repro that I have looked at, the page is
> returned by SpGistNewBuffer()->GetFreeIndexPage(). Which afaict should only
> contain empty pages. It's easy to see how that could happen with concurrency,
> but there's none in the test.

I think I see how that happens - we reenter the page into the FSM during the
VACUUM that's part of the repro, but we previously also "recorded" it with
SpGistSetLastUsedPage(), presumably before it was emptied during vacuum.  The
we reuse the page first via the "last used page" cache, making it not
empty. Then we get the same page via the FSM, which wasn't updated when we
started filling the page via the last-used-page mechanism.


> I was just trying to repro this again while writing this message, and
> interestingly I got the same issue in nbtree this time. Which a) confirms
> Peter's statement that the "conditionally locking a buffer we already locked"
> issue exists for nbtree b) makes me suspect something odd is happening around
> indexfsm.

Not sure how that happens in a single threaded workload yet, though.


> diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
> index 6f935648ae9..f5602f4e7e1 100644
> --- a/src/backend/storage/buffer/bufmgr.c
> +++ b/src/backend/storage/buffer/bufmgr.c
> @@ -5895,6 +5895,13 @@ BufferLockUnlock(Buffer buffer, BufferDesc *buf_hdr)
>  
>  /*
>   * Acquire the content lock for the buffer, but only if we don't have to wait.
> + *
> + * It is allowed to try to conditionally acquire a lock on a buffer that this
> + * backend has already locked, but the lock acquisition will always fail, even
> + * if the new lock acquisition does not conflict with an already held lock
> + * (e.g. two share locks). This is because we don't track per-backend
> + * ownership of multiple lock levels.  That is ok for the current uses of
> + * BufferLockConditional().
>   */
>  static bool
>  BufferLockConditional(Buffer buffer, BufferDesc *buf_hdr, BufferLockMode mode)
> @@ -5902,11 +5909,16 @@ BufferLockConditional(Buffer buffer, BufferDesc *buf_hdr, BufferLockMode mode)
>  	PrivateRefCountEntry *entry = GetPrivateRefCountEntry(buffer, true);
>  	bool		mustwait;
>  
> -	/*
> -	 * We better not already hold a lock on the buffer.
> -	 */
>  	Assert(entry->data.lockmode == BUFFER_LOCK_UNLOCK);
>  

Err, this should have been removed, I accidentally re-added the hunk while
experimenting.

Greetings,

Andres Freund

  84. Re: Buffer locking is special (hints, checksums, AIO writes)

    Peter Geoghegan <pg@bowt.ie> — 2026-01-29T17:50:38Z

    On Thu, Jan 29, 2026 at 12:42 PM Andres Freund <andres@anarazel.de> wrote:
> > -     /*
> > -      * We better not already hold a lock on the buffer.
> > -      */
> >       Assert(entry->data.lockmode == BUFFER_LOCK_UNLOCK);
> >
>
> Err, this should have been removed, I accidentally re-added the hunk while
> experimenting.

I've been running into this assertion failure from time to time while
working on index prefetching. It seems to happen after a hard crash.
I've just been running initdb whenever it happens. It would be nice to
not have to do this again.

-- 
Peter Geoghegan

  85. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-01-29T18:06:22Z

    Hi,

On 2026-01-29 12:50:38 -0500, Peter Geoghegan wrote:
> On Thu, Jan 29, 2026 at 12:42 PM Andres Freund <andres@anarazel.de> wrote:
> > > -     /*
> > > -      * We better not already hold a lock on the buffer.
> > > -      */
> > >       Assert(entry->data.lockmode == BUFFER_LOCK_UNLOCK);
> > >
> >
> > Err, this should have been removed, I accidentally re-added the hunk while
> > experimenting.
> 
> I've been running into this assertion failure from time to time while
> working on index prefetching. It seems to happen after a hard crash.
> I've just been running initdb whenever it happens. It would be nice to
> not have to do this again.

Sure, I'm planning to give folks a bit longer to chime in whether the proposed
behavior is sane and will then push (with an amended commit message and the
fixup discussed here).


After a crash it seems that btree often ends up getting the page from FSM that
it's currently inserting on. Which makes some sense, because presumably that
was the page we filled just before a crash. Wonder if - independent of this
issue - it could make sense to update the FSM during nbtree WAL recovery...

Greetings,

Andres Freund

  86. Re: Buffer locking is special (hints, checksums, AIO writes)

    Peter Geoghegan <pg@bowt.ie> — 2026-01-29T18:12:35Z

    On Thu, Jan 29, 2026 at 12:27 PM Andres Freund <andres@anarazel.de> wrote:
> I was just trying to repro this again while writing this message, and
> interestingly I got the same issue in nbtree this time. Which a) confirms
> Peter's statement that the "conditionally locking a buffer we already locked"
> issue exists for nbtree b) makes me suspect something odd is happening around
> indexfsm.

I don't think that there's anything mysterious about it. This is just
how index vacuuming does free space management. It's a consequence of
the fact that VACUUM notices that a page can go in the FSM at one
point, but only actually updates the FSM at some other point.

Nothing stops a backend from finding a recyclable page in the FSM
after a concurrent VACUUM decides that that same page can go in the
FSM, but before actually placing that page in the FSM.  VACUUM doesn't
consider that the FSM might have already known about that page *at
all* -- and so it certainly doesn't try to avoid these kinds of race
conditions. Hence the need for _bt_allocbuf to worry about buffer lock
deadlocks, including even self-deadlock.

-- 
Peter Geoghegan

  87. Re: Buffer locking is special (hints, checksums, AIO writes)

    Peter Geoghegan <pg@bowt.ie> — 2026-01-29T18:33:02Z

    On Thu, Jan 29, 2026 at 1:06 PM Andres Freund <andres@anarazel.de> wrote:
> Wonder if - independent of this
> issue - it could make sense to update the FSM during nbtree WAL recovery...

Maybe that would make sense. But I tend to think that we should have a
fully atomic, crash-safe approach to free space management.

Particularly in index AMs, where free space can only ever come in
BLCKSZ units -- the data structure/concurrency rules can be a lot
simpler if it only has to accommodate index AM requirements. Maybe the
WAL-logging could be built into existing index AM record types.

-- 
Peter Geoghegan

  88. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-01-29T19:29:27Z

    Hi,

On 2026-01-29 13:33:02 -0500, Peter Geoghegan wrote:
> On Thu, Jan 29, 2026 at 1:06 PM Andres Freund <andres@anarazel.de> wrote:
> > Wonder if - independent of this
> > issue - it could make sense to update the FSM during nbtree WAL recovery...
> 
> Maybe that would make sense. But I tend to think that we should have a
> fully atomic, crash-safe approach to free space management.

I agree that would be nice, but realistically (as you also say below) that
would have to be embedded into the WAL records that use the page that was
acquired from the FSM.  Maybe we could accept a dedicated WAL record for the
index case, but certainly not in the heap case.

Given that we'd need to embed the record somehow anyway, just adding, for now,
a RecordUsedIndexPage() to the redo of XLOG_BTREE_SPLIT* and
XLOG_BTREE_NEWROOT or such could make sense...

It doesn't seem like it'd be great to have a completely outdated index fsm
after a failover. If the index FSM on the newly promoted node is completely
outdated, due to having been copied at a much earlier time while there were a
lot of free pages, a _bt_allocbuf() could take quite a while...

I'm somewhat surprised it doesn't cause more performance issues to keep btree
pages exclusively locked while extending the relation... If that has to write
out pages and flush the WAL...


> Particularly in index AMs, where free space can only ever come in
> BLCKSZ units -- the data structure/concurrency rules can be a lot
> simpler if it only has to accommodate index AM requirements. Maybe the
> WAL-logging could be built into existing index AM record types.

Yea, I have my doubt that makes sense to share code between the index and heap
use cases. I doubt that having one FSM implementation support variable amount
of "space tracking granularity" really makes sense.

Greetings,

Andres Freund

  89. Re: Buffer locking is special (hints, checksums, AIO writes)

    Tom Lane <tgl@sss.pgh.pa.us> — 2026-01-29T20:24:23Z

    Andres Freund <andres@anarazel.de> writes:
> Anyway, independent of that, the behavior clearly needs to be allowed. Here's
> a proposed patch.

> At first I was thinking of just removing the assertion without anything else
> in place - but I think that's not quite right: We could e.g. be trying to
> acquire a share or share-exclusive lock when holding a share lock (or the
> reverse), but we can't currently don't keep track of two different lock modes
> for the same lock.  Therefore it seems safer to just define it so that
> acquiring a conditional lock on a buffer that is already locked by us will
> always fail, regardless of what existing lock mode we already hold.  I think
> all current callers good with that.

> Does that sound reasonable?

I didn't read the patch, but I agree with this description of what the
behavior should be.

> We could add support for locking the same buffer multiple times, but I don't
> think it'd be worth the complexity and (small) overhead that would bring with
> it?

Also agreed --- I think that's behavior we actively don't want.

> It also seems like allowing that would make it more likely for a backend
> to trample over its own state higher up in the call tree.

Precisely.

			regards, tom lane

  90. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-01-29T21:49:36Z

    Hi,

On 2026-01-29 13:06:22 -0500, Andres Freund wrote:
> On 2026-01-29 12:50:38 -0500, Peter Geoghegan wrote:
> > On Thu, Jan 29, 2026 at 12:42 PM Andres Freund <andres@anarazel.de> wrote:
> > > > -     /*
> > > > -      * We better not already hold a lock on the buffer.
> > > > -      */
> > > >       Assert(entry->data.lockmode == BUFFER_LOCK_UNLOCK);
> > > >
> > >
> > > Err, this should have been removed, I accidentally re-added the hunk while
> > > experimenting.
> > 
> > I've been running into this assertion failure from time to time while
> > working on index prefetching. It seems to happen after a hard crash.
> > I've just been running initdb whenever it happens. It would be nice to
> > not have to do this again.
> 
> Sure, I'm planning to give folks a bit longer to chime in whether the proposed
> behavior is sane and will then push (with an amended commit message and the
> fixup discussed here).

Pushed the fix now.

Greetings,

Andres Freund

  91. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-02-02T22:33:10Z

    Hi,

On 2026-01-14 16:20:58 -0500, Andres Freund wrote:
> I'm now working on cleaning up the last two commits. The most crucial bit is
> to simplify what happens in MarkSharedBufferDirtyHint(), we afaict can delete
> the use of DELAY_CHKPT_START etc and just go to marking the buffer dirty first
> and then do the WAL logging, just like normal WAL logging. The previous order
> was only required because we were dirtying the page while holding only a
> shared lock, which did not conflict with the lock held by SyncBuffers() etc.

I've been working on that.

- A lot of what was special about MarkBufferDirtyHint() isn't needed anymore:

  - The "abnormal" order of WAL logging before marking the buffer dirty was
    only needed because we marked buffers dirty. Which in turn was only needed
    because setting hint bits didn't conflict with flushing the page. With
    share-exclusive they do conflict, and we can switch to the normal order of
    operations, where marking a buffer dirty makes checkpoint wait when the
    buffer is encountered (due to wanting to flush the buffer but not getting
    the lock)


  - Now that we use the normal order of WAL logging, we don't need to delay
    checkpoint starts anymore.

    I think the explanation for why that is ok is correct [1], but it needs to
    be looked at by somebody with experience around this. Maybe Heikki?


  - Thanks to holding share-exclusive lock, nothing can concurrently dirty or
    undirty the buffer. Therefore the comments about spurious failures to mark
    the buffer dirty can be removed.


- I realized that, now that buffers cannot be dirtied while IO is ongoing, we
  don't need BM_JUST_DIRTIED anymore.


- The way MarkBufferDirtyHint() operates was copied into
  heap_inplace_update_and_unlock(). Now that MarkBufferDirtyHint() won't work
  that way anymore, it seems better to go with the alternative approach the
  comments already outlined, namely to only delay updating of the buffer
  contents.

  I've done this in a prequisite commit, as it doesn't actually depend on any
  of the other changes.  Noah, any chance you could take a look at this?


- Lots of minor polish


Greetings,

Andres Freund

[1]
	/*
	 * Update RedoRecPtr so that we can make the right decision. It's possible
	 * that a new checkpoint will start just after GetRedoRecPtr(), but that
	 * is ok, as the buffer is already dirty, ensuring that any BufferSync()
	 * started after the buffer was marked dirty cannot complete without
	 * flushing this buffer.  If a checkpoint started between marking the
	 * buffer dirty and this check, we will emit an unnecessary WAL record (as
	 * the buffer will be written out as part of the checkpoint), but the
	 * window for that is small.
	 */

  92. Re: Buffer locking is special (hints, checksums, AIO writes)

    Kirill Reshke <reshkekirill@gmail.com> — 2026-02-06T21:18:21Z

    On Tue, 3 Feb 2026 at 03:33, Andres Freund <andres@anarazel.de> wrote:
>
> Hi,
>
> On 2026-01-14 16:20:58 -0500, Andres Freund wrote:
> > I'm now working on cleaning up the last two commits. The most crucial bit is
> > to simplify what happens in MarkSharedBufferDirtyHint(), we afaict can delete
> > the use of DELAY_CHKPT_START etc and just go to marking the buffer dirty first
> > and then do the WAL logging, just like normal WAL logging. The previous order
> > was only required because we were dirtying the page while holding only a
> > shared lock, which did not conflict with the lock held by SyncBuffers() etc.
>
> I've been working on that.
>
> - A lot of what was special about MarkBufferDirtyHint() isn't needed anymore:
>
>   - The "abnormal" order of WAL logging before marking the buffer dirty was
>     only needed because we marked buffers dirty. Which in turn was only needed
>     because setting hint bits didn't conflict with flushing the page. With
>     share-exclusive they do conflict, and we can switch to the normal order of
>     operations, where marking a buffer dirty makes checkpoint wait when the
>     buffer is encountered (due to wanting to flush the buffer but not getting
>     the lock)
>
>
>   - Now that we use the normal order of WAL logging, we don't need to delay
>     checkpoint starts anymore.
>
>     I think the explanation for why that is ok is correct [1], but it needs to
>     be looked at by somebody with experience around this. Maybe Heikki?
>
>
>   - Thanks to holding share-exclusive lock, nothing can concurrently dirty or
>     undirty the buffer. Therefore the comments about spurious failures to mark
>     the buffer dirty can be removed.
>
>
> - I realized that, now that buffers cannot be dirtied while IO is ongoing, we
>   don't need BM_JUST_DIRTIED anymore.
>
>
> - The way MarkBufferDirtyHint() operates was copied into
>   heap_inplace_update_and_unlock(). Now that MarkBufferDirtyHint() won't work
>   that way anymore, it seems better to go with the alternative approach the
>   comments already outlined, namely to only delay updating of the buffer
>   contents.
>
>   I've done this in a prequisite commit, as it doesn't actually depend on any
>   of the other changes.  Noah, any chance you could take a look at this?
>
>
> - Lots of minor polish
>
>
> Greetings,
>
> Andres Freund
>
> [1]
>         /*
>          * Update RedoRecPtr so that we can make the right decision. It's possible
>          * that a new checkpoint will start just after GetRedoRecPtr(), but that
>          * is ok, as the buffer is already dirty, ensuring that any BufferSync()
>          * started after the buffer was marked dirty cannot complete without
>          * flushing this buffer.  If a checkpoint started between marking the
>          * buffer dirty and this check, we will emit an unnecessary WAL record (as
>          * the buffer will be written out as part of the checkpoint), but the
>          * window for that is small.
>          */

Hi!
I was reviewing new patches in this thread, and noticed your changes
in v12-0002, in gistkillitems. This makes me think you can be
interested in reviewing [0].

Anyway, v12-0003, on other patches I don't have an opinion (yet).

[0] https://commitfest.postgresql.org/patch/6399/
-- 
Best regards,
Kirill Reshke

  93. Re: Buffer locking is special (hints, checksums, AIO writes)

    Heikki Linnakangas <hlinnaka@iki.fi> — 2026-02-07T10:44:25Z

    On 03/02/2026 00:33, Andres Freund wrote:
> - The way MarkBufferDirtyHint() operates was copied into
>    heap_inplace_update_and_unlock(). Now that MarkBufferDirtyHint() won't work
>    that way anymore, it seems better to go with the alternative approach the
>    comments already outlined, namely to only delay updating of the buffer
>    contents.
> 
>    I've done this in a prequisite commit, as it doesn't actually depend on any
>    of the other changes.  Noah, any chance you could take a look at this?

Patch 0001 Looks correct to me. However:

> 	 * ["D" is a VACUUM (ONLY_DATABASE_STATS)]
> 	 * ["R" is a VACUUM tbl]
> 	 * D: vac_update_datfrozenxid() -> systable_beginscan(pg_class)
> 	 * D: systable_getnext() returns pg_class tuple of tbl
> 	 * R: memcpy() into pg_class tuple of tbl
> 	 * D: raise pg_database.datfrozenxid, XLogInsert(), finish
> 	 * [crash]
> 	 * [recovery restores datfrozenxid w/o relfrozenxid]
> 	 *
> 	 * As we hold an exclusive lock - preventing the buffer from being written
> 	 * out once dirty - we can work around this as follows: MarkBufferDirty(),
> 	 * XLogInsert(), memcpy().

That last reference to 'memcpy' is a little orphaned now. The comment 
used to talk about the stack copy of the page, but now there's no 
mention of that except for this reference to memcpy(). To make things 
worse, the steps have "memcpy() into pg_class tuple of tbl", so one 
could think that the "memcpy" refers to that.

How about this:

	 * We avoid that by using a temporary copy of the buffer to hide our
	 * change from other backends until it's been WAL-logged. We apply our
	 * change to the temporary copy and WAL-log it before modifying the real
	 * page. That way any action a reader of the in-place-updated value takes
	 * will be WAL logged after this change.

- Heikki

  94. Re: Buffer locking is special (hints, checksums, AIO writes)

    Heikki Linnakangas <hlinnaka@iki.fi> — 2026-02-07T12:38:53Z

    On 03/02/2026 00:33, Andres Freund wrote:
>    - Now that we use the normal order of WAL logging, we don't need to delay
>      checkpoint starts anymore.
> 
>      I think the explanation for why that is ok is correct [1], but it needs to
>      be looked at by somebody with experience around this. Maybe Heikki?

So that's patch 0004 "bufmgr: Switch to standard order in 
MarkBufferDirtyHint()". Yes, looks correct to me.

> 	/*
> 	 * Update RedoRecPtr so that we can make the right decision. It's possible
> 	 * that a new checkpoint will start just after GetRedoRecPtr(), but that
> 	 * is ok, as the buffer is already dirty, ensuring that any BufferSync()
> 	 * started after the buffer was marked dirty cannot complete without
> 	 * flushing this buffer.  If a checkpoint started between marking the
> 	 * buffer dirty and this check, we will emit an unnecessary WAL record (as
> 	 * the buffer will be written out as part of the checkpoint), but the
> 	 * window for that is small.
> 	 */
> 	RedoRecPtr = GetRedoRecPtr();

That "small window" is actually pretty big if you think of it a little 
more loosely. Our rule is that we write the full page image if a 
checkpoint has started since the page LSN, but that's very conservative 
already. It would be sufficient to write the full page image only if the 
checkpoint has already flushed the page. This small window is just a 
special case of that conservatism.

I've been thinking of trying track that more accurately for a long time, 
because it would smoothen the WAL spike when a checkpoint begins.

That gets off-topic, but my point is that it feels a little silly to 
mention that small window when there's the other giant panoramic window 
next to it.

- Heikki

  95. Re: Buffer locking is special (hints, checksums, AIO writes)

    Heikki Linnakangas <hlinnaka@iki.fi> — 2026-02-07T12:59:34Z

    A few minor nitpicks on v12 below. Other than these and the comments I 
wrote in separate emails, looks good to me.

> @@ -371,8 +382,6 @@ _bt_killitems(IndexScanDesc scan)
>         }
>  
>         /*
> -        * Since this can be redone later if needed, mark as dirty hint.
> -        *
>          * Whenever we mark anything LP_DEAD, we also set the page's
>          * BTP_HAS_GARBAGE flag, which is likewise just a hint.  (Note that we
>          * only rely on the page-level flag in !heapkeyspace indexes.)

Seems a bit random to remove that.

> +/*
> + * Try to set a single hint bit in a buffer.
> + *
> + * This is a bit faster than BufferBeginSetHintBits() /
> + * BufferFinishSetHintBits() when setting a single hint bit, but slower than
> + * the former when setting several hint bits.
> + */
> +bool
> +BufferSetHintBits16(uint16 *ptr, uint16 val, Buffer buffer)

This could use some more explanation. The point is that this does "*ptr 
= val", if it's allowed to set hint bits. That's not obvious. And 
"single hint bit" isn't really accurate, as you could update multiple 
bits in *ptr with one call.

> 	/*
> 	 * If the buffer was dirty, try to write it out.  There is a race
> 	 * condition here, in that someone might dirty it after we released the
> 	 * buffer header lock above.  We will recheck the dirty bit after
> 	 * re-locking the buffer header.
> 	 */

It's not clear what "above" means in that paragraph. Where do we release 
the buffer header lock? In StrategyGetBuffer?

(This is not actually new with this patch; it goes back to commit 
5e89985928. Before that, there was a call to PinBuffer_Locked() which 
released the spinlock.)

> @@ -2516,18 +2515,21 @@ again:
>                 /*
>                  * If using a nondefault strategy, and writing the buffer would
>                  * require a WAL flush, let the strategy decide whether to go ahead
> -                * and write/reuse the buffer or to choose another victim.  We need a
> -                * lock to inspect the page LSN, so this can't be done inside
> +                * and write/reuse the buffer or to choose another victim.  We need to
> +                * hold the content lock in at least share-exclusive mode to safely
> +                * inspect the page LSN, so this couldn't have been done inside
>                  * StrategyGetBuffer.
>                  */
>                 if (strategy != NULL)
>                 {
>                         XLogRecPtr      lsn;
>  
> -                       /* Read the LSN while holding buffer header lock */
> -                       buf_state = LockBufHdr(buf_hdr);
> +                       /*
> +                        * As we now hold at least a share-exclusive lock on the buffer,
> +                        * the LSN cannot change during the flush (and thus can't be
> +                        * torn).
> +                        */
>                         lsn = BufferGetLSN(buf_hdr);
> -                       UnlockBufHdr(buf_hdr);
>  
>                         if (XLogNeedsFlush(lsn)
>                                 && StrategyRejectBuffer(strategy, buf_hdr, from_ring))

I think the second comment is redundant with the first one. Let's just 
remove it.

> +/*
> + * Helper for BufferBeginSetHintBits() and BufferSetHintBits16().
> + *
> + * This checks if the current lock mode already suffices to allow hint bits
> + * being set and, if not, whether the current lock can be upgraded.
> + *
> + * Updates *lockstate when returning true.
> + */
> +static inline bool
> +SharedBufferBeginSetHintBits(Buffer buffer, BufferDesc *buf_hdr, uint64 *lockstate)

Would be good to be more explicit what returning true/false here means.

- Heikki

  96. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-02-08T18:38:42Z

    Hi,

On 2026-02-07 14:38:53 +0200, Heikki Linnakangas wrote:
> On 03/02/2026 00:33, Andres Freund wrote:
> >    - Now that we use the normal order of WAL logging, we don't need to delay
> >      checkpoint starts anymore.
> >
> >      I think the explanation for why that is ok is correct [1], but it needs to
> >      be looked at by somebody with experience around this. Maybe Heikki?
>
> So that's patch 0004 "bufmgr: Switch to standard order in
> MarkBufferDirtyHint()". Yes, looks correct to me.

Thanks for checking!  Somehow I went back and forth about it being right
multiple times...


> > 	/*
> > 	 * Update RedoRecPtr so that we can make the right decision. It's possible
> > 	 * that a new checkpoint will start just after GetRedoRecPtr(), but that
> > 	 * is ok, as the buffer is already dirty, ensuring that any BufferSync()
> > 	 * started after the buffer was marked dirty cannot complete without
> > 	 * flushing this buffer.  If a checkpoint started between marking the
> > 	 * buffer dirty and this check, we will emit an unnecessary WAL record (as
> > 	 * the buffer will be written out as part of the checkpoint), but the
> > 	 * window for that is small.
> > 	 */
> > 	RedoRecPtr = GetRedoRecPtr();
>
> That "small window" is actually pretty big if you think of it a little more
> loosely. Our rule is that we write the full page image if a checkpoint has
> started since the page LSN, but that's very conservative already. It would
> be sufficient to write the full page image only if the checkpoint has
> already flushed the page. This small window is just a special case of that
> conservatism.

I mainly want to mention that window because I have to think about it when
analyzing the correctness of the approach. If the window is not mentioned, at
least I have to think about whether the window is dangerous in some form.


> It would be sufficient to write the full page image only if the checkpoint
> has already flushed the page.

Today that would probably not quite be sufficient, due to issues around
re-dirtying the page during checkpointer's flush (and thus needing to be
written out again, with the chance of a torn write that has no FPI to repair
it). But that will soon be impossible.


I think the actual rule would need to be more complicated, I think we would
need to generate an FPI for the first modification after the checkpoint flush,
even though the LSN is newer than the redo LSN, because we didn't generate one
earlier?  Otherwise we could get into a situation where there is no non-torn
on-disk page version after a later crash, I think?

Consider:

1) modify page w/ FPI
2) redo pointer determined at X
3) modify page w/o FPI, as the page hasn't yet been flushed at X+1
4) checkpointer flushes page
5) checkpoint completes, at X+2
6) page is dirtied, w/o FPI X+3, as X+1 > X
7) in the middle of writing out the page, we crash, the page is torn

For recovery we will replay starting from position X. Then will replay the
record from 3), which will be skipped due to the LSN. Then we will replay X+3,
which either will be skipped due to the LSN condition (if the page header
survived the torn page), leading to the changes to the "old portion" of the
torn page not being replayed, or we will replay the WAL record, applying it to
a torn page (or failing to read in the page due to checksum errors).

If we only needed to think about buffers that stay in memory, we could "just"
tackle this by remember that the page will need to be FPId during the next
modification in the BufferDesc, but that doesn't help us if the page is
evicted and reread...



> I've been thinking of trying track that more accurately for a long time,
> because it would smoothen the WAL spike when a checkpoint begins.

It'd indeed be nice to improve that. Another thing it'd be helpful is widening
when we can write out hint bits on standbys.

If the rule were just that we can skip an FPI if the page still needs to be
written out by the checkpoint, it'd be fairly simple - we could utilize
BM_CHECKPOINT_NEEDED. But as hinted at above, I think it's a it more
complicated.


Greetings,

Andres Freund

  97. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-02-09T01:52:41Z

    Hi,

On 2026-02-07 14:59:34 +0200, Heikki Linnakangas wrote:
> A few minor nitpicks on v12 below. Other than these and the comments I wrote
> in separate emails, looks good to me.
> 
> > @@ -371,8 +382,6 @@ _bt_killitems(IndexScanDesc scan)
> >         }
> >         /*
> > -        * Since this can be redone later if needed, mark as dirty hint.
> > -        *
> >          * Whenever we mark anything LP_DEAD, we also set the page's
> >          * BTP_HAS_GARBAGE flag, which is likewise just a hint.  (Note that we
> >          * only rely on the page-level flag in !heapkeyspace indexes.)
> 
> Seems a bit random to remove that.

Fair.


> > +/*
> > + * Try to set a single hint bit in a buffer.
> > + *
> > + * This is a bit faster than BufferBeginSetHintBits() /
> > + * BufferFinishSetHintBits() when setting a single hint bit, but slower than
> > + * the former when setting several hint bits.
> > + */
> > +bool
> > +BufferSetHintBits16(uint16 *ptr, uint16 val, Buffer buffer)
> 
> This could use some more explanation. The point is that this does "*ptr =
> val", if it's allowed to set hint bits. That's not obvious. And "single hint
> bit" isn't really accurate, as you could update multiple bits in *ptr with
> one call.

Agreed.  I updated it to

 * Try to set hint bits on a single 16bit value in a buffer.
 *
 * If hint bits are allowed to be set, set *ptr = val, try mark the buffer
 * dirty and return true. Otherwise false is returned.
 *
 * *ptr needs to be a pointer to memory within the buffer.
 *
 * This is a bit faster than BufferBeginSetHintBits() /
 * BufferFinishSetHintBits() when setting hints once in a buffer, but slower
 * than the former when setting hint bits multiple times in the same buffer.



> > 	/*
> > 	 * If the buffer was dirty, try to write it out.  There is a race
> > 	 * condition here, in that someone might dirty it after we released the
> > 	 * buffer header lock above.  We will recheck the dirty bit after
> > 	 * re-locking the buffer header.
> > 	 */
> 
> It's not clear what "above" means in that paragraph. Where do we release the
> buffer header lock? In StrategyGetBuffer?
>
> (This is not actually new with this patch; it goes back to commit
> 5e89985928. Before that, there was a call to PinBuffer_Locked() which
> released the spinlock.)

Yea, looks like I should have edited the comment in that commit. Updated to:

	 * If the buffer was dirty, try to write it out.  There is a race
	 * condition here, another backend could dirty the buffer between
	 * StrategyGetBuffer() checking that it is not in use and invalidating the
	 * buffer below. That's addressed by InvalidateVictimBuffer() verifying
	 * that the buffer is not dirty.


> > @@ -2516,18 +2515,21 @@ again:
> >                 /*
> >                  * If using a nondefault strategy, and writing the buffer would
> >                  * require a WAL flush, let the strategy decide whether to go ahead
> > -                * and write/reuse the buffer or to choose another victim.  We need a
> > -                * lock to inspect the page LSN, so this can't be done inside
> > +                * and write/reuse the buffer or to choose another victim.  We need to
> > +                * hold the content lock in at least share-exclusive mode to safely
> > +                * inspect the page LSN, so this couldn't have been done inside
> >                  * StrategyGetBuffer.
> >                  */
> >                 if (strategy != NULL)
> >                 {
> >                         XLogRecPtr      lsn;
> > -                       /* Read the LSN while holding buffer header lock */
> > -                       buf_state = LockBufHdr(buf_hdr);
> > +                       /*
> > +                        * As we now hold at least a share-exclusive lock on the buffer,
> > +                        * the LSN cannot change during the flush (and thus can't be
> > +                        * torn).
> > +                        */
> >                         lsn = BufferGetLSN(buf_hdr);
> > -                       UnlockBufHdr(buf_hdr);
> >                         if (XLogNeedsFlush(lsn)
> >                                 && StrategyRejectBuffer(strategy, buf_hdr, from_ring))
> 
> I think the second comment is redundant with the first one. Let's just
> remove it.

Agreed.


> > +/*
> > + * Helper for BufferBeginSetHintBits() and BufferSetHintBits16().
> > + *
> > + * This checks if the current lock mode already suffices to allow hint bits
> > + * being set and, if not, whether the current lock can be upgraded.
> > + *
> > + * Updates *lockstate when returning true.
> > + */
> > +static inline bool
> > +SharedBufferBeginSetHintBits(Buffer buffer, BufferDesc *buf_hdr, uint64 *lockstate)
> 
> Would be good to be more explicit what returning true/false here means.

Hm, ISTM that'd just end up restating the comments for
BufferBeginSetHintBits(). Given SharedBufferBeginSetHintBits() is just an
implementation detail for BufferBeginSetHintBits/BufferSetHintBits16, I don't
think it's worth restating the details here.

Greetings,

Andres Freund

  98. Re: Buffer locking is special (hints, checksums, AIO writes)

    Heikki Linnakangas <hlinnaka@iki.fi> — 2026-02-09T10:14:47Z

    On 09/02/2026 03:52, Andres Freund wrote:
> On 2026-02-07 14:59:34 +0200, Heikki Linnakangas wrote:
>>> +/*
>>> + * Try to set a single hint bit in a buffer.
>>> + *
>>> + * This is a bit faster than BufferBeginSetHintBits() /
>>> + * BufferFinishSetHintBits() when setting a single hint bit, but slower than
>>> + * the former when setting several hint bits.
>>> + */
>>> +bool
>>> +BufferSetHintBits16(uint16 *ptr, uint16 val, Buffer buffer)
>>
>> This could use some more explanation. The point is that this does "*ptr =
>> val", if it's allowed to set hint bits. That's not obvious. And "single hint
>> bit" isn't really accurate, as you could update multiple bits in *ptr with
>> one call.
> 
> Agreed.  I updated it to
> 
>   * Try to set hint bits on a single 16bit value in a buffer.
>   *
>   * If hint bits are allowed to be set, set *ptr = val, try mark the buffer
>   * dirty and return true. Otherwise false is returned.
>   *
>   * *ptr needs to be a pointer to memory within the buffer.
>   *
>   * This is a bit faster than BufferBeginSetHintBits() /
>   * BufferFinishSetHintBits() when setting hints once in a buffer, but slower
>   * than the former when setting hint bits multiple times in the same buffer.

+1. Instead of "try mark the buffer dirty", I'd say just "mark the 
buffer dirty". The only reason it might not to mark the buffer dirty is 
that it was already marked dirty, right? I wouldn't call that a failure.

>>> 	/*
>>> 	 * If the buffer was dirty, try to write it out.  There is a race
>>> 	 * condition here, in that someone might dirty it after we released the
>>> 	 * buffer header lock above.  We will recheck the dirty bit after
>>> 	 * re-locking the buffer header.
>>> 	 */
>>
>> It's not clear what "above" means in that paragraph. Where do we release the
>> buffer header lock? In StrategyGetBuffer?
>>
>> (This is not actually new with this patch; it goes back to commit
>> 5e89985928. Before that, there was a call to PinBuffer_Locked() which
>> released the spinlock.)
> 
> Yea, looks like I should have edited the comment in that commit. Updated to:
> 
> 	 * If the buffer was dirty, try to write it out.  There is a race
> 	 * condition here, another backend could dirty the buffer between
> 	 * StrategyGetBuffer() checking that it is not in use and invalidating the
> 	 * buffer below. That's addressed by InvalidateVictimBuffer() verifying
> 	 * that the buffer is not dirty.

+1

>>> +/*
>>> + * Helper for BufferBeginSetHintBits() and BufferSetHintBits16().
>>> + *
>>> + * This checks if the current lock mode already suffices to allow hint bits
>>> + * being set and, if not, whether the current lock can be upgraded.
>>> + *
>>> + * Updates *lockstate when returning true.
>>> + */
>>> +static inline bool
>>> +SharedBufferBeginSetHintBits(Buffer buffer, BufferDesc *buf_hdr, uint64 *lockstate)
>>
>> Would be good to be more explicit what returning true/false here means.
> 
> Hm, ISTM that'd just end up restating the comments for
> BufferBeginSetHintBits(). Given SharedBufferBeginSetHintBits() is just an
> implementation detail for BufferBeginSetHintBits/BufferSetHintBits16, I don't
> think it's worth restating the details here.

Ok fair.

- Heikki

  99. Re: Buffer locking is special (hints, checksums, AIO writes)

    Antonin Houska <ah@cybertec.at> — 2026-02-09T11:42:25Z

    Andres Freund <andres@anarazel.de> wrote:

> On 2026-01-12 12:45:03 -0500, Andres Freund wrote:
> > I'm doing another pass through 0003 and will push that if I don't find
> > anything significant.
> 
> Done, after adjust two comments in minor ways.

I suppose this is commit 0b96e734c590.

While troubleshooting REPACK issue [1], I realized that
HeapTupleSatisfiesMVCCBatch() can also be called during logical decoding - in
that case we need to use a historic MVCC snapshot. My proposal to fix the
problem is attached.

[1] https://www.postgresql.org/message-id/CADzfLwWNv5QDn6qmxCRV-p_ijSTGwNcEZFCOXt09+RmpSG2=+w@mail.gmail.com

-- 
Antonin Houska
Web: https://www.cybertec-postgresql.com

  100. Re: Buffer locking is special (hints, checksums, AIO writes)

    Kirill Reshke <reshkekirill@gmail.com> — 2026-02-09T19:54:27Z

    On Sun, 8 Feb 2026 at 23:38, Andres Freund <andres@anarazel.de> wrote:
>

> Consider:
>
> 1) modify page w/ FPI
> 2) redo pointer determined at X
> 3) modify page w/o FPI, as the page hasn't yet been flushed at X+1
> 4) checkpointer flushes page
> 5) checkpoint completes, at X+2
> 6) page is dirtied, w/o FPI X+3, as X+1 > X
> 7) in the middle of writing out the page, we crash, the page is torn
>
> For recovery we will replay starting from position X. Then will replay the
> record from 3), which will be skipped due to the LSN. Then we will replay X+3,
> which either will be skipped due to the LSN condition (if the page header
> survived the torn page), leading to the changes to the "old portion" of the
> torn page not being replayed, or we will replay the WAL record, applying it to
> a torn page (or failing to read in the page due to checksum errors).
>
> If we only needed to think about buffers that stay in memory, we could "just"
> tackle this by remember that the page will need to be FPId during the next
> modification in the BufferDesc, but that doesn't help us if the page is
> evicted and reread...
>
>

Hmm, after thinking about this, I wonder if we can actually have a TAP
test for this sequence of events?
Maybe it would be desirable to execute some rare recovery code path.
But I'm unsure if there is any reliable way to have an OS to have a
buffer in page cache, but not on disk when evicted.


-- 
Best regards,
Kirill Reshke

  101. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-02-09T22:16:38Z

    Hi,

On 2026-02-09 12:42:25 +0100, Antonin Houska wrote:
> Andres Freund <andres@anarazel.de> wrote:
> 
> > On 2026-01-12 12:45:03 -0500, Andres Freund wrote:
> > > I'm doing another pass through 0003 and will push that if I don't find
> > > anything significant.
> > 
> > Done, after adjust two comments in minor ways.
> 
> I suppose this is commit 0b96e734c590.
> 
> While troubleshooting REPACK issue [1], I realized that
> HeapTupleSatisfiesMVCCBatch() can also be called during logical decoding - in
> that case we need to use a historic MVCC snapshot.

Huh. Indeed. That's unintentional - the path should never have been reached,
we are checking that an MVCC snapshot is used. Unfortunately, somebody
(i.e. probably me) at some point defined the relevant macro as

/* This macro encodes the knowledge of which snapshots are MVCC-safe */
#define IsMVCCSnapshot(snapshot)  \
	((snapshot)->snapshot_type == SNAPSHOT_MVCC || \
	 (snapshot)->snapshot_type == SNAPSHOT_HISTORIC_MVCC)

Which makes sense for some places, but not for plenty others.

The reason this didn't cause more widespread issues is that during logical
decoding we mostly don't use sequential scans etc that are affected by the
these paths.


> My proposal to fix the problem is attached.

That's imo not at all the right fix - it'd make visibility during seqscans
checking noticeably slower.


I think we ought to instead restrict the page-at-a-time scans to only happen
with "real" mvcc snapshots. I.e. this:

	/*
	 * Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
	 */
	if (!(snapshot && IsMVCCSnapshot(snapshot)))
		scan->rs_base.rs_flags &= ~SO_ALLOW_PAGEMODE;

should trigger for historic snapshots as well.


Does that fix the issue for you?


What's your reproducer?


Greetings,

Andres Freund

  102. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-02-09T22:19:06Z

    Hi,

On 2026-02-09 12:14:47 +0200, Heikki Linnakangas wrote:
> On 09/02/2026 03:52, Andres Freund wrote:
> > On 2026-02-07 14:59:34 +0200, Heikki Linnakangas wrote:
> > > > +/*
> > > > + * Try to set a single hint bit in a buffer.
> > > > + *
> > > > + * This is a bit faster than BufferBeginSetHintBits() /
> > > > + * BufferFinishSetHintBits() when setting a single hint bit, but slower than
> > > > + * the former when setting several hint bits.
> > > > + */
> > > > +bool
> > > > +BufferSetHintBits16(uint16 *ptr, uint16 val, Buffer buffer)
> > > 
> > > This could use some more explanation. The point is that this does "*ptr =
> > > val", if it's allowed to set hint bits. That's not obvious. And "single hint
> > > bit" isn't really accurate, as you could update multiple bits in *ptr with
> > > one call.
> > 
> > Agreed.  I updated it to
> > 
> >   * Try to set hint bits on a single 16bit value in a buffer.
> >   *
> >   * If hint bits are allowed to be set, set *ptr = val, try mark the buffer
> >   * dirty and return true. Otherwise false is returned.
> >   *
> >   * *ptr needs to be a pointer to memory within the buffer.
> >   *
> >   * This is a bit faster than BufferBeginSetHintBits() /
> >   * BufferFinishSetHintBits() when setting hints once in a buffer, but slower
> >   * than the former when setting hint bits multiple times in the same buffer.
> 
> +1. Instead of "try mark the buffer dirty", I'd say just "mark the buffer
> dirty". The only reason it might not to mark the buffer dirty is that it was
> already marked dirty, right? I wouldn't call that a failure.

It's not quite the only reason:

		/*
		 * If we need to protect hint bit updates from torn writes, WAL-log a
		 * full page image of the page. This full page image is only necessary
		 * if the hint bit update is the first change to the page since the
		 * last checkpoint.
		 *
		 * We don't check full_page_writes here because that logic is included
		 * when we call XLogInsert() since the value changes dynamically.
		 */
		if (XLogHintBitIsNeeded() && (lockstate & BM_PERMANENT))
		{
			/*
			 * If we must not write WAL, due to a relfilelocator-specific
			 * condition or being in recovery, don't dirty the page.  We can
			 * set the hint, just not dirty the page as a result so the hint
			 * is lost when we evict the page or shutdown.
			 *
			 * See src/backend/storage/page/README for longer discussion.
			 */
			if (RecoveryInProgress() ||
				RelFileLocatorSkippingWAL(BufTagGetRelFileLocator(&bufHdr->tag)))
				return;

			wal_log = true;
		}

Greetings,

Andres Freund

  103. Re: Buffer locking is special (hints, checksums, AIO writes)

    Antonin Houska <ah@cybertec.at> — 2026-02-10T07:46:27Z

    Andres Freund <andres@anarazel.de> wrote:

> > While troubleshooting REPACK issue [1], I realized that
> > HeapTupleSatisfiesMVCCBatch() can also be called during logical decoding - in
> > that case we need to use a historic MVCC snapshot.
> 
> Huh. Indeed. That's unintentional - the path should never have been reached,
> we are checking that an MVCC snapshot is used. Unfortunately, somebody
> (i.e. probably me) at some point defined the relevant macro as
> 
> /* This macro encodes the knowledge of which snapshots are MVCC-safe */
> #define IsMVCCSnapshot(snapshot)  \
> 	((snapshot)->snapshot_type == SNAPSHOT_MVCC || \
> 	 (snapshot)->snapshot_type == SNAPSHOT_HISTORIC_MVCC)
> 
> Which makes sense for some places, but not for plenty others.
> 
> The reason this didn't cause more widespread issues is that during logical
> decoding we mostly don't use sequential scans etc that are affected by the
> these paths.

> > My proposal to fix the problem is attached.
> 
> That's imo not at all the right fix - it'd make visibility during seqscans
> checking noticeably slower.

ok

> I think we ought to instead restrict the page-at-a-time scans to only happen
> with "real" mvcc snapshots. I.e. this:
> 
> 	/*
> 	 * Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
> 	 */
> 	if (!(snapshot && IsMVCCSnapshot(snapshot)))
> 		scan->rs_base.rs_flags &= ~SO_ALLOW_PAGEMODE;
>
> should trigger for historic snapshots as well.

I suppose you mean changing it to

	if (!(snapshot && IsMVCCSnapshot(snapshot) &&
		  !IsHistoricMVCCSnapshot(snapshot)))
		scan->rs_base.rs_flags &= ~SO_ALLOW_PAGEMODE;

> Does that fix the issue for you?

Yes, with this change, I don't hit the problem anymore.

> What's your reproducer?

Check out this branch

https://github.com/michail-nikolaev/postgres/tree/repack_concurrently_repro_22

and run t/008_repack_concurrently.pl in contrib/amcheck. The error we saw in
most cases was "ERROR: cache lookup failed for relation". I noticed that the
related pg_class entries had hint bits set incorrectly, so I added the
following to see when exactly it happens (actually I used lower elevel first,
to find out that the decoding worker is responsible for the problem):

diff --git a/src/backend/access/heap/heapam_visibility.c b/src/backend/access/heap/heapam_visibility.c
index 75ae268d753..ebf38460873 100644
--- a/src/backend/access/heap/heapam_visibility.c
+++ b/src/backend/access/heap/heapam_visibility.c
@@ -73,6 +73,7 @@
 #include "access/transam.h"
 #include "access/xact.h"
 #include "access/xlog.h"
+#include "commands/cluster.h"
 #include "storage/bufmgr.h"
 #include "storage/procarray.h"
 #include "utils/builtins.h"
@@ -938,6 +939,8 @@ HeapTupleSatisfiesMVCC(HeapTuple htup, Snapshot snapshot,
                                                HeapTupleHeaderGetRawXmin(tuple));
                else
                {
+                       if (am_decoding_for_repack())
+                               elog(PANIC, "HEAP_XMIN_INVALID set");
                        /* it must have aborted or crashed */
                        SetHintBits(tuple, buffer, HEAP_XMIN_INVALID,
                                                InvalidTransactionId);

The backtrace looked like:

    #3  0x0000000000c367ad errfinish (postgres + 0x8367ad)
    #4  0x0000000000517ee1 HeapTupleSatisfiesMVCC (postgres + 0x117ee1)
    #5  0x0000000000518e14 HeapTupleSatisfiesMVCCBatch (postgres + 0x118e14)
    #6  0x000000000050483e page_collect_tuples (postgres + 0x10483e)
    #7  0x0000000000504a8e heap_prepare_pagescan (postgres + 0x104a8e)
    #8  0x0000000000505344 heapgettup_pagemode (postgres + 0x105344)
    #9  0x0000000000505cff heap_getnextslot (postgres + 0x105cff)
    #10 0x000000000052ca08 table_scan_getnextslot (postgres + 0x12ca08)
    #11 0x000000000052d4ed systable_getnext (postgres + 0x12d4ed)
    #12 0x0000000000c2098e ScanPgRelation (postgres + 0x82098e)
    #13 0x0000000000c23aa2 RelationReloadIndexInfo (postgres + 0x823aa2)
    #14 0x0000000000c24376 RelationRebuildRelation (postgres + 0x824376)
    #15 0x0000000000c23784 RelationIdGetRelation (postgres + 0x823784)
    #16 0x00000000004b558f relation_open (postgres + 0xb558f)
    #17 0x000000000052e073 index_open (postgres + 0x12e073)
    #18 0x000000000052d0c5 systable_beginscan (postgres + 0x12d0c5)
    #19 0x0000000000c2097e ScanPgRelation (postgres + 0x82097e)
    #20 0x0000000000c23dde RelationReloadNailed (postgres + 0x823dde)
    #21 0x0000000000c24399 RelationRebuildRelation (postgres + 0x824399)
    #22 0x0000000000c23784 RelationIdGetRelation (postgres + 0x823784)
    #23 0x00000000004b558f relation_open (postgres + 0xb558f)
    #24 0x0000000000573ab1 table_open (postgres + 0x173ab1)
    #25 0x0000000000c20919 ScanPgRelation (postgres + 0x820919)
    #26 0x0000000000c21c98 RelationBuildDesc (postgres + 0x821c98)
    #27 0x0000000000c237d6 RelationIdGetRelation (postgres + 0x8237d6)
    #28 0x00000000009a028a ReorderBufferProcessTXN (postgres + 0x5a028a)
    #29 0x00000000009a10ed ReorderBufferReplay (postgres + 0x5a10ed)
    #30 0x00000000009a116b ReorderBufferCommit (postgres + 0x5a116b)
    #31 0x000000000098c7fe DecodeCommit (postgres + 0x58c7fe)
    #32 0x000000000098ba67 xact_decode (postgres + 0x58ba67)
    #33 0x000000000098b685 LogicalDecodingProcessRecord (postgres + 0x58b685)
    #34 0x00000000006a3e4b decode_concurrent_changes (postgres + 0x2a3e4b)
    #35 0x00000000006a5ea3 repack_worker_internal (postgres + 0x2a5ea3)
    #36 0x00000000006a5d5e RepackWorkerMain (postgres + 0x2a5d5e)
    #37 0x0000000000961b2b BackgroundWorkerMain (postgres + 0x561b2b)
    #38 0x0000000000964a61 postmaster_child_launch (postgres + 0x564a61)
    #39 0x000000000096b58d StartBackgroundWorker (postgres + 0x56b58d)
    #40 0x000000000096b80a maybe_start_bgworkers (postgres + 0x56b80a)
    #41 0x000000000096a5d9 LaunchMissingBackgroundProcesses (postgres + 0x56a5d9)
    #42 0x00000000009683fc ServerLoop (postgres + 0x5683fc)
    #43 0x0000000000967d37 PostmasterMain (postgres + 0x567d37)
    #44 0x0000000000813421 main (postgres + 0x413421)

Thanks.

-- 
Antonin Houska
Web: https://www.cybertec-postgresql.com

  104. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-02-10T16:49:36Z

    Hi,

On 2026-02-10 08:46:27 +0100, Antonin Houska wrote:
> Andres Freund <andres@anarazel.de> wrote:
> > I think we ought to instead restrict the page-at-a-time scans to only happen
> > with "real" mvcc snapshots. I.e. this:
> > 
> > 	/*
> > 	 * Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
> > 	 */
> > 	if (!(snapshot && IsMVCCSnapshot(snapshot)))
> > 		scan->rs_base.rs_flags &= ~SO_ALLOW_PAGEMODE;
> >
> > should trigger for historic snapshots as well.
> 
> I suppose you mean changing it to
> 
> 	if (!(snapshot && IsMVCCSnapshot(snapshot) &&
> 		  !IsHistoricMVCCSnapshot(snapshot)))
> 		scan->rs_base.rs_flags &= ~SO_ALLOW_PAGEMODE;

Yes.

For something committable, I think we should probably split IsMVCCSnapshot
into IsMVCCSnapshot(), just accepting SNAPSHOT_MVCC, and IsMVCCLikeSnapshot()
accepting both SNAPSHOT_MVCC and SNAPSHOT_HISTORIC_MVCC. And then go through
all the existing callers of IsMVCCSnapshot() - only about half should stay
as-is, I think.


> > Does that fix the issue for you?
> 
> Yes, with this change, I don't hit the problem anymore.

Great!

Greetings,

Andres Freund

  105. Re: Buffer locking is special (hints, checksums, AIO writes)

    Antonin Houska <ah@cybertec.at> — 2026-02-12T10:36:08Z

    Andres Freund <andres@anarazel.de> wrote:

> For something committable, I think we should probably split IsMVCCSnapshot
> into IsMVCCSnapshot(), just accepting SNAPSHOT_MVCC, and IsMVCCLikeSnapshot()
> accepting both SNAPSHOT_MVCC and SNAPSHOT_HISTORIC_MVCC. And then go through
> all the existing callers of IsMVCCSnapshot() - only about half should stay
> as-is, I think.

The attached patch tries to do that.

-- 
Antonin Houska
Web: https://www.cybertec-postgresql.com

  106. Re: Buffer locking is special (hints, checksums, AIO writes)

    Noah Misch <noah@leadboat.com> — 2026-02-15T19:52:39Z

    On Sat, Feb 07, 2026 at 12:44:25PM +0200, Heikki Linnakangas wrote:
> On 03/02/2026 00:33, Andres Freund wrote:
> > - The way MarkBufferDirtyHint() operates was copied into
> >    heap_inplace_update_and_unlock(). Now that MarkBufferDirtyHint() won't work
> >    that way anymore, it seems better to go with the alternative approach the
> >    comments already outlined, namely to only delay updating of the buffer
> >    contents.
> > 
> >    I've done this in a prequisite commit, as it doesn't actually depend on any
> >    of the other changes.  Noah, any chance you could take a look at this?

v12-0001-heapam-Don-t-mimic-MarkBufferDirtyHint-in-inplac.patch looks good.

> Patch 0001 Looks correct to me. However:
> 
> > 	 * ["D" is a VACUUM (ONLY_DATABASE_STATS)]
> > 	 * ["R" is a VACUUM tbl]
> > 	 * D: vac_update_datfrozenxid() -> systable_beginscan(pg_class)
> > 	 * D: systable_getnext() returns pg_class tuple of tbl
> > 	 * R: memcpy() into pg_class tuple of tbl
> > 	 * D: raise pg_database.datfrozenxid, XLogInsert(), finish
> > 	 * [crash]
> > 	 * [recovery restores datfrozenxid w/o relfrozenxid]
> > 	 *
> > 	 * As we hold an exclusive lock - preventing the buffer from being written
> > 	 * out once dirty - we can work around this as follows: MarkBufferDirty(),
> > 	 * XLogInsert(), memcpy().
> 
> That last reference to 'memcpy' is a little orphaned now. The comment used
> to talk about the stack copy of the page, but now there's no mention of that
> except for this reference to memcpy(). To make things worse, the steps have
> "memcpy() into pg_class tuple of tbl", so one could think that the "memcpy"
> refers to that.

"memcpy" does refer to "memcpy() into pg_class tuple of tbl", so I don't see
that as orphaned.  Nonetheless:

> How about this:
> 
> 	 * We avoid that by using a temporary copy of the buffer to hide our
> 	 * change from other backends until it's been WAL-logged. We apply our
> 	 * change to the temporary copy and WAL-log it before modifying the real
> 	 * page. That way any action a reader of the in-place-updated value takes
> 	 * will be WAL logged after this change.

Either v12 or v12 w/ this edit is fine with me.  I find this proposed text
redundant with nearby comment "register block matching what buffer will look
like after changes", so I mildly prefer v12.

  107. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-03-11T22:40:41Z

    Hi,

On 2026-02-15 11:52:39 -0800, Noah Misch wrote:
> On Sat, Feb 07, 2026 at 12:44:25PM +0200, Heikki Linnakangas wrote:
> > On 03/02/2026 00:33, Andres Freund wrote:
> > > - The way MarkBufferDirtyHint() operates was copied into
> > >    heap_inplace_update_and_unlock(). Now that MarkBufferDirtyHint() won't work
> > >    that way anymore, it seems better to go with the alternative approach the
> > >    comments already outlined, namely to only delay updating of the buffer
> > >    contents.
> > > 
> > >    I've done this in a prequisite commit, as it doesn't actually depend on any
> > >    of the other changes.  Noah, any chance you could take a look at this?
> 
> v12-0001-heapam-Don-t-mimic-MarkBufferDirtyHint-in-inplac.patch looks good.

> > How about this:
> > 
> > 	 * We avoid that by using a temporary copy of the buffer to hide our
> > 	 * change from other backends until it's been WAL-logged. We apply our
> > 	 * change to the temporary copy and WAL-log it before modifying the real
> > 	 * page. That way any action a reader of the in-place-updated value takes
> > 	 * will be WAL logged after this change.
> 
> Either v12 or v12 w/ this edit is fine with me.  I find this proposed text
> redundant with nearby comment "register block matching what buffer will look
> like after changes", so I mildly prefer v12.

Thanks for the review!


I pushed this and many of the later patches in the series.  Here are updated
versions of the remaining changes.  The last two previously were one commit
with "WIP" in the title. The first one has, I think, not had a lot of review -
but it's also not a complicated change.


I see decent performance improvements with a fully s_b resident pipelined
pgbench -S with 0002+0003, ~7-8% on an older small two socket machine.

The improvement is just from reducing the number of atomic operations on
contended cachelines (i.e. inner btree pages).

Without pipelining the difference is smaller (1-2%), because of the context
switches are the bigger bottleneck.


More extreme worloads involving an index nested loop join benefit
more. E.g. the setup and query from
https://anarazel.de/talks/2024-05-29-pgconf-dev-c2c/postgres-perf-c2c.pdf
slide 23, show a 25% improvement on the same 2 socket machine.


We could probably do something similar for the also very common combination of
PinBuffer() + LockBuffer(), but I think it'd be a fair bit more complicated,
and would require new APIs, rather than just using existing APIs more widely.

Greetings,

Andres Freund

  108. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-03-11T23:09:26Z

    Hi,

On 2026-02-12 11:36:08 +0100, Antonin Houska wrote:
> Andres Freund <andres@anarazel.de> wrote:
>
> > For something committable, I think we should probably split IsMVCCSnapshot
> > into IsMVCCSnapshot(), just accepting SNAPSHOT_MVCC, and IsMVCCLikeSnapshot()
> > accepting both SNAPSHOT_MVCC and SNAPSHOT_HISTORIC_MVCC. And then go through
> > all the existing callers of IsMVCCSnapshot() - only about half should stay
> > as-is, I think.
>
> The attached patch tries to do that.

Thanks!


> From dcdbaf3095e632a1f7f65f3abc43eccff0249d4c Mon Sep 17 00:00:00 2001
> From: Antonin Houska <ah@cybertec.at>
> Date: Thu, 12 Feb 2026 11:14:00 +0100
> Subject: [PATCH] Refine checking of snapshot type.
>
> It appears to be confusing if IsMVCCSnapshot() evaluates to true for both
> "regular" and "historic" MVCC snapshot. This patch restricts the meaning of
> the macro to the "regular" MVCC snapshot, and introduces a new macro
> IsMVCCLikeSnapshot() to recognize both types.
>
> IsMVCCLikeSnapshot() is only used in functions that can (supposedly) be called
> during logical decoding.

I think I agree with where you selected IsMVCCSnapshot() and where you
selected IsMVCCLikeSnapshot().


> diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h
> index b8c01a291a1..dd5aaae6953 100644
> --- a/src/include/utils/snapmgr.h
> +++ b/src/include/utils/snapmgr.h
> @@ -53,12 +53,14 @@ extern PGDLLIMPORT SnapshotData SnapshotToastData;
>
>  /* This macro encodes the knowledge of which snapshots are MVCC-safe */
>  #define IsMVCCSnapshot(snapshot)  \
> -	((snapshot)->snapshot_type == SNAPSHOT_MVCC || \
> -	 (snapshot)->snapshot_type == SNAPSHOT_HISTORIC_MVCC)
> +	((snapshot)->snapshot_type == SNAPSHOT_MVCC)
>
>  #define IsHistoricMVCCSnapshot(snapshot)  \
>  	((snapshot)->snapshot_type == SNAPSHOT_HISTORIC_MVCC)
>
> +#define IsMVCCLikeSnapshot(snapshot)  \
> +	(IsMVCCSnapshot(snapshot) || IsHistoricMVCCSnapshot(snapshot))
> +
>  extern Snapshot GetTransactionSnapshot(void);
>  extern Snapshot GetLatestSnapshot(void);
>  extern void SnapshotSetCommandId(CommandId curcid);

Probably need to update the comments a bit.  What about something like


/*
 * Is the snapshot implemented as an MVCC snapshot (i.e. it uses
 * SNAPSHOT_MVCC).  If so, there will be at most be one visible row in a chain
 * of updated tuples, and each visible tuple will be seen exactly once.
 */
#define IsMVCCSnapshot(snapshot)  \
...

/*
 * Is the snapshot either an MVCC snapshot or has equivalent visibility
 * semantics (see IsMVCCSnapshot()).
 */
#define IsMVCCLikeSnapshot(snapshot)  \


Greetings,

Andres Freund

  109. Re: Buffer locking is special (hints, checksums, AIO writes)

    Alexander Lakhin <exclusion@gmail.com> — 2026-03-13T08:00:00Z

    Hello Andres,

12.03.2026 00:40, Andres Freund wrote:
> I pushed this and many of the later patches in the series.  Here are updated
> versions of the remaining changes.  The last two previously were one commit
> with "WIP" in the title. The first one has, I think, not had a lot of review -
> but it's also not a complicated change.

I've discovered that starting from 82467f627, the following query:
SET cpu_operator_cost = 1000;
CREATE TABLE t (i INT);
INSERT INTO T SELECT 1 FROM generate_series(1, 1000) a;
CREATE INDEX hi on t USING HASH (i);
DELETE FROM t WHERE i = 1;
DELETE FROM t WHERE i = 1;

triggers
TRAP: failed Assert("BufferIsValid(buffer)"), File: "bufmgr.c", Line: 497, PID: 3942058

#4  0x000079a60ae288ff in __GI_abort () at ./stdlib/abort.c:79
#5  0x00005a68d9343eef in ExceptionalCondition (conditionName=conditionName@entry=0x5a68d93ac27d "BufferIsValid(buffer)",
     fileName=fileName@entry=0x5a68d93c99ef "bufmgr.c", lineNumber=lineNumber@entry=497) at assert.c:65
#6  0x00005a68d91a18eb in GetPrivateRefCountEntry (do_move=true, buffer=<optimized out>) at bufmgr.c:497
#7  SharedBufferBeginSetHintBits (lockstate=<synthetic pointer>, buf_hdr=0x79e5febbbc40, buffer=<optimized out>)
     at bufmgr.c:6830
#8  BufferBeginSetHintBits (buffer=<optimized out>) at bufmgr.c:6931
#9  0x00005a68d8e3c862 in _hash_kill_items (scan=<optimized out>) at hashutil.c:603
#10 0x00005a68d8e3b7c3 in _hash_next (scan=0x5a68e735f938, dir=<optimized out>) at hashsearch.c:69
#11 0x00005a68d8e616ce in index_getnext_tid (scan=scan@entry=0x5a68e735f938, direction=direction@entry=ForwardScanDirection)
     at indexam.c:647
...
#25 0x00005a68d91eb4ad in exec_simple_query (query_string=0x5a68e7270120 "DELETE FROM t WHERE i = 1;") at postgres.c:1277
...

Could you please look at this?

Best regards,
Alexander

  110. Re: Buffer locking is special (hints, checksums, AIO writes)

    Antonin Houska <ah@cybertec.at> — 2026-03-13T15:01:40Z

    Andres Freund <andres@anarazel.de> wrote:

> Probably need to update the comments a bit.  What about something like
> 
> 
> /*
>  * Is the snapshot implemented as an MVCC snapshot (i.e. it uses
>  * SNAPSHOT_MVCC).  If so, there will be at most be one visible row in a chain
>  * of updated tuples, and each visible tuple will be seen exactly once.
>  */
> #define IsMVCCSnapshot(snapshot)  \

The ", and each visible tuple ..." part seemed to me redundant, so I omitted
it. If you think I'm wrong, please add it yourself when committing the patch.

I also added a comment to the IsHistoricMVCCSnapshot(), trying to explain what
"historic" means.

-- 
Antonin Houska
Web: https://www.cybertec-postgresql.com

  111. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-03-13T15:55:53Z

    Hi,

On 2026-03-13 10:00:00 +0200, Alexander Lakhin wrote:
> Hello Andres,
> 
> 12.03.2026 00:40, Andres Freund wrote:
> > I pushed this and many of the later patches in the series.  Here are updated
> > versions of the remaining changes.  The last two previously were one commit
> > with "WIP" in the title. The first one has, I think, not had a lot of review -
> > but it's also not a complicated change.
> 
> I've discovered that starting from 82467f627, the following query:
> SET cpu_operator_cost = 1000;
> CREATE TABLE t (i INT);
> INSERT INTO T SELECT 1 FROM generate_series(1, 1000) a;
> CREATE INDEX hi on t USING HASH (i);
> DELETE FROM t WHERE i = 1;
> DELETE FROM t WHERE i = 1;
> 
> triggers
> TRAP: failed Assert("BufferIsValid(buffer)"), File: "bufmgr.c", Line: 497, PID: 3942058
> 
> #4  0x000079a60ae288ff in __GI_abort () at ./stdlib/abort.c:79
> #5  0x00005a68d9343eef in ExceptionalCondition (conditionName=conditionName@entry=0x5a68d93ac27d "BufferIsValid(buffer)",
>     fileName=fileName@entry=0x5a68d93c99ef "bufmgr.c", lineNumber=lineNumber@entry=497) at assert.c:65
> #6  0x00005a68d91a18eb in GetPrivateRefCountEntry (do_move=true, buffer=<optimized out>) at bufmgr.c:497
> #7  SharedBufferBeginSetHintBits (lockstate=<synthetic pointer>, buf_hdr=0x79e5febbbc40, buffer=<optimized out>)
>     at bufmgr.c:6830
> #8  BufferBeginSetHintBits (buffer=<optimized out>) at bufmgr.c:6931
> #9  0x00005a68d8e3c862 in _hash_kill_items (scan=<optimized out>) at hashutil.c:603
> #10 0x00005a68d8e3b7c3 in _hash_next (scan=0x5a68e735f938, dir=<optimized out>) at hashsearch.c:69
> #11 0x00005a68d8e616ce in index_getnext_tid (scan=scan@entry=0x5a68e735f938, direction=direction@entry=ForwardScanDirection)
>     at indexam.c:647
> ...
> #25 0x00005a68d91eb4ad in exec_simple_query (query_string=0x5a68e7270120 "DELETE FROM t WHERE i = 1;") at postgres.c:1277
> ...
> 
> Could you please look at this?

Yea, it's a stupid small mistake. Alexander Kuzmenkov reported it late
afternoon yesterday, privately as I just noticed, and I was too tired to make
sure an added test wouldn't have stability issues.

Will fix in the next few hours.

Greetings,

Andres Freund

  112. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-03-13T17:55:55Z

    Hi,

On 2026-03-13 16:01:40 +0100, Antonin Houska wrote:
> Andres Freund <andres@anarazel.de> wrote:
> 
> > Probably need to update the comments a bit.  What about something like
> > 
> > 
> > /*
> >  * Is the snapshot implemented as an MVCC snapshot (i.e. it uses
> >  * SNAPSHOT_MVCC).  If so, there will be at most be one visible row in a chain
> >  * of updated tuples, and each visible tuple will be seen exactly once.
> >  */
> > #define IsMVCCSnapshot(snapshot)  \
> 
> The ", and each visible tuple ..." part seemed to me redundant, so I omitted
> it. If you think I'm wrong, please add it yourself when committing the patch.

It's relevant in that many non-mvcc scan types do *not* guarantee that (i.e. a
tuple may never be seen, e.g. because the new version of the tuple is placed
before the current scan position of a scan and the old version of the tuple is
not considered visible anymore).


> I also added a comment to the IsHistoricMVCCSnapshot(), trying to explain what
> "historic" means.

Good idea.


Pushed with slightly revised comments and a different commit message (I
thought it was important to explain that this fixes breakage during logical
decoding, even if currently hard to reach).


Thanks for the report and patch!

- Andres

  113. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-03-17T20:50:09Z

    Hi,

On 2026-03-13 11:55:53 -0400, Andres Freund wrote:
> On 2026-03-13 10:00:00 +0200, Alexander Lakhin wrote:
> > Hello Andres,
> > 
> > 12.03.2026 00:40, Andres Freund wrote:
> > > I pushed this and many of the later patches in the series.  Here are updated
> > > versions of the remaining changes.  The last two previously were one commit
> > > with "WIP" in the title. The first one has, I think, not had a lot of review -
> > > but it's also not a complicated change.
> > 
> > I've discovered that starting from 82467f627, the following query:
> > SET cpu_operator_cost = 1000;
> > CREATE TABLE t (i INT);
> > INSERT INTO T SELECT 1 FROM generate_series(1, 1000) a;
> > CREATE INDEX hi on t USING HASH (i);
> > DELETE FROM t WHERE i = 1;
> > DELETE FROM t WHERE i = 1;
> > 
> > triggers
> > TRAP: failed Assert("BufferIsValid(buffer)"), File: "bufmgr.c", Line: 497, PID: 3942058
> > 
> > #4  0x000079a60ae288ff in __GI_abort () at ./stdlib/abort.c:79
> > #5  0x00005a68d9343eef in ExceptionalCondition (conditionName=conditionName@entry=0x5a68d93ac27d "BufferIsValid(buffer)",
> >     fileName=fileName@entry=0x5a68d93c99ef "bufmgr.c", lineNumber=lineNumber@entry=497) at assert.c:65
> > #6  0x00005a68d91a18eb in GetPrivateRefCountEntry (do_move=true, buffer=<optimized out>) at bufmgr.c:497
> > #7  SharedBufferBeginSetHintBits (lockstate=<synthetic pointer>, buf_hdr=0x79e5febbbc40, buffer=<optimized out>)
> >     at bufmgr.c:6830
> > #8  BufferBeginSetHintBits (buffer=<optimized out>) at bufmgr.c:6931
> > #9  0x00005a68d8e3c862 in _hash_kill_items (scan=<optimized out>) at hashutil.c:603
> > #10 0x00005a68d8e3b7c3 in _hash_next (scan=0x5a68e735f938, dir=<optimized out>) at hashsearch.c:69
> > #11 0x00005a68d8e616ce in index_getnext_tid (scan=scan@entry=0x5a68e735f938, direction=direction@entry=ForwardScanDirection)
> >     at indexam.c:647
> > ...
> > #25 0x00005a68d91eb4ad in exec_simple_query (query_string=0x5a68e7270120 "DELETE FROM t WHERE i = 1;") at postgres.c:1277
> > ...
> > 
> > Could you please look at this?
> 
> Yea, it's a stupid small mistake. Alexander Kuzmenkov reported it late
> afternoon yesterday, privately as I just noticed, and I was too tired to make
> sure an added test wouldn't have stability issues.
> 
> Will fix in the next few hours.

Took longer, sorry.  But it's pushed now.

Greetings,

Andres Freund

  114. Re: Buffer locking is special (hints, checksums, AIO writes)

    Melanie Plageman <melanieplageman@gmail.com> — 2026-03-25T21:34:33Z

    On Wed, Mar 11, 2026 at 6:40 PM Andres Freund <andres@anarazel.de> wrote:
>
> I pushed this and many of the later patches in the series.  Here are updated
> versions of the remaining changes.  The last two previously were one commit
> with "WIP" in the title. The first one has, I think, not had a lot of review -
> but it's also not a complicated change.

0001 looks good except for the comment above PageSetChecksum() that
says it is only for shared buffers and a stray reference to the
no-longer-present bufToWrite variable in a comment around line 4490 in
bufmgr.c

0002
diff --git a/src/backend/access/nbtree/nbtpage.c
b/src/backend/access/nbtree/nbtpage.c
index cc9c45dc40c..ad700e590e8 100644
--- a/src/backend/access/nbtree/nbtpage.c
+++ b/src/backend/access/nbtree/nbtpage.c
@@ -1011,24 +1011,48 @@ _bt_relandgetbuf(Relation rel, Buffer obuf,
BlockNumber blkno, int access)
    Assert(BlockNumberIsValid(blkno));
    if (BufferIsValid(obuf))
-       _bt_unlockbuf(rel, obuf);
-   buf = ReleaseAndReadBuffer(obuf, rel, blkno);
-   _bt_lockbuf(rel, buf, access);
+   {
+       if (BufferGetBlockNumber(obuf) == blkno)
+       {
+           /* trade in old lock mode for new lock */
+           _bt_unlockbuf(rel, obuf);
+           buf = obuf;
+       }
+       else
+       {
+           /* release lock and pin at once, that's a bit more efficient */
+           _bt_relbuf(rel, obuf);
+           buf = ReadBuffer(rel, blkno);
+       }
+   }
+   else
+       buf = ReadBuffer(rel, blkno);

Not related to this patch, but why do we unlock and relock it when
obuf has the block we need? Couldn't we pass lock mode and then just
do nothing if it is the right lockmode?

Setting that aside, I presume we don't need to check the fork and
relfilelocator (as ReleaseAndReadBuffer() did) because this code knows
it will be the same?

Anyway, LGTM.

0003
AFAICT, this does what you claim. I don't really know what else to
look when reviewing it, if I'm being honest. As such, I diligently fed
it through AI which suggested you may have lost a
        VALGRIND_MAKE_MEM_NOACCESS(BufHdrGetBlock(buf), BLCKSZ);
which sounds right to me and like something you should fix.

Also, I'd say this comment
+   /*
+    * Now okay to allow cancel/die interrupts again, were held when the lock
+    * was acquired.
+    */

needs a "which" after the comma to read smoothly.

- Melanie

  115. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-03-25T22:35:55Z

    Hi,

On 2026-03-25 17:34:33 -0400, Melanie Plageman wrote:
> On Wed, Mar 11, 2026 at 6:40 PM Andres Freund <andres@anarazel.de> wrote:
> >
> > I pushed this and many of the later patches in the series.  Here are updated
> > versions of the remaining changes.  The last two previously were one commit
> > with "WIP" in the title. The first one has, I think, not had a lot of review -
> > but it's also not a complicated change.
> 
> 0001 looks good except for the comment above PageSetChecksum() that
> says it is only for shared buffers and a stray reference to the
> no-longer-present bufToWrite variable in a comment around line 4490 in
> bufmgr.c

Thanks for catching these.

Updated the PageSetChecksum() comment to

 * Set checksum on a page.
 *
 * If the page is in shared buffers, it needs to be locked in at least
 * share-exclusive mode.
...


> 0002
> diff --git a/src/backend/access/nbtree/nbtpage.c
> b/src/backend/access/nbtree/nbtpage.c
> index cc9c45dc40c..ad700e590e8 100644
> --- a/src/backend/access/nbtree/nbtpage.c
> +++ b/src/backend/access/nbtree/nbtpage.c
> @@ -1011,24 +1011,48 @@ _bt_relandgetbuf(Relation rel, Buffer obuf,
> BlockNumber blkno, int access)
>     Assert(BlockNumberIsValid(blkno));
>     if (BufferIsValid(obuf))
> -       _bt_unlockbuf(rel, obuf);
> -   buf = ReleaseAndReadBuffer(obuf, rel, blkno);
> -   _bt_lockbuf(rel, buf, access);
> +   {
> +       if (BufferGetBlockNumber(obuf) == blkno)
> +       {
> +           /* trade in old lock mode for new lock */
> +           _bt_unlockbuf(rel, obuf);
> +           buf = obuf;
> +       }
> +       else
> +       {
> +           /* release lock and pin at once, that's a bit more efficient */
> +           _bt_relbuf(rel, obuf);
> +           buf = ReadBuffer(rel, blkno);
> +       }
> +   }
> +   else
> +       buf = ReadBuffer(rel, blkno);
> 
> Not related to this patch, but why do we unlock and relock it when
> obuf has the block we need? Couldn't we pass lock mode and then just
> do nothing if it is the right lockmode?

I think it's very unlikely that it's called at any frequency with the same
buffer and lockmode. What would be the point of calling _bt_relandgetbuf() if
that's the case.


> Setting that aside, I presume we don't need to check the fork and
> relfilelocator (as ReleaseAndReadBuffer() did) because this code knows
> it will be the same?

Yea, it's a single index, so there can't be a different relfilenode.


> 0003
> AFAICT, this does what you claim. I don't really know what else to
> look when reviewing it, if I'm being honest. As such, I diligently fed
> it through AI which suggested you may have lost a
>         VALGRIND_MAKE_MEM_NOACCESS(BufHdrGetBlock(buf), BLCKSZ);
> which sounds right to me and like something you should fix.

Good catch Melai.


> Also, I'd say this comment
> +   /*
> +    * Now okay to allow cancel/die interrupts again, were held when the lock
> +    * was acquired.
> +    */
> 
> needs a "which" after the comma to read smoothly.

Fixed.


Running it through valgrind and then will work on reading through one more
time and pushing them.

Greetings,

Andres Freund

  116. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-03-27T20:00:47Z

    Hi,

On 2026-03-25 18:35:55 -0400, Andres Freund wrote:
> Running it through valgrind and then will work on reading through one more
> time and pushing them.

And done.

Phew, this project took way longer than I'd though it'd take.

Greetings,

Andres

  117. Re: Buffer locking is special (hints, checksums, AIO writes)

    Yura Sokolov <y.sokolov@postgrespro.ru> — 2026-03-31T16:02:33Z

    27.03.2026 23:00, Andres Freund wrote:
> Hi,
> 
> On 2026-03-25 18:35:55 -0400, Andres Freund wrote:
>> Running it through valgrind and then will work on reading through one more
>> time and pushing them.
> 
> And done.
> 
> Phew, this project took way longer than I'd though it'd take.

In addition to bug with BM_IO_ERROR [1] , I found race condition in
PinBuffer in this lines of code:

	if (unlikely(skip_if_not_valid && !(old_buf_state & BM_VALID)))
		return false;

	/*
	 * We're not allowed to increase the refcount while the buffer
	 * header spinlock is held. Wait for the lock to be released.
	 */
	if (old_buf_state & BM_LOCKED)
		old_buf_state = WaitBufHdrUnlocked(buf);

While we waited for buffer header for being unlocked, it may become
invalid, isn't it?
Therefore, check related to skip_if_not_valid have to happen after waiting.

....

Another question: previously we had to wait for buffer for being unlocked
because UnlockBufHdr wrote to buf->state unconditionally, therefore our pin
increment could be lost.
Now UnlockBufHdr and UnlockBufHdrExt does proper atomic operations and
preserves concurrent changes. Are we still need to wait?

Most of time PinBuffer is called protected by BufTable's partition LWLock,
therefore buffer may not be changed in dramatic way.

But call in ReadRecentBuffer is the exception. It is not protected by
partition lock and have to make additional checks. That is why you
introduced skip_if_not_valid.

Does optimization of ReadRecentBuffer pays for WaitBufHdrUnlocked?

[1]
https://www.postgresql.org/message-id/57a8ea70-32a8-4596-bb68-5d2990b83380%40postgrespro.ru

-- 
regards
Yura Sokolov aka funny-falcon

  118. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-03-31T22:05:46Z

    Hi,

On 2026-03-31 19:02:33 +0300, Yura Sokolov wrote:
> 27.03.2026 23:00, Andres Freund wrote:
> > On 2026-03-25 18:35:55 -0400, Andres Freund wrote:
> >> Running it through valgrind and then will work on reading through one more
> >> time and pushing them.
> > 
> > And done.
> > 
> > Phew, this project took way longer than I'd though it'd take.
> 
> In addition to bug with BM_IO_ERROR [1] , I found race condition in
> PinBuffer in this lines of code:
> 
> 	if (unlikely(skip_if_not_valid && !(old_buf_state & BM_VALID)))
> 		return false;
> 
> 	/*
> 	 * We're not allowed to increase the refcount while the buffer
> 	 * header spinlock is held. Wait for the lock to be released.
> 	 */
> 	if (old_buf_state & BM_LOCKED)
> 		old_buf_state = WaitBufHdrUnlocked(buf);
> 
> While we waited for buffer header for being unlocked, it may become
> invalid, isn't it?
> Therefore, check related to skip_if_not_valid have to happen after waiting.

Yea, that does seem wrong.  Not sure how it ended up that way.

I think it may be better to add a continue after the WaitBufHdrUnlocked(), so
that we restart the loop, rather than moving the skip_if_not_valid check.


> ....
> 
> Another question: previously we had to wait for buffer for being unlocked
> because UnlockBufHdr wrote to buf->state unconditionally, therefore our pin
> increment could be lost.
> Now UnlockBufHdr and UnlockBufHdrExt does proper atomic operations and
> preserves concurrent changes. Are we still need to wait?

Yes.


> Most of time PinBuffer is called protected by BufTable's partition LWLock,
> therefore buffer may not be changed in dramatic way.

I don't think the partition locks are sufficient protection for everything. We
have a few places in the code that want to be able to modify the buffer state
depending on whether the buffer is already pinned, and I don't think all of
them currently hold the relevant buffer mapping partition's lock.  If pinning
were not to wait for an existing header lock, such checks would not easily be
doable.

Perhaps we could fix all the relevant places by acquiring the partition lock
in a few more places. But I think that'd be going in exactly the opposite
direction we should to. The partition locks are quite contended locks and we
should work on getting rid of them eventually. Building them into the
protection model seems quite unwise.


I think many of the places that currently do rely on the buffer header
spinlock can be converted to CAS loops.

I'm not really sure how much that's worth though - the WaitBufHdrLocked() in
PinBuffer() is pretty hard to hit in realistic workloads. What would be really
nice, is to be able to replace the CAS() with an atomic add (since those are
considerably faster), but that's not really possible regardless of the need
for WaitBufHdrLocked(), because we can't just add BUF_USAGECOUNT_ONE, as that
would allow increasing the usage count too far.

I would like to eventually narrow the definition of the buffer header spinlock
to just be about the "identity" of the buffer, which then would only be needed
by things like DropRelationBuffers() and when changing the buffer's identity.


> But call in ReadRecentBuffer is the exception. It is not protected by
> partition lock and have to make additional checks. That is why you
> introduced skip_if_not_valid.
> 
> Does optimization of ReadRecentBuffer pays for WaitBufHdrUnlocked?

As mentioned above, I don't think it's just ReadRecentBuffer that relies on
the buffer header spinlock preventing new pins.

Greetings,

Andres Freund

  119. Re: Buffer locking is special (hints, checksums, AIO writes)

    Andres Freund <andres@anarazel.de> — 2026-04-01T00:29:02Z

    Hi,

On 2026-03-31 18:05:46 -0400, Andres Freund wrote:
> On 2026-03-31 19:02:33 +0300, Yura Sokolov wrote:
> > 27.03.2026 23:00, Andres Freund wrote:
> > > On 2026-03-25 18:35:55 -0400, Andres Freund wrote:
> > >> Running it through valgrind and then will work on reading through one more
> > >> time and pushing them.
> > >
> > > And done.
> > >
> > > Phew, this project took way longer than I'd though it'd take.
> >
> > In addition to bug with BM_IO_ERROR [1] , I found race condition in
> > PinBuffer in this lines of code:
> >
> > 	if (unlikely(skip_if_not_valid && !(old_buf_state & BM_VALID)))
> > 		return false;
> >
> > 	/*
> > 	 * We're not allowed to increase the refcount while the buffer
> > 	 * header spinlock is held. Wait for the lock to be released.
> > 	 */
> > 	if (old_buf_state & BM_LOCKED)
> > 		old_buf_state = WaitBufHdrUnlocked(buf);
> >
> > While we waited for buffer header for being unlocked, it may become
> > invalid, isn't it?
> > Therefore, check related to skip_if_not_valid have to happen after waiting.
>
> Yea, that does seem wrong.  Not sure how it ended up that way.
>
> I think it may be better to add a continue after the WaitBufHdrUnlocked(), so
> that we restart the loop, rather than moving the skip_if_not_valid check.

Done that way. Thanks for finding & reporting this, well spotted!

Greetings,

Andres

  120. Re: Buffer locking is special (hints, checksums, AIO writes)

    cca5507 <cca5507@qq.com> — 2026-04-03T10:06:46Z

    Hi,

I find some outdated comments in src/backend/storage/buffer/README:

```
Note that a buffer header's spinlock does not control access to the data
held within the buffer.  Each buffer header also contains an LWLock, the
"buffer content lock", that *does* represent the right to access the data
in the buffer.  It is used per the rules above.
```

"Each buffer header also contains an LWLock" is outdated.

```
The background writer takes shared content lock on a buffer while writing it
out (and anyone else who flushes buffer contents to disk must do so too).
This ensures that the page image transferred to disk is reasonably consistent.
We might miss a hint-bit update or two but that isn't a problem, for the same
reasons mentioned under buffer access rules.
```

"The background writer takes shared content lock ...", should be "share-exclusive content lock".

"We might miss a hint-bit update or two ...", maybe already fixed by share-exclusive content lock?

--
Regards,
ChangAo Chen