Horton

Re: Use WALReadFromBuffers in more places

Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>

View thread
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
To: Jeff Davis <pgsql@j-davis.com>
Cc: Jingtang Zhang <mrdrivingduck@gmail.com>, pgsql-hackers@lists.postgresql.org, Nitin Jadhav <nitinjadhavpostgres@gmail.com>
Date: 2025-09-24T14:26:18Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Read WAL directly from WAL buffers.

Hi,

On Mon, Sep 22, 2025 at 8:26 AM Jeff Davis <pgsql@j-davis.com> wrote:
>
> On Sat, 2025-09-13 at 22:04 -0700, Bharath Rupireddy wrote:
> > Thanks for looking at this. Yes, the WAL writers can zero out flushed
> > buffers before WALReadFromBuffers gets to them. However,
> > WALReadFromBuffers was intentionally designed as an opportunistic
> > optimization - it's a "try this first, quickly" approach before
> > falling back to reading from WAL files.
>
> IIRC, one motivation (perhaps the primary motivation?) was to make it
> possible to read buffers before they are flushed. It was always
> possible to read already-flushed buffers.
>
> The benefit of reading unflushed buffers is that we can replicate the
> WAL sooner (though it can't be replayed until the primary flushes it).
> Is that right?

Right. Reading unflushed WAL buffers for replication was one of the
motivations. But, in general, WALReadFromBuffers has more benefits
since it lets WAL buffers act as a cache for reads, avoiding the need
to re-read WAL from disk for (both physical and logical) replication.
For example, it makes the use of direct I/O for WAL more realistic and
can provide significant performance benefits [1].

[1] https://www.postgresql.org/message-id/20230114203403.4zpg72fw2qb34awf%40awork3.anarazel.de

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com