Re: bgwriter doesn't flush WAL stats

On Wed, 21 Jun 2023 at 13:04, Nazir Bilal Yavuz <byavuz81@gmail.com> wrote:
>
> Hi,
>
> I was trying to add WAL stats to pg_stat_io. While doing that I was comparing pg_stat_wal and pg_stat_io's WAL stats and there was some inequality between the total number of WALs. I found that the difference comes from bgwriter's WALs. bgwriter generates WAL but it doesn't flush them because the pgstat_report_wal() function isn't called in bgwriter. I attached a small patch for calling the pgstat_report_wal() function in bgwriter.
>
> bgwriter generates WAL by calling functions in this order:
> bgwriter.c -> BackgroundWriterMain() -> BgBufferSync() -> SyncOneBuffer() -> FlushBuffer() -> XLogFlush() -> XLogWrite()

I was quite confused here, as XLogWrite() does not generate any WAL;
it only writes existing WAL from buffers to disk.
In a running PostgreSQL instance, WAL is only generated through
XLogInsert(xloginsert.c) and serialized / written to buffers in its
call to XLogInsertRecord(xlog.c); XLogFlush and XLogWrite are only
responsible for writing those buffers to disk.

The only path that I see in XLogWrite() that could potentially put
anything into WAL is through RequestCheckpoint(), but that only writes
out a checkpoint when it is not in a postmaster environment - in all
other cases it will wake up the checkpointer and wait for that
checkpoint to finish.

I also got confused with your included views; they're not included in
the patch and the current master branch doesn't emit object=wal, so I
can't really check that the patch works as intended.

But on the topic of reporting the WAL stats in bgwriter; that seems
like a good idea to fix, yes.

+1

Kind regards,

Matthias van de Meent
Neon, Inc.