Horton

Re: Show WAL write and fsync stats in pg_stat_io

Nazir Bilal Yavuz <byavuz81@gmail.com>

View thread
From: Nazir Bilal Yavuz <byavuz81@gmail.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: Melanie Plageman <melanieplageman@gmail.com>, Andres Freund <andres@anarazel.de>, pgsql-hackers <pgsql-hackers@postgresql.org>, "bharath.rupireddyforpostgres@gmail.com" <bharath.rupireddyforpostgres@gmail.com>, Thomas Munro <thomas.munro@gmail.com>, Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: 2025-01-24T08:31:02Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Fix copy-paste error related to the autovacuum launcher in pgstat_io.c

  2. Move SQL tests of pg_stat_io for WAL data to recovery test 029_stats_restart

  3. Add data for WAL in pg_stat_io and backend statistics

  4. Improve comment on top of pgstat_count_io_op_time()

  5. Refactor pgstat_prepare_io_time() with an input argument instead of a GUC

Attachments

Hi,

On Wed, 22 Jan 2025 at 03:14, Michael Paquier <michael@paquier.xyz> wrote:
>
> On Thu, Jan 16, 2025 at 11:40:51AM +0300, Nazir Bilal Yavuz wrote:
> > I encountered another problem while rebasing the patch. The problem is
> > basically we do not expect any pending stats while restoring the stats
> > at the initdb. However, WAL IOs (WAL read and WAL init IOs for now)
> > may happen before restoring the stats, so we end up having pending
> > stats before restoring them and that causes initdb to fail.
>
> On top of 4feba03d8b92, I've reused something close to the patch you
> have posted previously in case, and the issue with allocations for
> pending stats should be gone.

Yes, they are fixed; thanks!

> Could it be possible to post a new version of the patch?  You should
> be able to reuse pgstat_count_backend_io_op[_time]() for your work
> with WAL data in pg_stat_io if you need a low-level control of things,
> but I suspect that calling pgstat_count_io_op() & the other should be
> enough to get the job done with a new IOObject.

I think there is only one problem remaining now. walsenders have stats
to report with this patch and they may shutdown after the
checkpointer, which causes '027_stream_regress.pl' test to fail.
Andres is already working on fixing that issue [1],
'027_stream_regress.pl' test passes after applying Andres' proposed
fix.

v9 is rebased and attached as three patches. The first one is a
squashed patch for the current version of Andres' proposed fix to pass
the CI, the second one is for adding WAL stats to pg_stat_io and the
third one is for fetching timing columns from pg_stat_io in the
pg_stat_wal view.

There is a change in the main patch (0002). Now, stats are being
flushed after the main loop in the PerformWalRecovery() function in
the xlogrecovery.c file. Stats were flushed in the main loop before
but I thought that might be costly so moved it to after main loop.

[1] postgr.es/m/flat/kgng5nrvnlv335evmsuvpnh354rw7qyazl73kdysev2cr2v5zu%40m3cfzxicm5kp

--
Regards,
Nazir Bilal Yavuz
Microsoft