Re: per backend I/O statistics

Nazir Bilal Yavuz <byavuz81@gmail.com>

From: Nazir Bilal Yavuz <byavuz81@gmail.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>, pgsql-hackers@lists.postgresql.org
Date: 2025-01-15T08:03:54Z
Lists: pgsql-hackers
Hi,

On Thu, 19 Dec 2024 at 07:22, Michael Paquier <michael@paquier.xyz> wrote:
>
> Fixed that, bumped the two version counters, and done.

I encountered a problem while trying to add WAL stats to pg_stat_io
and I wanted to hear your thoughts.

Right now, pgstat_prep_backend_pending() is called in both
pgstat_count_io_op() and pgstat_count_io_op_time() to create a local
PgStat_BackendPending entry. In that process,
pgstat_prep_pending_entry() -> MemoryContextAllocZero() is called. The
problem is that MemoryContextAllocZero() can not be called in the
critical sections.

For example, here is what happens in the walsender backend:

'''
... ->
exec_replication_command() ->
SendBaseBackup() ->
...  ->
XLogInsertRecord() ->
START_CRIT_SECTION() /* Now we are in the critical section */ ->
... ->
XLogWrite() ->
pgstat_count_io_op_time() for the pg_pwrite() IO ->
pgstat_prep_backend_pending() ->
pgstat_prep_pending_entry() ->
MemoryContextAllocZero() ->
Failed at Assert("CritSectionCount == 0 || (context)->allowInCritSection")
'''

With this commit it may not be possible to count IOs in the critical
sections. I think the problem happens only if the local
PgStat_BackendPending entry is being created for the first time for
this backend in the critical section.

--
Regards,
Nazir Bilal Yavuz
Microsoft



Commits

  1. Rework handling of pending data for backend statistics

  2. Rename some pgstats callbacks related to flush of entries

  3. Relax regression test for fsync check of backend-level stats

  4. Add backend-level statistics to pgstats

  5. Extract logic filling pg_stat_get_io()'s tuplestore into its own routine

  6. Tweak some comments related to variable-numbered stats in pgstat.c