Horton

Re: per backend I/O statistics

Michael Paquier <michael@paquier.xyz>

View thread View on postgr.es
From: Michael Paquier <michael@paquier.xyz>
To: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Cc: Andres Freund <andres@anarazel.de>, Nazir Bilal Yavuz <byavuz81@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>, pgsql-hackers@lists.postgresql.org
Date: 2025-01-20T06:34:41Z
Lists: pgsql-hackers

Attachments

On Sat, Jan 18, 2025 at 05:53:31PM +0900, Michael Paquier wrote:
> Hmm.  Such special complexities in pgstat.c are annoying.  There is
> a stupid thing I am wondering here.  For the WAL stats, why couldn't
> we place some calls of pgstat_prep_backend_pending() in strategic
> places like XLogBeginInsert() to force all the allocation steps of the
> pending entry to happen before we would enter the critical sections
> when doing a WAL insertion?  As far as I can see, there is a special
> case with 2PC where XLogBeginInsert() could be called in a critical
> section, but that seems to be the only one at quick glance.

I've looked first at this idea over the week-end with a quick hack,
and it did not finish well.

And then it struck me that with a bit of redesign of the callbacks, so
as the existing flush_fixed_cb and have_fixed_pending_cb are usable
with variable-numbered stats, we should be able to avoid the exception
your first version of the patch has introduced.  Attached is a patch
to achieve what I have in mind, which is more generic than your
previous patch.  I've built my stuff as 0002 on top of your 0001.

One key choice I have made is to hide PendingBackendStats within
pgstat_backend.c so as it is possible to enforce some sanity checks
there.
--
Michael

Commits

  1. Rework handling of pending data for backend statistics

  2. Rename some pgstats callbacks related to flush of entries

  3. Relax regression test for fsync check of backend-level stats

  4. Add backend-level statistics to pgstats

  5. Extract logic filling pg_stat_get_io()'s tuplestore into its own routine

  6. Tweak some comments related to variable-numbered stats in pgstat.c