Horton

Re: per backend WAL statistics

Nazir Bilal Yavuz <byavuz81@gmail.com>

View thread View on postgr.es
From: Nazir Bilal Yavuz <byavuz81@gmail.com>
To: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Cc: Michael Paquier <michael@paquier.xyz>, Andres Freund <andres@anarazel.de>, Xuneng Zhou <xunengzhou@gmail.com>, pgsql-hackers@lists.postgresql.org
Date: 2025-03-10T16:07:05Z
Lists: pgsql-hackers 
Hi,

On Mon, 10 Mar 2025 at 17:43, Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:
> On Mon, Mar 10, 2025 at 03:08:49PM +0300, Nazir Bilal Yavuz wrote:
>
> > I think that every time we flush IO or WAL stats, we want(?) to flush
> > backend stats as well,
>
> Yeah, I think that's happening anyway.
>
> > so would it make sense to move
> > pgstat_flush_backend() calls to inside of pgstat_flush_io() and
> > pgstat_wal_flush_cb()?
>
> I don't think so because pgstat_flush_backend() still needs to be called by the
> pgstat_backend_flush_cb() (i.e flush_static_cb) callback (I mean I think this
> makes sense to keep this callback around and that it does "really" something).
>
> So for example, for the WAL case, that would mean the backend WAL stats would be
> flushed twice: one time from pgstat_wal_flush_cb() (i.e flush_static_cb) callback
> and one time from the pgstat_backend_flush_cb() (another flush_static_cb) callback.
>
> I think it's better to keep them separate and reason as they are distinct
> types of stats (which they really are). I think we had the same kind of reasoning
> while working on [1].

I got it, that makes sense. Thanks for the explanation.

> > I see that backend statistics are not collected
> > for some of the backend types but that is already checked in the
> > pgstat_flush_backend() with pgstat_tracks_backend_bktype().
>
> Sorry, I don't get it. Do you have a question around that?

Sorry for the confusion. I did not think of the explanation that you
did above. I was thinking that we do not want to call
pgstat_flush_backend(.., PGSTAT_BACKEND_FLUSH_IO) for the (let's say)
checkpointer as its backend statistics are not collected and that is
the reason why we do not want to put pgstat_flush_backend() inside of
pgstat_flush_io(). Your explanation made it clear now, no other
questions.

-- 
Regards,
Nazir Bilal Yavuz
Microsoft

Commits

  1. Fix use-after-free in pgstat_fetch_stat_backend_by_pid()

  2. Remove initialization from PendingBackendStats

  3. Add WAL data to backend statistics

  4. Improve check for detection of pending data in backend statistics

  5. Fix some gaps in pg_stat_io with WAL receiver and WAL summarizer

  6. Handle auxiliary processes in SQL functions of backend statistics

  7. Invent pgstat_fetch_stat_backend_by_pid()

  8. Refactor code of pg_stat_get_wal() building result tuple

  9. Adding new PgStat_WalCounters structure in pgstat.h

  10. Remove pgstat_flush_wal()

  11. Refactor some code related to backend statistics

  12. Add backend-level statistics to pgstats