Re: per backend I/O statistics

Bertrand Drouvot <bertranddrouvot.pg@gmail.com>

From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: Andres Freund <andres@anarazel.de>, Nazir Bilal Yavuz <byavuz81@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>, pgsql-hackers@lists.postgresql.org
Date: 2025-01-17T10:23:48Z
Lists: pgsql-hackers

Attachments

Hi,

On Fri, Jan 17, 2025 at 03:12:36PM +0900, Michael Paquier wrote:
> On Fri, Jan 17, 2025 at 06:06:35AM +0000, Bertrand Drouvot wrote:
> > On Fri, Jan 17, 2025 at 09:08:02AM +0900, Michael Paquier wrote:
> >> I could tweak that around the beginning of next week with a proposal
> >> of patch.  Bertrand, perhaps you'd prefer hack on this one?
> > 
> > Yeah, I had in mind to work on it (in the exact same line as you are describing
> > above). I'll work on it.

Please find attached a patch implementing the ideas above, meaning:

- It creates a new PendingBackendStats variable
- It uses this variable to increment and flush per backend pending IO statistics

That way we get rid of the memory allocation for pending IO statistics.

One remark: a special case has been added in pgstat_flush_pending_entries(). The
reason is that while the per backend stats are "variable-numbered" stats, it could
be that their pending stats are not part of pgStatPending. This is currently the
case (with this patch) as the per backend pending IO stats are not tracked with
pgstat_prep_backend_pending() and friends anymore.

Thoughts?

Regards,

-- 
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Commits

  1. Rework handling of pending data for backend statistics

  2. Rename some pgstats callbacks related to flush of entries

  3. Relax regression test for fsync check of backend-level stats

  4. Add backend-level statistics to pgstats

  5. Extract logic filling pg_stat_get_io()'s tuplestore into its own routine

  6. Tweak some comments related to variable-numbered stats in pgstat.c