Re: per backend I/O statistics

Bertrand Drouvot <bertranddrouvot.pg@gmail.com>

From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: Nazir Bilal Yavuz <byavuz81@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>, pgsql-hackers@lists.postgresql.org, Andres Freund <andres@anarazel.de>
Date: 2025-01-16T16:05:35Z
Lists: pgsql-hackers
Hi,

On Thu, Jan 16, 2025 at 06:48:58AM +0000, Bertrand Drouvot wrote:
> Hi,
> 
> On Thu, Jan 16, 2025 at 09:55:10AM +0900, Michael Paquier wrote:
> > On Wed, Jan 15, 2025 at 05:20:57PM +0300, Nazir Bilal Yavuz wrote:
> > > I think allowing only pgStatPendingContext to have
> > > MemoryContextAllowInCriticalSection() is not enough. We need to allow
> > > at least pgStatSharedRefContext as well to have
> > > MemoryContextAllowInCriticalSection() as it can be allocated too.
> > > 
> > > '''
> > > pgstat_prep_pending_entry() ->
> > > pgstat_get_entry_ref() ->
> > > pgstat_get_entry_ref_cached() ->
> > > MemoryContextAlloc(pgStatSharedRefContext, sizeof(PgStat_EntryRef))
> > > '''
> > 
> > Yep, I was pretty sure that we have a bit more going on.  Last time I
> > began digging into the issue I was loading injection_points in
> > shared_preload_libraries with stats enabled to see how much I could
> > break, and this was one pattern once I've forced the pending part.  I
> > didn't get through the whole exercise.
> 
> I'll look at it and come back with a proposal as part of [1].

I looked at it and shared the outcome in [1]. It looks like it's more complicated
as we could also hit a failed assertion in MemoryContextCreate(), like:

TRAP: failed Assert("CritSectionCount == 0"), File: "mcxt.c", Line: 1107, PID: 3295726
pg18/bin/postgres(ExceptionalCondition+0xbb)[0x59668bee1f6d]
pg18/bin/postgres(MemoryContextCreate+0x46)[0x59668bf2a8fe]
pg18/bin/postgres(AllocSetContextCreateInternal+0x1df)[0x59668bf1bb11]
pg18/bin/postgres(pgstat_prep_pending_entry+0x86)[0x59668bcff8cc]
pg18/bin/postgres(pgstat_prep_backend_pending+0x2b)[0x59668bd024a9]

I propose that as of now we discuss the issue in [1] instead of here.

[1]: https://www.postgresql.org/message-id/Z4ks4%2BwnAz8eEyX9%40ip-10-97-1-34.eu-west-3.compute.internal

Regards,

-- 
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com



Commits

  1. Rework handling of pending data for backend statistics

  2. Rename some pgstats callbacks related to flush of entries

  3. Relax regression test for fsync check of backend-level stats

  4. Add backend-level statistics to pgstats

  5. Extract logic filling pg_stat_get_io()'s tuplestore into its own routine

  6. Tweak some comments related to variable-numbered stats in pgstat.c