Re: per backend I/O statistics

Michael Paquier <michael@paquier.xyz>

From: Michael Paquier <michael@paquier.xyz>
To: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Cc: Nazir Bilal Yavuz <byavuz81@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>, pgsql-hackers@lists.postgresql.org
Date: 2024-10-08T04:46:23Z
Lists: pgsql-hackers
On Mon, Oct 07, 2024 at 09:54:21AM +0000, Bertrand Drouvot wrote:
> On Fri, Sep 20, 2024 at 01:26:49PM +0900, Michael Paquier wrote:
>> This would rely on the fact that we would use the ProcNumber for the
>> dshash key, and this information is not provided in pg_stat_activity.
>> Perhaps we should add this information in pg_stat_activity so as it
>> would be easily possible to do joins with a SQL function that returns
>> a SRF with all the stats associated with a given connection slot
>> (auxiliary or backend process)?
> 
> I'm not sure that's needed. What has been done in the previous versions is
> to get the stats based on the pid (see pg_stat_get_backend_io()) where the
> procnumber is retrieved with something like GetNumberFromPGProc(BackendPidGetProc(pid)).

Ah, I see.  So you could just have the proc number in the key to
control the upper-bound on the number of possible stats entries in the
dshash.

Assuming that none of this data is persisted to the stats file at
shutdown and that the stats of a single entry are reset each time a
new backend reuses a previous proc slot, that would be OK by me, I
guess.

>> The active PIDs of the live sessions are not stored in the active
>> stats, why not? 
> 
> That was not needed. We can still retrieve the stats based on the pid thanks
> to something like GetNumberFromPGProc(BackendPidGetProc(pid)) without having
> to actually store the pid in the stats. I think that's fine because the pid
> only matters at "display" time (pg_stat_get_backend_io()).

Okay, per the above and the persistency of the stats.
--
Michael

Commits

  1. Rework handling of pending data for backend statistics

  2. Rename some pgstats callbacks related to flush of entries

  3. Relax regression test for fsync check of backend-level stats

  4. Add backend-level statistics to pgstats

  5. Extract logic filling pg_stat_get_io()'s tuplestore into its own routine

  6. Tweak some comments related to variable-numbered stats in pgstat.c