Horton

Re: Pluggable cumulative statistics

Michael Paquier <michael@paquier.xyz>

View thread View on postgr.es
From: Michael Paquier <michael@paquier.xyz>
To: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Cc: Postgres hackers <pgsql-hackers@lists.postgresql.org>
Date: 2024-07-08T05:30:23Z
Lists: pgsql-hackers

Attachments

On Fri, Jul 05, 2024 at 09:35:19AM +0900, Michael Paquier wrote:
> On Thu, Jul 04, 2024 at 11:30:17AM +0000, Bertrand Drouvot wrote:
>> On Wed, Jul 03, 2024 at 06:47:15PM +0900, Michael Paquier wrote:
>>> - PgStat_Snapshot holds an array of void* also indexed by
>>> PGSTAT_NUM_KINDS, pointing to the fixed stats stored in the
>>> snapshots.
>> 
>> Same, that's just a 96 bytes overhead (8 * PGSTAT_NUM_KINDS) as compared to now.
> 
> Still Andres does not seem to like that much, well ;)

Please find attached a rebased patch set labelled v4.  Built-in
fixed-numbered stats are still attached to the snapshot and shmem
control structures, and custom fixed stats kinds are tracked in the
same way as v3 with new members tracking data stored in
TopMemoryContext for the snapshots and shmem for the control data.
So, the custom and built-in stats kinds are separated into separate
parts of the structures, including the "valid" flags for the
snapshots.  And this avoids any redirection when looking at the
built-in fixed-numbered stats.

I've tried at address all the previous comments (there could be stuff
I've missed while rebasing, of course).

The first three patches are refactoring pieces to make the rest more
edible, while 0004~ implement the main logic with templates in
modules/injection_points:
- 0001 refactors pgstat_write_statsfile() so as a loop om
PgStat_KindInfo is used to write the data.  This is done with the
addition of snapshot_ctl_off in PgStat_KindInfo, to point to the area
in PgStat_Snapshot where the data is located for fixed stats.
9004abf6206e has done the same for the read part.
- 0002 adds an init_shmem callback, to let stats kinds initialize
states based on what's been allocated.
- 0003 refactors the read/write to use a new entry type in the stats
file for fixed-numbered stats.
- 0004 switches PgStat_Kind from an enum to uint32, adding a better
type for pluggability.
- 0005 is the main implementation.
- 0006 adds some docs.
- 0007 (variable-numbered stats) and 0008 (fixed-numbered stats) are
the examples demonstrating how to make pluggable stats for both types,
with tests of their own.
--
Michael

Commits

  1. injection_points: Add some fixed-numbered statistics

  2. injection_points: Add some cumulative stats for injection points

  3. Add helper routines to retrieve data for custom fixed-numbered pgstats

  4. Introduce pluggable APIs for Cumulative Statistics

  5. Switch PgStat_Kind from an enum to a uint32 type

  6. Improve comment of pgstat_read_statsfile()

  7. Add a new 'F' entry type for fixed-numbered stats in pgstats file

  8. Add PgStat_KindInfo.init_shmem_cb

  9. Use pgstat_kind_infos to write fixed shared statistics

  10. Replace hardcoded identifiers of pgstats file by #defines