Re: [Proposal] Adding callback support for custom statistics kinds

Sami Imseih <samimseih@gmail.com>

From: Sami Imseih <samimseih@gmail.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2025-11-19T20:10:42Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →
  1. Fix const correctness in pgstat data serialization callbacks

  2. test_custom_stats: Add tests with read/write of auxiliary data

  3. Allow cumulative statistics to read/write auxiliary data from/to disk

  4. test_custom_stats: Test module for custom cumulative statistics

  5. injection_points: Remove portions related to custom pgstats

  6. Rename routines for write/read of pgstats file

Attachments

Sorry for the delay here.

v1 is the first attempt to address the feedback from the POC.

1/ A user is now able to register as many extra files as they
wish, and the files will be named pgstat.<kind_id>.<file no>.stat,
where file_no starts at 0 up to the number of files specified
by the user with .num_serialized_extra_files.

2/ The callbacks now provide both the core stat file as a FILE
pointer and an array of FILE pointers for the extra files.
IN the write callback, the extra file pointer is accessed
like extra_files[0], extra_files[1], etc., and the same for
the read callback.

3/ The patch centralizes the creation and cleanup of the files
with 2 new routines pgstat_allocate_files and pgstat_cleanup_files,
which both operate on a new local struct which tracks the file
names and descriptors in the read and write stats routines.

```
typedef struct PgStat_SerializeFiles
{
    char      **tmpfiles;
    char      **statfiles;
    FILE      **fd;
    int            num_files;
}            PgStat_SerializeFiles;
```

plug-ins are not made aware of this struct because they don't need
to. The callbacks are provided the FILE pointers they need to care
about for their kind only.

4/ In terms of testing, patch 0002, I did not want to invent a new module
for custom kinds, so I piggybacked off og injection_points as I did in the
POC, but I added on the existing recovery tests, because essentially that
is what we care. Does the data persist after a clean shutdown? do the
.tmp files get removed properly? etc. So I added tests in
recovery/t/029_stats_restart.pl for this.

--
Sami Imseih
Amazon Web Services (AWS)