Move wal_buffers_full to WalUsage (and report it in pgss/explain)
Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
To: pgsql-hackers@lists.postgresql.org
Date: 2025-02-06T10:27:17Z
Lists: pgsql-hackers
Attachments
- v1-0001-Move-wal_buffers_full-to-pgWalUsage.patch (text/x-diff)
Hi hackers, While working on per-backend WAL statistics ([1]) it has been discussed to move wal_buffers_full to WalUsage (It's currently tracked in PgStat_PendingWalStats). There is some benefits doing so: - Simplifies [1] - Adds the ability to report it in pg_stat_statements (among wal_records, wal_fpi and wal_bytes) - Adds the ability to report it in explain/auto_explain (among wal_records, wal_fpi and wal_bytes) I did a bit of archeology to try to understand why it's not already the case. From what I can see, in commit time order: 1. df3b181499 introduced the WalUsage structure 2. 6b466bf5f2 added the wal usage in pg_stat_statements 3. 33e05f89c5 added the wal usage in EXPLAIN 4. 8d9a935965f added pg_stat_wal (and wal_buffers_full) 5. 01469241b2f added the wal usage in pg_stat_wal So, wal_buffers_full has been introduced after the WalUsage structure was there but I don't see any reason in the emails/threads as to why it's not in the WalUsage structure. ==== A few thoughts: About explain: One could not be 100% sure that the statement being explained is fully responsible of the wal buffer being full (as it could just be a "victim" of an already almost full wal buffer). But OTOH that might help to understand why an EXPLAIN analyze is slower than another one (i.e one generating wal buffer full and the other not). About pg_stat_statements: That could also provide valuable information. Say if one can observe that a statement generates (at least) a WAL buffer full each time it's executed or not. The later could explain "un-stable" performance for the statement (if WAL buffer full is not zero). The former could try to be improved if needed. === Patch's structure The patch series is made of 3 "small" sub-patches: 0001: to move wal_buffers_full to WalUsage 0002: to report wal_buffers_full in pg_stat_statements 0003: to report wal_buffers_full in explain/auto_explain ==== Some remarks: 0002: it doesn't bump the version of pg_stat_statements as the same is done for this release in commit cf54a2c00254. 0002 and 0003 don't add tests. It looks like there is no existing tests for wal_buffers_full, reason probably is that it would not be easy to have a stable and performant test. [1]: https://www.postgresql.org/message-id/Z6M7/zq4vgE4lch2%40ip-10-97-1-34.eu-west-3.compute.internal Looking forward to your feedback, Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com
Commits
-
Add information about WAL buffers full to VACUUM/ANALYZE (VERBOSE)
- 6a8a7ce476bd 18.0 landed
-
Add information about WAL buffers being full to EXPLAIN (WAL)
- 320545bfcfee 18.0 landed
-
pg_stat_statements: Add wal_buffers_full
- ce5bcc4a9f26 18.0 landed
-
Move wal_buffers_full from PgStat_PendingWalStats to WalUsage
- eaf502747bac 18.0 landed
-
pg_stat_statements: Add columns to track parallel worker activity
- cf54a2c00254 18.0 cited