Re: Stack-based tracking of per-node WAL/buffer usage
Lukas Fittl <lukas@fittl.com>
From: Lukas Fittl <lukas@fittl.com>
To: Andres Freund <andres@anarazel.de>
Cc: Heikki Linnakangas <hlinnaka@iki.fi>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>,
Tomas Vondra <tomas@vondra.me>, Peter Smith <smithpb2250@gmail.com>,
Zsolt Parragi <zsolt.parragi@percona.com>
Date: 2026-04-07T20:30:11Z
Lists: pgsql-hackers
Attachments
- v16-0002-instrumentation-Move-ExecProcNodeInstr-to-allow-.patch (application/x-patch) patch v16-0002
- v16-0004-instrumentation-Replace-direct-changes-of-pgBuff.patch (application/x-patch) patch v16-0004
- v16-0003-instrumentation-Use-Instrumentation-instead-of-m.patch (application/x-patch) patch v16-0003
- v16-0001-instrumentation-Allocate-queryDesc-totaltime-in-.patch (application/x-patch) patch v16-0001
- v16-0005-instrumentation-Add-additional-regression-tests-.patch (application/x-patch) patch v16-0005
- v16-0007-instrumentation-Use-Instrumentation-struct-for-p.patch (application/x-patch) patch v16-0007
- v16-0006-Optimize-measuring-WAL-buffer-usage-through-stac.patch (application/x-patch) patch v16-0006
- v16-0008-instrumentation-Optimize-ExecProcNodeInstr-instr.patch (application/x-patch) patch v16-0008
- v16-0010-Add-test_session_buffer_usage-test-module.patch (application/x-patch) patch v16-0010
- v16-0009-Index-scans-Show-table-buffer-accesses-separatel.patch (application/x-patch) patch v16-0009
On Mon, Apr 6, 2026 at 5:39 PM Lukas Fittl <lukas@fittl.com> wrote: > > On Mon, Apr 6, 2026 at 3:46 PM Zsolt Parragi <zsolt.parragi@percona.com> wrote: > > > > I couldn't find any issues with v15, all comments are stylistic/minor, > > except maybe the first one. > > Thanks for reviewing! > > ... > > I'll hold off on posting another patch round since what you raised > were just small stylistic issues, and they don't apply to the > remaining prep patches before the stack patch itself. Attached v16, rebased with Zsolt's feedback addressed. I've also re-ordered as follows: 0001 is the change to make queryDesc->totaltime be allocated by ExecutorStart instead of plugins themselves, and adds a queryDesc->totaltime_options to have plugins request which level of summary instrumentation they need. This change is pretty simple, and could still make sense to get into 19. Because of the earlier Instrumentation refactoring that was pushed (thanks!) we're already asking extensions allocating queryDesc->totaltime to modify their use of InstrAlloc, so I think we might as well clean this up now. 0002 is just ExecProcNodeInstr moved to instrument.c, as Andres had suggested previously. We still get some quick performance wins from doing that (see end of email), and again, its a simple change, so could be considered if someone has bandwidth remaining. I've added a later patch that then does the more complex inlining and gets us the full speed up. At this point I'd say its safe to say that we should push out later changes to PG20, because it needs another good look over, and I don't think Andres or Heikki have the capacity for that today (but I really appreciate all the effort put in by both of you!). --- 0002 measurements (with current master and TSC clock source used for timing, best of three): CREATE TABLE lotsarows(key int not null); INSERT INTO lotsarows SELECT generate_series(1, 50000000); VACUUM FREEZE lotsarows; master: 265.319 ms actual runtime 308.532 ms TIMING OFF, BUFFERS OFF 375.810 ms TIMING OFF, BUFFERS ON 381.701 ms TIMING ON, BUFFERS OFF 437.722 ms TIMING ON, BUFFERS ON 0002: 265.207 ms actual runtime 291.799 ms TIMING OFF, BUFFERS OFF 364.653 ms TIMING OFF, BUFFERS ON 359.759 ms TIMING ON, BUFFERS OFF 433.023 ms TIMING ON, BUFFERS ON full patch set: 265.763 ms actual runtime 273.222 ms TIMING OFF, BUFFERS OFF 293.621 ms TIMING OFF, BUFFERS ON 331.926 ms TIMING ON, BUFFERS OFF 363.055 ms TIMING ON, BUFFERS ON Thanks, Lukas -- Lukas Fittl
Commits
Same data as JSON:
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
instrumentation: Allocate query level instrumentation in ExecutorStart
- 2c16deee2f7d 19 (unreleased) landed
-
instrumentation: Move ExecProcNodeInstr to allow inlining
- 544000288ec8 19 (unreleased) landed
-
instrumentation: Separate per-node logic from other uses
- 5a79e78501f4 19 (unreleased) landed
-
instrumentation: Separate trigger logic from other uses
- 7d9b74df53e9 19 (unreleased) landed
-
instrumentation: Rename INSTR_TIME_LT macro to INSTR_TIME_GT
- 3218825271bd 19 (unreleased) landed
-
instrumentation: Keep time fields as instrtime, convert in callers
- e5a5e0a90750 19 (unreleased) landed