Horton
Menu

Re: Stack-based tracking of per-node WAL/buffer usage

Lukas Fittl <lukas@fittl.com>

View thread
From: Lukas Fittl <lukas@fittl.com>
To: Andres Freund <andres@anarazel.de>
Cc: Heikki Linnakangas <hlinnaka@iki.fi>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Zsolt Parragi <zsolt.parragi@percona.com>, Tomas Vondra <tomas@vondra.me>, Peter Smith <smithpb2250@gmail.com>
Date: 2026-04-06T09:58:39Z
Lists: pgsql-hackers

Attachments

On Sun, Apr 5, 2026 at 4:12 PM Andres Freund <andres@anarazel.de> wrote:
>
> Hi,
>
> On 2026-04-05 17:02:28 -0400, Andres Freund wrote:
> > With that I pushed 0001.
>
> For 0002 I:
> - fixed a few comments still refering to node in the generic Instr* functions
> - added comment about the async_mode buglet to the commit message
> - added an async_mode argument to InstrInitNode(), as its callsite already
>   needed to be touched, and it felt wrong that InstrAllocNode() could do
>   things that were not possible with InstrInitNode()
> - Deduplicated the code between InstrStop() and InstrStotNode() by introducing
>   InstrStopCommon()
>
> After those (and some testing) I pushed this.
>

Thanks for pushing these two! And appreciate the refinements, they
make sense to me.

Attached v15. Quick summary:

0001 converts direct users of pgBufferUsage/pgWalUsage to the new
general purpose Instrumentation just pushed.
0002 introduces the macros needed for the stack-based instrumentation,
same as before.
0003 adds additional test coverage for buffer usage, same as before.
0004 is new, and adds queryDesc->totaltime_options for extensions to
request a certain level of totaltime measurement (this solves the
problem Andres noted in a review comment)
0005 is the stack-based instrumentation commit, now smaller and more
digestible, with the same performance benefits.

-- if we get up to here, we get the main benefit --

0006 is the parallel instrumentation cleanup. I don't think we need
this right now unless the EXPLAIN (IO) work changes course.
0007 is the same ExecProcNodeInstr change as before (this one we could
simplify by simply moving the function, getting about half the
possible speedup)
0008 is the table-specific buffer measurement for index scans (for
current master)
0009 is the test module for top level instrumentation data.

I've also attached an alternate for 0008, that works on top of the
index prefetch work (v23) - the change actually gets smaller because
heap fetches are better encapsulated then.

Thanks,
Lukas

-- 
Lukas Fittl

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. instrumentation: Allocate query level instrumentation in ExecutorStart

  2. instrumentation: Move ExecProcNodeInstr to allow inlining

  3. instrumentation: Separate per-node logic from other uses

  4. instrumentation: Separate trigger logic from other uses

  5. instrumentation: Rename INSTR_TIME_LT macro to INSTR_TIME_GT

  6. instrumentation: Keep time fields as instrtime, convert in callers