Horton

Re: Vacuum statistics

Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>

View thread
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Alena Rybakina <a.rybakina@postgrespro.ru>, Alexander Korotkov <aekorotkov@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, Jim Nasby <jnasby@upgrade.com>, Bertrand Drouvot <bertranddrouvot.pg@gmail.com>, Ilia Evdokimov <ilya.evdokimov@tantorlabs.com>, Kirill Reshke <reshkekirill@gmail.com>, Andrei Zubkov <zubkov@moonset.ru>, Masahiko Sawada <sawada.mshk@gmail.com>, Melanie Plageman <melanieplageman@gmail.com>, jian he <jian.universality@gmail.com>, a.lepikhov@postgrespro.ru, Sami Imseih <samimseih@gmail.com>, vignesh C <vignesh21@gmail.com>
Date: 2025-09-25T00:03:08Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Move wal_buffers_full from PgStat_PendingWalStats to WalUsage

Hi,

On Mon, May 12, 2025 at 5:30 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Fri, May 9, 2025 at 5:34 PM Alena Rybakina <a.rybakina@postgrespro.ru> wrote:
> >
> > I did a rebase and finished the part with storing statistics separately from the relation statistics - now it is possible to disable the collection of statistics for relationsh using gucs and
> > this allows us to solve the problem with the memory consumed.
> >
>
> I think this patch is trying to collect data similar to what we do for
> pg_stat_statements for SQL statements. So, can't we follow a similar
> idea such that these additional statistics will be collected once some
> external module like pg_stat_statements is enabled? That module should
> be responsible for accumulating and resetting the data, so we won't
> have this memory consumption issue.
>
> BTW, how will these new statistics be used to autotune a vacuum? And
> do we need all the statistics proposed by this patch?

Thanks for working on this. I agree with the general idea of having
minimal changes to the core. I think a simple approach would be to
have a hook in heap_vacuum_rel at the end, where vacuum stats are
prepared in a buffer for emitting LOG messages. External modules can
then handle storing, rotating, interpreting, aggregating (per
relation/per database), and exposing the stats to end-users via SQL.
The core can define a common data structure, fill it, and send it to
external modules. I haven't had a chance to read the whole thread or
review the patches; I'm sure this has been discussed.

-- 
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com