Horton

Re: WAL usage calculation patch

Kirill Bychik <kirill.bychik@gmail.com>

View thread
From: Kirill Bychik <kirill.bychik@gmail.com>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: Craig Ringer <craig@2ndquadrant.com>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Date: 2020-02-19T07:27:50Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Change the display of WAL usage statistics in Explain.

  2. Cosmetic fixups for WAL usage work.

  3. Allow parallel create index to accumulate buffer usage stats.

  4. Allow autovacuum to log WAL usage statistics.

  5. Add the option to report WAL usage in EXPLAIN and auto_explain.

  6. Allow pg_stat_statements to track WAL usage statistics.

  7. Add infrastructure to track WAL usage.

  8. Include information on buffer usage during planning phase, in EXPLAIN output, take two.

  9. Include information on buffer usage during planning phase, in EXPLAIN output.

  10. Allow parallel vacuum to accumulate buffer usage.

  11. Allow pg_stat_statements to track planning statistics.

вт, 18 февр. 2020 г. в 06:23, Thomas Munro <thomas.munro@gmail.com>:
> On Mon, Feb 10, 2020 at 8:20 PM Craig Ringer <craig@2ndquadrant.com> wrote:
> > On Wed, 5 Feb 2020 at 21:36, Kirill Bychik <kirill.bychik@gmail.com> wrote:
> > > Patch is separated in two parts: core changes and pg_stat_statements
> > > additions. Essentially the extension has its schema updated to allow
> > > two more fields, docs updated to reflect the change. Patch is prepared
> > > against master branch.
> > >
> > > Please provide your comments and/or code findings.
> >
> > I like the concept, I'm a big fan of anything that affordably improves
> > visibility into Pg's I/O and activity.
>
> +1
>
> > To date I've been relying on tools like systemtap to do this sort of
> > thing. But that's a bit specialised, and Pg currently lacks useful
> > instrumentation for it so it can be a pain to match up activity by
> > parallel workers and that sort of thing. (I aim to find time to submit
> > a patch for that.)
>
> (I'm interested in seeing your conference talk about that!  I did a
> bunch of stuff with static probes to measure PHJ behaviour around
> barrier waits and so on but it was hard to figure out what stuff like
> that to put in the actual tree, it was all a bit
> use-once-to-test-a-theory-and-then-throw-away.)
>
> Kirill, I noticed that you included a regression test that is failing.  Can
> this possibly be stable across machines or even on the same machine?
> Does it still pass for you or did something change on the master
> branch to add a new WAL record since you posted the patch?

Thank you for testing the patch and running extension checks. I assume
the patch applies without problems.

As for the regr test, it apparently requires some rework. I didn't pay
attention enough to make sure the data I check is actually meaningful
and isolated enough to be repeatable.

Please consider the extension part of the patch as WIP, I'll resubmit
the patch once I get a stable and meanngful test up. Thanks for
finding it!

> query | calls | rows | wal_write_bytes | wal_write_records
>  -------------------------------------------+-------+------+-----------------+-------------------
> - CREATE INDEX test_b ON test(b)            |     1 |    0 | 1673 |
>             16
> - DROP FUNCTION IF EXISTS PLUS_ONE(INTEGER) |     1 |    0 |   56 |
>              1
> + CREATE INDEX test_b ON test(b)            |     1 |    0 | 1755 |
>             17
> + DROP FUNCTION IF EXISTS PLUS_ONE(INTEGER) |     1 |    0 |    0 |
>              0