Horton

Re: WAL usage calculation patch

Julien Rouhaud <rjuju123@gmail.com>

View thread
From: Julien Rouhaud <rjuju123@gmail.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Dilip Kumar <dilipbalaut@gmail.com>, Kuntal Ghosh <kuntalghosh.2007@gmail.com>, Fujii Masao <masao.fujii@oss.nttdata.com>, Kirill Bychik <kirill.bychik@gmail.com>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>, Thomas Munro <thomas.munro@gmail.com>
Date: 2020-04-04T06:03:03Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Change the display of WAL usage statistics in Explain.

  2. Cosmetic fixups for WAL usage work.

  3. Allow parallel create index to accumulate buffer usage stats.

  4. Allow autovacuum to log WAL usage statistics.

  5. Add the option to report WAL usage in EXPLAIN and auto_explain.

  6. Allow pg_stat_statements to track WAL usage statistics.

  7. Add infrastructure to track WAL usage.

  8. Include information on buffer usage during planning phase, in EXPLAIN output, take two.

  9. Include information on buffer usage during planning phase, in EXPLAIN output.

  10. Allow parallel vacuum to accumulate buffer usage.

  11. Allow pg_stat_statements to track planning statistics.

On Sat, Apr 04, 2020 at 10:38:14AM +0530, Amit Kapila wrote:
> On Fri, Apr 3, 2020 at 7:36 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >
> > On Fri, Apr 3, 2020 at 9:40 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > >
> > > On Fri, Apr 3, 2020 at 9:35 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
> > > >
> > > > I have analyzed the WAL and there could be multiple reasons for the
> > > > same.  With small data, I have noticed that while inserting in the
> > > > system index there was a Page Split and that created extra WAL.
> > > >
> > >
> > > Thanks for the investigation.  I think it is clear that we can't
> > > expect the same WAL size even if we repeat the same operation unless
> > > it is a fresh database.
> > >
> >
> > Attached find the latest patches.  I have modified based on our
> > discussion on user interface thread [1], ran pgindent on all patches,
> > slightly modified one comment based on Dilip's input and added commit
> > messages.  I think the patches are in good shape.  I would like to
> > commit the first patch in this series tomorrow unless I see more
> > comments or any other objections.
> >
> 
> Pushed.


Thanks!


> >  The patch-2 might need to be
> > rebased if the other related patch [2] got committed first and we
> > might need to tweak a bit based on the input from other thread [1]
> > where we are discussing user interface for it.
> >
> 
> The primary question for patch-2 is whether we want to include WAL
> usage information for the planning phase as we did for BUFFERS in
> recent commit ce77abe63c (Include information on buffer usage during
> planning phase, in EXPLAIN output, take two.).  Initially, I thought
> it might be a good idea to do the same for WAL but after reading the
> thread that leads to commit, I am not sure if there is any pressing
> need to include WAL information for the planning phase.  Because
> during planning we might not write much WAL (with the exception of WAL
> due to setting of hint-bits) so users might not care much.  What do
> you think?


I agree that WAL activity during planning shouldn't be very frequent, but it
might still be worthwhile to add it.  I'm wondering how stable the normalized
WAL information would be in some regression tests, as the counters are only
showed if non zero.  Maybe it'd be better to remove them from the output, same
as the buffers?