Horton

Re: WAL usage calculation patch

Julien Rouhaud <rjuju123@gmail.com>

View thread
From: Julien Rouhaud <rjuju123@gmail.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Dilip Kumar <dilipbalaut@gmail.com>, Kuntal Ghosh <kuntalghosh.2007@gmail.com>, Fujii Masao <masao.fujii@oss.nttdata.com>, Kirill Bychik <kirill.bychik@gmail.com>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>, Thomas Munro <thomas.munro@gmail.com>
Date: 2020-04-04T08:54:23Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Change the display of WAL usage statistics in Explain.

  2. Cosmetic fixups for WAL usage work.

  3. Allow parallel create index to accumulate buffer usage stats.

  4. Allow autovacuum to log WAL usage statistics.

  5. Add the option to report WAL usage in EXPLAIN and auto_explain.

  6. Allow pg_stat_statements to track WAL usage statistics.

  7. Add infrastructure to track WAL usage.

  8. Include information on buffer usage during planning phase, in EXPLAIN output, take two.

  9. Include information on buffer usage during planning phase, in EXPLAIN output.

  10. Allow parallel vacuum to accumulate buffer usage.

  11. Allow pg_stat_statements to track planning statistics.

On Sat, Apr 04, 2020 at 02:12:59PM +0530, Amit Kapila wrote:
> On Sat, Apr 4, 2020 at 11:33 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
> >
> > On Sat, Apr 04, 2020 at 10:38:14AM +0530, Amit Kapila wrote:
> >
> > > >  The patch-2 might need to be
> > > > rebased if the other related patch [2] got committed first and we
> > > > might need to tweak a bit based on the input from other thread [1]
> > > > where we are discussing user interface for it.
> > > >
> > >
> > > The primary question for patch-2 is whether we want to include WAL
> > > usage information for the planning phase as we did for BUFFERS in
> > > recent commit ce77abe63c (Include information on buffer usage during
> > > planning phase, in EXPLAIN output, take two.).  Initially, I thought
> > > it might be a good idea to do the same for WAL but after reading the
> > > thread that leads to commit, I am not sure if there is any pressing
> > > need to include WAL information for the planning phase.  Because
> > > during planning we might not write much WAL (with the exception of WAL
> > > due to setting of hint-bits) so users might not care much.  What do
> > > you think?
> >
> >
> > I agree that WAL activity during planning shouldn't be very frequent, but it
> > might still be worthwhile to add it.
> >
> 
> We can add if we want but I am not able to convince myself for that.
> Do you have any use case in mind?  I think in most of the cases
> (except for hint-bit WAL) it will be zero. If we are not sure of this
> we can also discuss it separately in a new thread once this
> patch-series is committed and see if anybody else sees the value of it
> and if so adding the code should be easy.


I'm mostly thinking of people trying to investigate possible slowdowns on a
hot-standby replica with a primary without wal_log_hints.  If they explicitly
ask for WAL information, we should provide them, even if it's quite unlikely to
happen.


> 
> >  I'm wondering how stable the normalized
> > WAL information would be in some regression tests, as the counters are only
> > showed if non zero.  Maybe it'd be better to remove them from the output, same
> > as the buffers?
> >
> 
> Which regression tests are you referring to? pg_stat_statements?  If
> so, why would it be unstable?  It should always generate WAL although
> the exact values may differ and we have already taken care of that in
> the patch, no?


I'm talking about a hypothetical new EXPLAIN (ALAYZE, WAL) regression test,
which could be unstable for similar reason to why the first attempt to add
BUFFERS in the planning part of EXPLAIN was unstable.  I thought that's why you
were hesitating of adding it.