Horton

Re: WAL usage calculation patch

Amit Kapila <amit.kapila16@gmail.com>

View thread
From: Amit Kapila <amit.kapila16@gmail.com>
To: Dilip Kumar <dilipbalaut@gmail.com>, Julien Rouhaud <rjuju123@gmail.com>
Cc: Kuntal Ghosh <kuntalghosh.2007@gmail.com>, Fujii Masao <masao.fujii@oss.nttdata.com>, Kirill Bychik <kirill.bychik@gmail.com>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>, Thomas Munro <thomas.munro@gmail.com>
Date: 2020-04-04T05:08:14Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Change the display of WAL usage statistics in Explain.

  2. Cosmetic fixups for WAL usage work.

  3. Allow parallel create index to accumulate buffer usage stats.

  4. Allow autovacuum to log WAL usage statistics.

  5. Add the option to report WAL usage in EXPLAIN and auto_explain.

  6. Allow pg_stat_statements to track WAL usage statistics.

  7. Add infrastructure to track WAL usage.

  8. Include information on buffer usage during planning phase, in EXPLAIN output, take two.

  9. Include information on buffer usage during planning phase, in EXPLAIN output.

  10. Allow parallel vacuum to accumulate buffer usage.

  11. Allow pg_stat_statements to track planning statistics.

On Fri, Apr 3, 2020 at 7:36 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Fri, Apr 3, 2020 at 9:40 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >
> > On Fri, Apr 3, 2020 at 9:35 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
> > >
> > > I have analyzed the WAL and there could be multiple reasons for the
> > > same.  With small data, I have noticed that while inserting in the
> > > system index there was a Page Split and that created extra WAL.
> > >
> >
> > Thanks for the investigation.  I think it is clear that we can't
> > expect the same WAL size even if we repeat the same operation unless
> > it is a fresh database.
> >
>
> Attached find the latest patches.  I have modified based on our
> discussion on user interface thread [1], ran pgindent on all patches,
> slightly modified one comment based on Dilip's input and added commit
> messages.  I think the patches are in good shape.  I would like to
> commit the first patch in this series tomorrow unless I see more
> comments or any other objections.
>

Pushed.

>  The patch-2 might need to be
> rebased if the other related patch [2] got committed first and we
> might need to tweak a bit based on the input from other thread [1]
> where we are discussing user interface for it.
>

The primary question for patch-2 is whether we want to include WAL
usage information for the planning phase as we did for BUFFERS in
recent commit ce77abe63c (Include information on buffer usage during
planning phase, in EXPLAIN output, take two.).  Initially, I thought
it might be a good idea to do the same for WAL but after reading the
thread that leads to commit, I am not sure if there is any pressing
need to include WAL information for the planning phase.  Because
during planning we might not write much WAL (with the exception of WAL
due to setting of hint-bits) so users might not care much.  What do
you think?

-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com