Horton

Re: WAL usage calculation patch

Amit Kapila <amit.kapila16@gmail.com>

View thread
From: Amit Kapila <amit.kapila16@gmail.com>
To: Julien Rouhaud <rjuju123@gmail.com>
Cc: Kirill Bychik <kirill.bychik@gmail.com>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>, Fujii Masao <masao.fujii@oss.nttdata.com>, Thomas Munro <thomas.munro@gmail.com>
Date: 2020-03-29T05:33:50Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Change the display of WAL usage statistics in Explain.

  2. Cosmetic fixups for WAL usage work.

  3. Allow parallel create index to accumulate buffer usage stats.

  4. Allow autovacuum to log WAL usage statistics.

  5. Add the option to report WAL usage in EXPLAIN and auto_explain.

  6. Allow pg_stat_statements to track WAL usage statistics.

  7. Add infrastructure to track WAL usage.

  8. Include information on buffer usage during planning phase, in EXPLAIN output, take two.

  9. Include information on buffer usage during planning phase, in EXPLAIN output.

  10. Allow parallel vacuum to accumulate buffer usage.

  11. Allow pg_stat_statements to track planning statistics.

On Sat, Mar 28, 2020 at 7:08 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
>
> On Sat, Mar 28, 2020 at 04:14:04PM +0530, Amit Kapila wrote:
> >
> > Basically,
> > I don't know changes done in ExecInitParallelPlan and friends allow us
> > to compute WAL for parallel operations.  Those will primarily cover
> > parallel queries that won't write WAL.  How you have tested those
> > changes?
>
> I didn't tested those, and I'm not even sure how to properly and reliably test
> that.  Do you have any advice on how to achieve that?
>
> However the patch is mimicking the buffer instrumentation that already exists,
> and the approach also looks correct to me.  Do you have a reason to believe
> that the approach that works for buffer usage wouldn't work for WAL records? (I
> of course agree that this should be tested anyway)
>

The buffer usage infrastructure is for read-only queries (for ex. for
stats like blks_hit, blks_read).  As far as I can think, there is no
easy way to test the WAL usage via that API.  It might or might not be
required in the future depending on whether we decide to use the same
infrastructure for parallel writes.  I think for now we should remove
that part of changes and rather think how to get that for parallel
operations that can write WAL.  For ex. we might need to do something
similar to what this patch has done in begin_parallel_vacuum and
end_parallel_vacuum.  Would you like to attempt that?

-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com