Horton

Re: pg_stat_statements issue with parallel maintenance (Was Re: WAL usage calculation patch)

Dilip Kumar <dilipbalaut@gmail.com>

View thread
From: Dilip Kumar <dilipbalaut@gmail.com>
To: Peter Geoghegan <pg@bowt.ie>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Masahiko Sawada <masahiko.sawada@2ndquadrant.com>, Julien Rouhaud <rjuju123@gmail.com>, Robert Haas <robertmhaas@gmail.com>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Date: 2020-04-02T03:43:05Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Change the display of WAL usage statistics in Explain.

  2. Cosmetic fixups for WAL usage work.

  3. Allow parallel create index to accumulate buffer usage stats.

  4. Allow autovacuum to log WAL usage statistics.

  5. Add the option to report WAL usage in EXPLAIN and auto_explain.

  6. Allow pg_stat_statements to track WAL usage statistics.

  7. Add infrastructure to track WAL usage.

  8. Include information on buffer usage during planning phase, in EXPLAIN output, take two.

  9. Include information on buffer usage during planning phase, in EXPLAIN output.

  10. Allow parallel vacuum to accumulate buffer usage.

  11. Allow pg_stat_statements to track planning statistics.

On Thu, Apr 2, 2020 at 8:34 AM Peter Geoghegan <pg@bowt.ie> wrote:
>
> On Wed, Apr 1, 2020 at 7:52 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > Peter, Is this behavior expected?
> >
> > Let me summarize the situation so that it would be easier for Peter to
> > comment.  Julien has noticed that parallel vacuum and parallel create
> > index doesn't seem to report correct values for buffer usage stats.
> > Sawada-San wrote a patch to fix the problem for both the cases.  We
> > expect that 'total_read_blks' as reported in pg_stat_statements should
> > give the same value for parallel and non-parallel operations.  We see
> > that is true for parallel vacuum and previously we have the same
> > observation for the parallel query. Now, for parallel create index
> > this doesn't seem to be true as test results by Dilip show that.  We
> > have two possibilities here (a) there is some bug in Sawada-San's
> > patch or (b) this is expected behavior for parallel create index.
> > What do you think?
>
> nbtree CREATE INDEX doesn't even go through the buffer manager.

Thanks for clarifying.  So IIUC, it will not go through the buffer
manager for the index pages,  but for the heap pages, it will still go
through the buffer manager.

> The
> difference that Dilip showed is probably due to extra catalog accesses
> in the two parallel workers -- pg_amproc lookups, and the like. Those
> are rather small differences, overall.

> Can Dilip demonstrate the the "extra" buffer accesses are
> proportionate to the number of workers launched in some constant,
> predictable way?

Okay, I will test this.

-- 
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com