Re: WAL usage calculation patch

Amit Kapila <amit.kapila16@gmail.com>

From: Amit Kapila <amit.kapila16@gmail.com>

To: Julien Rouhaud <rjuju123@gmail.com>

Cc: Kirill Bychik <kirill.bychik@gmail.com>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>, Fujii Masao <masao.fujii@oss.nttdata.com>, Thomas Munro <thomas.munro@gmail.com>

Date: 2020-03-28T10:44:04Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Change the display of WAL usage statistics in Explain.
- 69bfaf2e1de4 13.0 landed
Cosmetic fixups for WAL usage work.
- ef08ca113fe3 13.0 landed
Allow parallel create index to accumulate buffer usage stats.
- 5c71362174eb 13.0 landed
- f179e9f01b65 11.8 landed
- 13823ad3b012 12.3 landed
Allow autovacuum to log WAL usage statistics.
- b7ce6de93b59 13.0 landed
Add the option to report WAL usage in EXPLAIN and auto_explain.
- 33e05f89c53e 13.0 landed
Allow pg_stat_statements to track WAL usage statistics.
- 6b466bf5f2be 13.0 landed
Add infrastructure to track WAL usage.
- df3b181499b4 13.0 landed
Include information on buffer usage during planning phase, in EXPLAIN output, take two.
- ce77abe63cfc 13.0 cited
Include information on buffer usage during planning phase, in EXPLAIN output.
- ed7a5095716e 13.0 cited
Allow parallel vacuum to accumulate buffer usage.
- 3a5e22138a8d 13.0 landed
Allow pg_stat_statements to track planning statistics.
- 17e03282241c 13.0 cited

On Sat, Mar 28, 2020 at 12:54 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
>
> On Fri, Mar 27, 2020 at 8:21 PM Kirill Bychik <kirill.bychik@gmail.com> wrote:
> >
> >
> > All these are really valuable objections. Unfortunately, I won't be
> > able to get all sorted out soon, due to total lack of time. I would be
> > very glad if somebody could step in for this patch.
>
> I'll try to do that tomorrow!
>

I see some basic problems with the patch.  The way it tries to compute
WAL usage for parallel stuff doesn't seem right to me.  Can you share
or point me to any test done where we have computed WAL for parallel
operations like Parallel Vacuum or Parallel Create Index?  Basically,
I don't know changes done in ExecInitParallelPlan and friends allow us
to compute WAL for parallel operations.  Those will primarily cover
parallel queries that won't write WAL.  How you have tested those
changes?

-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com