Re: WAL usage calculation patch

Amit Kapila <amit.kapila16@gmail.com>

From: Amit Kapila <amit.kapila16@gmail.com>

To: Julien Rouhaud <rjuju123@gmail.com>

Cc: Fujii Masao <masao.fujii@oss.nttdata.com>, Kirill Bychik <kirill.bychik@gmail.com>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>, Thomas Munro <thomas.munro@gmail.com>

Date: 2020-03-31T06:53:17Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Change the display of WAL usage statistics in Explain.
- 69bfaf2e1de4 13.0 landed
Cosmetic fixups for WAL usage work.
- ef08ca113fe3 13.0 landed
Allow parallel create index to accumulate buffer usage stats.
- 5c71362174eb 13.0 landed
- f179e9f01b65 11.8 landed
- 13823ad3b012 12.3 landed
Allow autovacuum to log WAL usage statistics.
- b7ce6de93b59 13.0 landed
Add the option to report WAL usage in EXPLAIN and auto_explain.
- 33e05f89c53e 13.0 landed
Allow pg_stat_statements to track WAL usage statistics.
- 6b466bf5f2be 13.0 landed
Add infrastructure to track WAL usage.
- df3b181499b4 13.0 landed
Include information on buffer usage during planning phase, in EXPLAIN output, take two.
- ce77abe63cfc 13.0 cited
Include information on buffer usage during planning phase, in EXPLAIN output.
- ed7a5095716e 13.0 cited
Allow parallel vacuum to accumulate buffer usage.
- 3a5e22138a8d 13.0 landed
Allow pg_stat_statements to track planning statistics.
- 17e03282241c 13.0 cited

On Mon, Mar 30, 2020 at 6:14 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
>
> On Mon, Mar 30, 2020 at 03:52:38PM +0530, Amit Kapila wrote:
> >
> > I think the right place to compute this information is
> > XLogRecordAssemble even though we update it at the place where you
> > have it in the patch.  You can probably compute that in local
> > variables and then transfer to pgWalUsage in XLogInsertRecord.  I am
> > fine if you can think of some other way but the current patch doesn't
> > seem correct to me.
>
> My previous approach was indeed totally broken.  v8 attached which hopefully
> will be ok.
>

This is better.  Few more comments:
1. The point (c) from my previous email doesn't seem to be fixed
properly.  Basically, the record data is only attached with FPW in
some particular cases like where REGBUF_KEEP_DATA is set, but the
patch assumes it is always set.

2.
+ /* Report a full page imsage constructed for the WAL record */
+ *num_fpw += 1;

Typo. /imsage/image

3.  We need to enhance the patch to cover WAL usage for parallel
vacuum and parallel create index based on Sawada-San's latest patch[1]
which fixed the case for buffer usage.

[1] - https://www.postgresql.org/message-id/CA%2Bfd4k5L4yVoWz0smymmqB4_SMHd2tyJExUgA_ACsL7k00B5XQ%40mail.gmail.com

-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com