Horton

Re: WAL usage calculation patch

Amit Kapila <amit.kapila16@gmail.com>

View thread
From: Amit Kapila <amit.kapila16@gmail.com>
To: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Cc: Julien Rouhaud <rjuju123@gmail.com>, Justin Pryzby <pryzby@telsasoft.com>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>, Dilip Kumar <dilipbalaut@gmail.com>, Kuntal Ghosh <kuntalghosh.2007@gmail.com>, Fujii Masao <masao.fujii@oss.nttdata.com>, Kirill Bychik <kirill.bychik@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, Thomas Munro <thomas.munro@gmail.com>
Date: 2020-04-23T09:05:07Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Change the display of WAL usage statistics in Explain.

  2. Cosmetic fixups for WAL usage work.

  3. Allow parallel create index to accumulate buffer usage stats.

  4. Allow autovacuum to log WAL usage statistics.

  5. Add the option to report WAL usage in EXPLAIN and auto_explain.

  6. Allow pg_stat_statements to track WAL usage statistics.

  7. Add infrastructure to track WAL usage.

  8. Include information on buffer usage during planning phase, in EXPLAIN output, take two.

  9. Include information on buffer usage during planning phase, in EXPLAIN output.

  10. Allow parallel vacuum to accumulate buffer usage.

  11. Allow pg_stat_statements to track planning statistics.

On Thu, Apr 23, 2020 at 12:16 PM Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:
>
> On 2020-04-23 07:31, Julien Rouhaud wrote:
> > I agree that full page writes can be used in this case, but I'm
> > wondering if that can be misleading for some reader which might e.g.
> > confuse with the full_page_writes GUC.  And as Justin pointed out, the
> > documentation for now usually mentions "full page image(s)" in such
> > cases.
>
> ISTM that in the context of this patch, "full-page image" is correct.  A
> "full-page write" is what you do to a table or index page when you are
> recovering a full-page image.
>

So what do we call when we log the page after it is touched after
checkpoint?  I thought we call that as full-page write.

>  The internal symbol for the WAL record is
> XLOG_FPI and xlogdesc.c prints it as "FPI".
>

That is just one way/reason we log the page.  There are others as
well.  I thought here we are computing the number of full-page writes
happened in the system due to various reasons like (a) a page is
operated upon first time after the checkpoint, (b) log the XLOG_FPI
record, (c) Guc for WAL consistency checker is on, etc.  If we see in
XLogRecordAssemble where we decide to log this information, there is a
comment " .... log a full-page write for the current block." and there
was an existing variable with 'fpw_lsn' which indicates to an extent
that what we are computing in this patch is full-page writes.  But
there is a reference to full-page image as well.  I think as
full_page_writes is an exposed variable that is well understood so
exposing information with similar name via this patch doesn't sound
illogical to me. Whatever we use here we need to be consistent all
throughout, even pg_stat_statements need to name exposed variable as
wal_fpi instead of wal_fpw.

To me, full-page writes sound more appealing with other WAL usage
variables like records and bytes. I might be more used to this term as
'fpw' that is why it occurred better to me.  OTOH, if most of us think
that a full-page image is better suited here, I am fine with changing
it at all places.

With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com