Re: WAL usage calculation patch
Amit Kapila <amit.kapila16@gmail.com>
From: Amit Kapila <amit.kapila16@gmail.com>
To: Julien Rouhaud <rjuju123@gmail.com>
Cc: Fujii Masao <masao.fujii@oss.nttdata.com>,
Kirill Bychik <kirill.bychik@gmail.com>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>,
Thomas Munro <thomas.munro@gmail.com>
Date: 2020-03-30T10:22:38Z
Lists: pgsql-hackers
Commits
Same data as JSON:
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Change the display of WAL usage statistics in Explain.
- 69bfaf2e1de4 13.0 landed
-
Cosmetic fixups for WAL usage work.
- ef08ca113fe3 13.0 landed
-
Allow parallel create index to accumulate buffer usage stats.
- 5c71362174eb 13.0 landed
- f179e9f01b65 11.8 landed
- 13823ad3b012 12.3 landed
-
Allow autovacuum to log WAL usage statistics.
- b7ce6de93b59 13.0 landed
-
Add the option to report WAL usage in EXPLAIN and auto_explain.
- 33e05f89c53e 13.0 landed
-
Allow pg_stat_statements to track WAL usage statistics.
- 6b466bf5f2be 13.0 landed
-
Add infrastructure to track WAL usage.
- df3b181499b4 13.0 landed
-
Include information on buffer usage during planning phase, in EXPLAIN output, take two.
- ce77abe63cfc 13.0 cited
-
Include information on buffer usage during planning phase, in EXPLAIN output.
- ed7a5095716e 13.0 cited
-
Allow parallel vacuum to accumulate buffer usage.
- 3a5e22138a8d 13.0 landed
-
Allow pg_stat_statements to track planning statistics.
- 17e03282241c 13.0 cited
On Sun, Mar 29, 2020 at 5:49 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
>
@@ -1249,6 +1250,16 @@ XLogInsertRecord(XLogRecData *rdata,
ProcLastRecPtr = StartPos;
XactLastRecEnd = EndPos;
+ /* Provide WAL update data to the instrumentation */
+ if (inserted)
+ {
+ pgWalUsage.wal_bytes += rechdr->xl_tot_len;
+ if (doPageWrites && fpw_lsn <= RedoRecPtr)
+ pgWalUsage.wal_fpw_records++;
+ else
+ pgWalUsage.wal_records++;
+ }
+
I think the above code has multiple problems. (a) fpw_lsn can be
InvalidXLogRecPtr and still there could be full-page image (for ex.
when REGBUF_FORCE_IMAGE flag for buffer is set). (b) There could be
multiple FPW records while inserting a record; consider when there are
multiple registered buffers. I think the right place to figure this
out is XLogRecordAssemble. (c) There are cases when we also attach the
record data even when we decide to write FPW (cf. REGBUF_KEEP_DATA),
so we might want to increment wal_fpw_records and wal_records for such
cases.
I think the right place to compute this information is
XLogRecordAssemble even though we update it at the place where you
have it in the patch. You can probably compute that in local
variables and then transfer to pgWalUsage in XLogInsertRecord. I am
fine if you can think of some other way but the current patch doesn't
seem correct to me.
--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com