Re: Wrong statistics for size of XLOG_SWITCH during pg_waldump.
movead.li@highgo.ca <movead.li@highgo.ca>
From: "movead.li@highgo.ca" <movead.li@highgo.ca>
To: Horiguchi <horikyota.ntt@gmail.com>, andres <andres@anarazel.de>
Cc: michael <michael@paquier.xyz>, pgsql-hackers <pgsql-hackers@postgresql.org>, ahsan.hadi <ahsan.hadi@highgo.ca>
Date: 2020-10-15T04:56:02Z
Lists: pgsql-hackers
Attachments
- fix_waldump_size_for_wal_switch_v3.patch (application/octet-stream) patch v3
Thanks for all the suggestions. >Yeah. In its current shape, it means that only pg_waldump would be >able to know this information. If you make this information part of >xlogdesc.c, any consumer of the WAL record descriptions would be able >to show this information, so it would provide a consistent output for >any kind of tools. I have change the implement, move some code into xlog_desc(). >On top of that, it seems to me that the calculation used in the patch >is wrong in two aspects at quick glance: >1) startSegNo and endSegNo point always to two different segments with >a XLOG_SWITCH record, so you should check that ReadRecPtr is not at a >segment border instead before extracting SizeOfXLogLongPHD, no? Yes you are right, and I use 'record->EndRecPtr - 1' instead. >2) This stuff should also check after the case of a WAL *page* border >where you'd need to adjust based on SizeOfXLogShortPHD instead. The 'junk_len -= SizeOfXLogLongPHD' code is considered for when the remain size of a wal segment can not afford a XLogRecord struct for XLOG_SWITCH, it needn't care *page* border. >I'm not sure the exact motive of this proposal, but if we show the >wasted length in the stats result, I think it should be other than >existing record types. Yes agree, and now it looks like below as new patch: movead@movead-PC:/h2/pg/bin$ ./pg_waldump -p ../walbk/ -s 0/3000000 -e 0/6000000 -z Type N (%) Record size (%) FPI size (%) Combined size (%) ---- - --- ----------- --- -------- --- ------------- --- XLOG 5 ( 31.25) 300 ( 0.00) 0 ( 0.00) 300 ( 0.00) XLOGSwitchJunk 3 ( 18.75) 50330512 (100.00) 0 ( 0.00) 50330512 (100.00) movead@movead-PC:/h2/pg/bin$ ./pg_waldump -p ../walbk/ -s 0/3000000 -e 0/6000000 --stat=record XLOG/SWITCH 3 ( 18.75) 72 ( 0.00) 0 ( 0.00) 72 ( 0.00) XLOG/SWITCH_JUNK 3 ( 18.75) 50330512 (100.00) 0 ( 0.00) 50330512 (100.00) The shortcoming now is I do not know how to handle the 'count' of SWITCH_JUNK in pg_waldump results. Currently I regard SWITCH_JUNK as one count. >By the way, I noticed that --stats=record shows two lines for >Transaction/COMMIT. The cause is that XLogDumpCountRecord assumes the >all four bits of xl_info is used to identify record id. Thanks,I didn't notice it before, and your patch added into v3 patch attached. Regards, Highgo Software (Canada/China/Pakistan) URL : www.highgo.ca EMAIL: mailto:movead(dot)li(at)highgo(dot)ca
Commits
-
pg_waldump: Fix bug in per-record statistics.
- e73068b0710b 9.6.22 landed
- 5386a8506d7d 10.17 landed
- 4a2627c41cfe 11.12 landed
- 4220e06f4c0c 12.7 landed
- 34279fd4fabd 13.3 landed
- 51893c846350 14.0 landed