RE: [Patch] Optimize dropping of relation buffers using dlist
k.jamison@fujitsu.com <k.jamison@fujitsu.com>
From: "k.jamison@fujitsu.com" <k.jamison@fujitsu.com>
To: 'Amit Kapila' <amit.kapila16@gmail.com>
Cc: Kyotaro Horiguchi <horikyota.ntt@gmail.com>, "tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com>, Andres Freund <andres@anarazel.de>, Tom Lane <tgl@sss.pgh.pa.us>, Thomas Munro <thomas.munro@gmail.com>, Robert Haas <robertmhaas@gmail.com>, Tomas Vondra <tomas.vondra@2ndquadrant.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2020-12-14T03:00:12Z
Lists: pgsql-hackers
Attachments
- v35-0001-Prevent-invalidating-blocks-in-smgrextend-during.patch (application/octet-stream) patch v35-0001
- v35-0002-Add-bool-param-in-smgrnblocks-for-cached-blocks.patch (application/octet-stream) patch v35-0002
- v35-0003-Optimize-DropRelFileNodeBuffers-during-recovery.patch (application/octet-stream) patch v35-0003
- v35-0004-Optimize-DropRelFileNodesAllBuffers-in-recovery.patch (application/octet-stream) patch v35-0004
On Friday, December 11, 2020 10:27 AM, Amit Kapila wrote: > On Fri, Dec 11, 2020 at 5:54 AM k.jamison@fujitsu.com > <k.jamison@fujitsu.com> wrote: > > So should I still not include that information? > > > > I think we can extend your existing comment like: "Otherwise if the size of a > relation fork is not cached, we proceed to a full scan of the whole buffer pool. > This can happen if there is no update to a particular fork during recovery." Attached are the final updated patches. I followed this advice and updated the source code comment a little bit. There are no changes from the previous except that and the unnecessary "cached" condition which Tsunakawa-san mentioned. Below is also the updated recovery performance test results for TRUNCATE. (1000 tables, 1MB per table, results measured in seconds) | s_b | Master | Patched | % Reg | |-------|--------|---------|---------| | 128MB | 0.406 | 0.406 | 0% | | 512MB | 0.506 | 0.406 | -25% | | 1GB | 0.806 | 0.406 | -99% | | 20GB | 15.224 | 0.406 | -3650% | | 100GB | 81.506 | 0.406 | -19975% | Because of the size of relation, it is expected to enter full-scan for the 128MB shared_buffers setting. And there was no regression. Similar to previous test results, the recovery time was constant for all shared_buffers setting with the patches applied. Regards, Kirk Jamison
Commits
-
Fix size overflow in calculation introduced by commits d6ad34f3 and bea449c6.
- 519e4c9ee21a 14.0 landed
-
Optimize DropRelFileNodesAllBuffers() for recovery.
- bea449c635c0 14.0 landed
-
Optimize DropRelFileNodeBuffers() for recovery.
- d6ad34f3410f 14.0 landed
-
Cache smgrnblocks() results in recovery.
- c5315f4f4484 14.0 cited
-
Add a check to prevent overwriting valid data if smgrnblocks() gives a
- ffae5cc5a602 8.2.0 cited