RE: [Patch] Optimize dropping of relation buffers using dlist

k.jamison@fujitsu.com <k.jamison@fujitsu.com>

From: "k.jamison@fujitsu.com" <k.jamison@fujitsu.com>

To: "tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com>, 'Thomas Munro' <thomas.munro@gmail.com>

Cc: Kyotaro Horiguchi <horikyota.ntt@gmail.com>, Amit Kapila <amit.kapila16@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>, Andres Freund <andres@anarazel.de>, Robert Haas <robertmhaas@gmail.com>, Tomas Vondra <tomas.vondra@2ndquadrant.com>, pgsql-hackers <pgsql-hackers@postgresql.org>

Date: 2020-10-28T12:52:08Z

Lists: pgsql-hackers

Attachments

v28-0001-Prevent-invalidating-blocks-in-smgrextend-during.patch (application/octet-stream) patch v28-0001
v28-0002-Add-bool-param-in-smgrnblocks-for-cached-blocks.patch (application/octet-stream) patch v28-0002
v28-0003-Optimize-DropRelFileNodeBuffers-during-recovery.patch (application/octet-stream) patch v28-0003
v28-0004-TRUNCATE-optimization.patch (application/octet-stream) patch v28-0004

Hi everyone,

Attached are the updated set of patches (V28).
 0004 - Truncate optimization is a new patch, while the rest are similar to V27.
This passes the build, regression and TAP tests.

Apologies for the delay.
I'll post the benchmark test results on SSD soon, considering the suggested benchmark of Horiguchi-san: 
> Currently BUF_DROP_FULL_SCAN_THRESHOLD is set to Nbuffers / 512,
> which is quite arbitrary that comes from a wild guess.
> 
> Perhaps we need to run benchmarks that drops one relation with several
> different ratios between the number of buffers to-be-dropped and Nbuffers,
> and preferably both on spinning rust and SSD.

Regards,
Kirk Jamison

Commits

Fix size overflow in calculation introduced by commits d6ad34f3 and bea449c6.
- 519e4c9ee21a 14.0 landed
Optimize DropRelFileNodesAllBuffers() for recovery.
- bea449c635c0 14.0 landed
Optimize DropRelFileNodeBuffers() for recovery.
- d6ad34f3410f 14.0 landed
Cache smgrnblocks() results in recovery.
- c5315f4f4484 14.0 cited
Add a check to prevent overwriting valid data if smgrnblocks() gives a
- ffae5cc5a602 8.2.0 cited