Re: [Patch] Optimize dropping of relation buffers using dlist
Kyotaro Horiguchi <horikyota.ntt@gmail.com>
From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
To: tsunakawa.takay@fujitsu.com
Cc: k.jamison@fujitsu.com, amit.kapila16@gmail.com, tgl@sss.pgh.pa.us,
andres@anarazel.de, robertmhaas@gmail.com, tomas.vondra@2ndquadrant.com,
pgsql-hackers@postgresql.org
Date: 2020-10-22T06:14:32Z
Lists: pgsql-hackers
At Thu, 22 Oct 2020 01:33:31 +0000, "tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com> wrote in > From: Jamison, Kirk/ジャミソン カーク <k.jamison@fujitsu.com> > > The table below shows the vacuum execution time for non-recovery case. > > I've also subtracted the execution time when VACUUM (truncate off) is set. > > > > [NON-RECOVERY CASE - VACUUM execution Time in seconds] > (snip) > > | 100GB | 65.456 | 1.795 | -3546.57% | > > So, the full shared buffer scan for 10,000 relations took about as long as 63 seconds (= 6.3 ms per relation). It's nice to shorten this long time. I'm not sure about the exact steps of the test, but it can be expected if we have many small relations to truncate. Currently BUF_DROP_FULL_SCAN_THRESHOLD is set to Nbuffers / 512, which is quite arbitrary that comes from a wild guess. Perhaps we need to run benchmarks that drops one relation with several different ratios between the number of buffers to-be-dropped and Nbuffers, and preferably both on spinning rust and SSD. regards. -- Kyotaro Horiguchi NTT Open Source Software Center
Commits
-
Fix size overflow in calculation introduced by commits d6ad34f3 and bea449c6.
- 519e4c9ee21a 14.0 landed
-
Optimize DropRelFileNodesAllBuffers() for recovery.
- bea449c635c0 14.0 landed
-
Optimize DropRelFileNodeBuffers() for recovery.
- d6ad34f3410f 14.0 landed
-
Cache smgrnblocks() results in recovery.
- c5315f4f4484 14.0 cited
-
Add a check to prevent overwriting valid data if smgrnblocks() gives a
- ffae5cc5a602 8.2.0 cited