Re: [Patch] Optimize dropping of relation buffers using dlist
Kyotaro Horiguchi <horikyota.ntt@gmail.com>
From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
To: amit.kapila16@gmail.com
Cc: k.jamison@fujitsu.com, tsunakawa.takay@fujitsu.com, tgl@sss.pgh.pa.us,
andres@anarazel.de, robertmhaas@gmail.com, tomas.vondra@2ndquadrant.com,
pgsql-hackers@postgresql.org
Date: 2020-11-10T03:26:54Z
Lists: pgsql-hackers
At Tue, 10 Nov 2020 08:33:26 +0530, Amit Kapila <amit.kapila16@gmail.com> wrote in > On Tue, Nov 10, 2020 at 8:19 AM k.jamison@fujitsu.com > <k.jamison@fujitsu.com> wrote: > > > > I repeated the recovery performance test for vacuum. (I made a mistake previously in NBuffers/128) > > The 3 kinds of thresholds are almost equally performant. I chose NBuffers/256 for this patch. > > > > | s_b | Master | NBuffers/512 | NBuffers/256 | NBuffers/128 | > > |-------|--------|--------------|--------------|--------------| > > | 128MB | 1.006 | 1.007 | 1.007 | 1.007 | > > | 1GB | 0.706 | 0.606 | 0.606 | 0.606 | > > | 20GB | 1.907 | 0.606 | 0.606 | 0.606 | > > | 100GB | 7.013 | 0.706 | 0.606 | 0.606 | > > > > I think this data is not very clear. What is the unit of time? What is > the size of the relation used for the test? Did the test use an > optimized path for all cases? If at 128MB, there is no performance > gain, can we consider the size of shared buffers as 256MB as well for > the threshold? In the previous testing, it was shown as: Recovery Time (in seconds) | s_b | master | patched | %reg | |-------|--------|---------|--------| | 128MB | 3.043 | 2.977 | -2.22% | | 1GB | 3.417 | 3.41 | -0.21% | | 20GB | 20.597 | 2.409 | -755% | | 100GB | 66.862 | 2.409 | -2676% | So... The numbers seems to be in seconds, but the master gets about 10 times faster than this result for uncertain reasons. It seems that the result contains something different from the difference by this patch as a larger part. regards. -- Kyotaro Horiguchi NTT Open Source Software Center
Commits
-
Fix size overflow in calculation introduced by commits d6ad34f3 and bea449c6.
- 519e4c9ee21a 14.0 landed
-
Optimize DropRelFileNodesAllBuffers() for recovery.
- bea449c635c0 14.0 landed
-
Optimize DropRelFileNodeBuffers() for recovery.
- d6ad34f3410f 14.0 landed
-
Cache smgrnblocks() results in recovery.
- c5315f4f4484 14.0 cited
-
Add a check to prevent overwriting valid data if smgrnblocks() gives a
- ffae5cc5a602 8.2.0 cited