Re: [Patch] Optimize dropping of relation buffers using dlist
Kyotaro Horiguchi <horikyota.ntt@gmail.com>
From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
To: amit.kapila16@gmail.com
Cc: k.jamison@fujitsu.com, tanghy.fnst@cn.fujitsu.com,
tsunakawa.takay@fujitsu.com, andres@anarazel.de, tgl@sss.pgh.pa.us,
thomas.munro@gmail.com, robertmhaas@gmail.com,
tomas.vondra@2ndquadrant.com, pgsql-hackers@postgresql.org,
jankirk.jamison@gmail.com
Date: 2021-01-13T02:09:14Z
Lists: pgsql-hackers
At Tue, 12 Jan 2021 08:49:53 +0530, Amit Kapila <amit.kapila16@gmail.com> wrote in > On Fri, Jan 8, 2021 at 7:03 AM Kyotaro Horiguchi > <horikyota.ntt@gmail.com> wrote: > > > > At Thu, 7 Jan 2021 09:25:22 +0000, "k.jamison@fujitsu.com" <k.jamison@fujitsu.com> wrote in: > > > > Thanks for the detailed tests. NBuffers/32 seems like an appropriate > > > > value for the threshold based on these results. I would like to > > > > slightly modify part of the commit message in the first patch as below > > > > [1], otherwise, I am fine with the changes. Unless you or anyone else > > > > has any more comments, I am planning to push the 0001 and 0002 > > > > sometime next week. > > > > > > > > [1] > > > > "The recovery path of DropRelFileNodeBuffers() is optimized so that > > > > scanning of the whole buffer pool can be avoided when the number of > > > > blocks to be truncated in a relation is below a certain threshold. For > > > > such cases, we find the buffers by doing lookups in BufMapping table. > > > > This improves the performance by more than 100 times in many cases > > > > when several small tables (tested with 1000 relations) are truncated > > > > and where the server is configured with a large value of shared > > > > buffers (greater than 100GB)." > > > > > > Thank you for taking a look at the results of the tests. And it's also > > > consistent with the results from Tang too. > > > The commit message LGTM. > > > > +1. > > > > I have pushed the 0001. Thank you for commiting this. regards. -- Kyotaro Horiguchi NTT Open Source Software Center
Commits
-
Fix size overflow in calculation introduced by commits d6ad34f3 and bea449c6.
- 519e4c9ee21a 14.0 landed
-
Optimize DropRelFileNodesAllBuffers() for recovery.
- bea449c635c0 14.0 landed
-
Optimize DropRelFileNodeBuffers() for recovery.
- d6ad34f3410f 14.0 landed
-
Cache smgrnblocks() results in recovery.
- c5315f4f4484 14.0 cited
-
Add a check to prevent overwriting valid data if smgrnblocks() gives a
- ffae5cc5a602 8.2.0 cited