Re: [Patch] Optimize dropping of relation buffers using dlist
Amit Kapila <amit.kapila16@gmail.com>
From: Amit Kapila <amit.kapila16@gmail.com>
To: "Tang, Haiying" <tanghy.fnst@cn.fujitsu.com>
Cc: "Tsunakawa, Takayuki" <tsunakawa.takay@fujitsu.com>, "Jamison,
Kirk" <k.jamison@fujitsu.com>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>,
Andres Freund <andres@anarazel.de>, Tom Lane <tgl@sss.pgh.pa.us>, Thomas Munro <thomas.munro@gmail.com>, Robert Haas <robertmhaas@gmail.com>,
Tomas Vondra <tomas.vondra@2ndquadrant.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2020-12-25T04:31:04Z
Lists: pgsql-hackers
On Fri, Dec 25, 2020 at 9:28 AM Tang, Haiying <tanghy.fnst@cn.fujitsu.com> wrote: > > Hi Amit, > > >But how can we conclude NBuffers/128 is the maximum relation size? > >Because the maximum size would be where the performance is worse than > >the master, no? I guess we need to try by NBuffers/64, NBuffers/32, > >.... till we get the threshold where master performs better. > > You are right, we should keep on testing until no optimization. > > >I think we should find a better way to display these numbers because in > >cases like where master takes 537.978s and patch takes 3.815s > > Yeah, I think we can change the %reg formula from (patched- master)/ master to (patched- master)/ patched. > > >Table size should be more than 8k to get all this data because 8k means > >just one block. I guess either it is a typo or some other mistake. > > 8k here is the relation size, not data size. > For example, when I tested recovery performance of 400M relation size, I used 51200 tables(8k per table). > Please let me know if you think this is not appropriate. > I think one table with a varying amount of data is sufficient for the vacuum test. I think with more number of tables there is a greater chance of variation. We have previously used multiple tables in one of the tests because of the Truncate operation (which uses DropRelFileNodesAllBuffers that takes multiple relations as input) and that is not true for Vacuum operation which I suppose you are testing here. -- With Regards, Amit Kapila.
Commits
-
Fix size overflow in calculation introduced by commits d6ad34f3 and bea449c6.
- 519e4c9ee21a 14.0 landed
-
Optimize DropRelFileNodesAllBuffers() for recovery.
- bea449c635c0 14.0 landed
-
Optimize DropRelFileNodeBuffers() for recovery.
- d6ad34f3410f 14.0 landed
-
Cache smgrnblocks() results in recovery.
- c5315f4f4484 14.0 cited
-
Add a check to prevent overwriting valid data if smgrnblocks() gives a
- ffae5cc5a602 8.2.0 cited