RE: [Patch] Optimize dropping of relation buffers using dlist
k.jamison@fujitsu.com <k.jamison@fujitsu.com>
From: "k.jamison@fujitsu.com" <k.jamison@fujitsu.com>
To: 'Amit Kapila' <amit.kapila16@gmail.com>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Cc: "tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com>, Andres Freund <andres@anarazel.de>, Tom Lane <tgl@sss.pgh.pa.us>, Thomas Munro <thomas.munro@gmail.com>, Robert Haas <robertmhaas@gmail.com>, Tomas Vondra <tomas.vondra@2ndquadrant.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2020-12-23T12:57:24Z
Lists: pgsql-hackers
Attachments
- v37-0001-Prevent-invalidating-blocks-in-smgrextend-during.patch (application/octet-stream) patch v37-0001
- v37-0002-Optimize-DropRelFileNodeBuffers-for-recovery.patch (application/octet-stream) patch v37-0002
- v37-0003-Optimize-DropRelFileNodesAllBuffers-in-recovery.patch (application/octet-stream) patch v37-0003
On Wed, December 23, 2020 5:57 PM (GMT+9), Amit Kapila wrote: > > > > At Wed, 23 Dec 2020 04:22:19 +0000, "tsunakawa.takay@fujitsu.com" > > <tsunakawa.takay@fujitsu.com> wrote in > > > From: Amit Kapila <amit.kapila16@gmail.com> > > > > + /* Get the number of blocks for a relation's fork */ block[i][j] > > > > + = smgrnblocks(smgr_reln[i], j, &cached); > > > > + > > > > + if (!cached) > > > > + goto buffer_full_scan; > > > > > > > > Why do we need to use goto here? We can simply break from the loop > > > > and then check if (cached && nBlocksToInvalidate < > > > > BUF_DROP_FULL_SCAN_THRESHOLD). I think we should try to avoid > goto > > > > if possible without much complexity. > > > > > > That's because two for loops are nested -- breaking there only exits > > > the inner loop. (I thought the same as you at first... And I > > > understand some people have alergy to goto, I think modest use of > > > goto makes the code readable.) > > > > I don't strongly oppose to goto's but in this case the outer loop can > > break on the same condition with the inner loop, since cached is true > > whenever the inner loop runs to the end. It is needed to initialize > > the variable cache with true, instead of false, though. > > > > +1. I think it is better to avoid goto here as it can be done without > introducing any complexity or making code any less readable. I also do not mind, so I have removed the goto and followed the advice of all reviewers. It works fine in the latest attached patch 0003. Attached herewith are the sets of patches. 0002 & 0003 have the following changes: 1. I have removed the modifications in smgrnblocks(). So the modifications of other functions that uses smgrnblocks() in the previous patch versions were also reverted. 2. Introduced a new API smgrnblocks_cached() instead which returns either a cached size for the specified fork or an InvalidBlockNumber. Since InvalidBlockNumber is used, I think it is logical not to use the additional boolean parameter "cached" in the function as it will be redundant. Although in 0003, I only used the "cached" as a Boolean variable to do the trick of not using goto. This function is called both in DropRelFileNodeBuffers() and DropRelFileNodesAllBuffers(). 3. Modified some minor comments from the patch and commit logs. It compiles. Passes the regression tests too. Your feedbacks are definitely welcome. Regards, Kirk Jamison
Commits
-
Fix size overflow in calculation introduced by commits d6ad34f3 and bea449c6.
- 519e4c9ee21a 14.0 landed
-
Optimize DropRelFileNodesAllBuffers() for recovery.
- bea449c635c0 14.0 landed
-
Optimize DropRelFileNodeBuffers() for recovery.
- d6ad34f3410f 14.0 landed
-
Cache smgrnblocks() results in recovery.
- c5315f4f4484 14.0 cited
-
Add a check to prevent overwriting valid data if smgrnblocks() gives a
- ffae5cc5a602 8.2.0 cited