RE: [Patch] Optimize dropping of relation buffers using dlist

tsunakawa.takay@fujitsu.com <tsunakawa.takay@fujitsu.com>

From: "tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com>
To: "k.jamison@fujitsu.com" <k.jamison@fujitsu.com>
Cc: 'Kyotaro Horiguchi' <horikyota.ntt@gmail.com>, "amit.kapila16@gmail.com" <amit.kapila16@gmail.com>, "tgl@sss.pgh.pa.us" <tgl@sss.pgh.pa.us>, "andres@anarazel.de" <andres@anarazel.de>, "robertmhaas@gmail.com" <robertmhaas@gmail.com>, "tomas.vondra@2ndquadrant.com" <tomas.vondra@2ndquadrant.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2020-11-10T06:09:56Z
Lists: pgsql-hackers
From: Jamison, Kirk/ジャミソン カーク <k.jamison@fujitsu.com>
> So I proceeded to update the patches using the "cached" parameter and
> updated the corresponding comments to it in 0002.

OK, I'm in favor of the name "cached" now, although I first agreed with Horiguchi-san in that it's better to use a name that represents the nature (accurate) of information rather than the implementation (cached).  Having a second thought, since smgr is a component that manages relation files on storage (file system), lseek(SEEK_END) is the accurate value for smgr.  The cached value holds a possibly stale size up to which the relation has extended.


The patch looks almost good except for the minor ones:

(1)
+extern BlockNumber smgrnblocks(SMgrRelation reln, ForkNumber forknum,
+							   bool *accurate);

It's still accurate here.


(2)
+ *		the buffer pool is sequentially scanned. Since buffers must not be
+ *		left behind, the latter way is executed unless the sizes of all the
+ *		involved forks are already cached. See smgrnblocks() for more details.
+ *		This is only called in recovery when the block count of any fork is
+ *		cached and the total number of to-be-invalidated blocks per relation

count of any fork is
-> counts of all forks are


(3)
In 0004, I thought you would add the invalidated block counts of all relations to determine if the optimization is done, as Horiguchi-san suggested.  But I find the current patch okay too.


Regards
Takayuki Tsunakawa




Commits

  1. Fix size overflow in calculation introduced by commits d6ad34f3 and bea449c6.

  2. Optimize DropRelFileNodesAllBuffers() for recovery.

  3. Optimize DropRelFileNodeBuffers() for recovery.

  4. Cache smgrnblocks() results in recovery.

  5. Add a check to prevent overwriting valid data if smgrnblocks() gives a