Re: [Patch] Optimize dropping of relation buffers using dlist
Konstantin Knizhnik <k.knizhnik@postgrespro.ru>
From: Konstantin Knizhnik <k.knizhnik@postgrespro.ru>
To: pgsql-hackers@lists.postgresql.org
Date: 2020-07-29T07:54:45Z
Lists: pgsql-hackers
On 17.06.2020 09:14, k.jamison@fujitsu.com wrote: > Hi, > > Since the last posted version of the patch fails, attached is a rebased version. > Written upthread were performance results and some benefits and challenges. > I'd appreciate your feedback/comments. > > Regards, > Kirk Jamison As far as i understand this patch can provide significant improvement of performance only in case of recovery of truncates of large number of tables. You have added shared hash of relation buffers and certainly if adds some extra overhead. According to your latest results this overhead is quite small. But it will be hard to prove that there will be no noticeable regression at some workloads. I wonder if you have considered case of local hash (maintained only during recovery)? If there is after-crash recovery, then there will be no concurrent access to shared buffers and this hash will be up-to-date. in case of hot-standby replica we can use some simple invalidation (just one flag or counter which indicates that buffer cache was updated). This hash also can be constructed on demand when DropRelFileNodeBuffers is called first time (so w have to scan all buffers once, but subsequent drop operation will be fast. i have not thought much about it, but it seems to me that as far as this problem only affects recovery, we do not need shared hash for it.
Commits
-
Fix size overflow in calculation introduced by commits d6ad34f3 and bea449c6.
- 519e4c9ee21a 14.0 landed
-
Optimize DropRelFileNodesAllBuffers() for recovery.
- bea449c635c0 14.0 landed
-
Optimize DropRelFileNodeBuffers() for recovery.
- d6ad34f3410f 14.0 landed
-
Cache smgrnblocks() results in recovery.
- c5315f4f4484 14.0 cited
-
Add a check to prevent overwriting valid data if smgrnblocks() gives a
- ffae5cc5a602 8.2.0 cited