RE: [Patch] Optimize dropping of relation buffers using dlist
tsunakawa.takay@fujitsu.com <tsunakawa.takay@fujitsu.com>
From: "tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com>
To: 'Kyotaro Horiguchi' <horikyota.ntt@gmail.com>, "k.jamison@fujitsu.com" <k.jamison@fujitsu.com>
Cc: "andres@anarazel.de" <andres@anarazel.de>, "amit.kapila16@gmail.com" <amit.kapila16@gmail.com>, "tgl@sss.pgh.pa.us" <tgl@sss.pgh.pa.us>, "thomas.munro@gmail.com" <thomas.munro@gmail.com>, "robertmhaas@gmail.com" <robertmhaas@gmail.com>, "tomas.vondra@2ndquadrant.com" <tomas.vondra@2ndquadrant.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2020-12-01T02:46:07Z
Lists: pgsql-hackers
From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
> We are relying on the "fact" that the first lseek() call of a
> (startup) process tells the truth. We added an assertion so that we
> make sure that the cached value won't be cleared during recovery. A
> possible remaining danger would be closing of an smgr object of a live
> relation just after a file extension failure. I think we are thinking
> that that doesn't happen during recovery. Although it seems to me
> true, I'm not confident.
>
> If that's true, we don't even need to look at the "cached" flag at all
> and always be able to rely on the returned value from msgrnblocks()
> during recovery. Otherwise, we need to avoid the danger situation.
Hmm, I've gotten to think that smgrnblocks() doesn't need the cached parameter, too. DropRel*Buffers() can just check InRecovery. Regarding the only concern about smgrclose() by startup process, I was afraid of the cache invalidation by CacheInvalidateSmgr(), but startup process doesn't receive shared inval messages. So, it doesn't call smgrclose*() due to shared cache invalidation.
[InitRecoveryTransactionEnvironment()]
/* Initialize shared invalidation management for Startup process, being
* Initialize shared invalidation management for Startup process, being
* careful to register ourselves as a sendOnly process so we don't need to
* read messages, nor will we get signaled when the queue starts filling
* up.
*/
SharedInvalBackendInit(true);
Kirk-san,
Can you check to see if smgrclose() and its friends are not called during recovery using the regression test?
Regards
Takayuki Tsunakawa
Commits
-
Fix size overflow in calculation introduced by commits d6ad34f3 and bea449c6.
- 519e4c9ee21a 14.0 landed
-
Optimize DropRelFileNodesAllBuffers() for recovery.
- bea449c635c0 14.0 landed
-
Optimize DropRelFileNodeBuffers() for recovery.
- d6ad34f3410f 14.0 landed
-
Cache smgrnblocks() results in recovery.
- c5315f4f4484 14.0 cited
-
Add a check to prevent overwriting valid data if smgrnblocks() gives a
- ffae5cc5a602 8.2.0 cited