RE: [Patch] Optimize dropping of relation buffers using dlist
k.jamison@fujitsu.com <k.jamison@fujitsu.com>
From: "k.jamison@fujitsu.com" <k.jamison@fujitsu.com>
To: 'Amit Kapila' <amit.kapila16@gmail.com>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Cc: "tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com>, Andres Freund <andres@anarazel.de>, Tom Lane <tgl@sss.pgh.pa.us>, Thomas Munro <thomas.munro@gmail.com>, Robert Haas <robertmhaas@gmail.com>, Tomas Vondra <tomas.vondra@2ndquadrant.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2020-12-10T08:09:55Z
Lists: pgsql-hackers
Attachments
- v34-0001-Prevent-invalidating-blocks-in-smgrextend-during.patch (application/octet-stream) patch v34-0001
- v34-0002-Add-bool-param-in-smgrnblocks-for-cached-blocks.patch (application/octet-stream) patch v34-0002
- v34-0003-Optimize-DropRelFileNodeBuffers-during-recovery.patch (application/octet-stream) patch v34-0003
- v34-0004-Optimize-DropRelFileNodesAllBuffers-in-recovery.patch (application/octet-stream) patch v34-0004
On Thursday, December 10, 2020 12:27 PM, Amit Kapila wrote: > On Thu, Dec 10, 2020 at 7:11 AM Kyotaro Horiguchi > <horikyota.ntt@gmail.com> wrote: > > > > At Wed, 9 Dec 2020 16:27:30 +0530, Amit Kapila > > <amit.kapila16@gmail.com> wrote in > > > On Wed, Dec 9, 2020 at 6:32 AM Kyotaro Horiguchi > > > > Mmm. At least btree doesn't need to call smgrnblocks except at > > > > expansion, so we cannot get to the optimized path in major cases > > > > of truncation involving btree (and/or maybe other indexes). > > > > > > > > > > AFAICS, btree insert should call smgrnblocks via > > > > btree_xlog_insert->XLogReadBufferForRedo->XLogReadBufferForRedoExte > nded->XLogReadBufferExtended->smgrnblocks. > > > Similarly delete should also call smgrnblocks. Can you be bit more > > > specific related to the btree case you have in mind? > > > > Oh, sorry. I wrongly looked to non-recovery path. smgrnblocks is > > called during buffer loading while recovery. So, smgrnblock is called > > for indexes if any update happens on the heap relation. > > > > Okay, so this means that we can get the benefit of optimization in many cases > in the Truncate code path as well even if we use 'cached' > flag? If so, then I would prefer to keep the code consistent for both vacuum > and truncate recovery code path. Yes, I have tested that optimization works for index relations. I have attached the V34, following the conditions that we use "cached" flag for both DropRelFileNodesBuffers() and DropRelFileNodesBuffers() for consistency. I added comment in 0004 the limitation of optimization when there are TOAST relations that use NON-PLAIN strategy. i.e. The optimization works if the data types used are integers, OID, bytea, etc. But for TOAST-able data types like text, the optimization will be skipped and force a full scan during recovery. Regards, Kirk Jamison
Commits
-
Fix size overflow in calculation introduced by commits d6ad34f3 and bea449c6.
- 519e4c9ee21a 14.0 landed
-
Optimize DropRelFileNodesAllBuffers() for recovery.
- bea449c635c0 14.0 landed
-
Optimize DropRelFileNodeBuffers() for recovery.
- d6ad34f3410f 14.0 landed
-
Cache smgrnblocks() results in recovery.
- c5315f4f4484 14.0 cited
-
Add a check to prevent overwriting valid data if smgrnblocks() gives a
- ffae5cc5a602 8.2.0 cited