Re: [Patch] Optimize dropping of relation buffers using dlist
Amit Kapila <amit.kapila16@gmail.com>
From: Amit Kapila <amit.kapila16@gmail.com>
To: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Cc: "k.jamison@fujitsu.com" <k.jamison@fujitsu.com>,
Andres Freund <andres@anarazel.de>, Tom Lane <tgl@sss.pgh.pa.us>, Robert Haas <robertmhaas@gmail.com>, Konstantin Knizhnik <knizhnik@garret.ru>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2020-08-07T03:49:42Z
Lists: pgsql-hackers
On Fri, Aug 7, 2020 at 3:03 AM Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote: > > >But I understand the sentiment on the added overhead on BufferAlloc. > >Regarding the case where the patch would potentially affect workloads > >that fit into RAM but not into shared buffers, could one of Andres' > >suggested idea/s above address that, in addition to this patch's > >possible shared invalidation fix? Could that settle the added overhead > >in BufferAlloc() as temporary fix? > > Not sure. > > >Thomas Munro is also working on caching relation sizes [1], maybe that > >way we could get the latest known relation size. Currently, it's > >possible only during recovery in smgrnblocks. > > It's not clear to me how would knowing the relation size help reducing > the overhead of this patch? > AFAICU the idea is to directly call BufTableLookup (similar to how we do in BufferAlloc) to find the buf_id in function DropRelFileNodeBuffers and then invalidate the required buffers. And, we need to do this when the size of the relation is less than some threshold. So, I think the crux would be to reliably get the number of blocks information. So, probably relation size cache stuff might help. -- With Regards, Amit Kapila.
Commits
-
Fix size overflow in calculation introduced by commits d6ad34f3 and bea449c6.
- 519e4c9ee21a 14.0 landed
-
Optimize DropRelFileNodesAllBuffers() for recovery.
- bea449c635c0 14.0 landed
-
Optimize DropRelFileNodeBuffers() for recovery.
- d6ad34f3410f 14.0 landed
-
Cache smgrnblocks() results in recovery.
- c5315f4f4484 14.0 cited
-
Add a check to prevent overwriting valid data if smgrnblocks() gives a
- ffae5cc5a602 8.2.0 cited