Re: [Patch] Optimize dropping of relation buffers using dlist

Amit Kapila <amit.kapila16@gmail.com>

From: Amit Kapila <amit.kapila16@gmail.com>
To: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Cc: "k.jamison@fujitsu.com" <k.jamison@fujitsu.com>, Andres Freund <andres@anarazel.de>, Tom Lane <tgl@sss.pgh.pa.us>, Robert Haas <robertmhaas@gmail.com>, Konstantin Knizhnik <knizhnik@garret.ru>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2020-08-07T03:49:42Z
Lists: pgsql-hackers
On Fri, Aug 7, 2020 at 3:03 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
>
> >But I understand the sentiment on the added overhead on BufferAlloc.
> >Regarding the case where the patch would potentially affect workloads
> >that fit into RAM but not into shared buffers, could one of Andres'
> >suggested idea/s above address that, in addition to this patch's
> >possible shared invalidation fix? Could that settle the added overhead
> >in BufferAlloc() as temporary fix?
>
> Not sure.
>
> >Thomas Munro is also working on caching relation sizes [1], maybe that
> >way we could get the latest known relation size. Currently, it's
> >possible only during recovery in smgrnblocks.
>
> It's not clear to me how would knowing the relation size help reducing
> the overhead of this patch?
>

AFAICU the idea is to directly call BufTableLookup (similar to how we
do in BufferAlloc) to find the buf_id in function
DropRelFileNodeBuffers and then invalidate the required buffers.  And,
we need to do this when the size of the relation is less than some
threshold. So, I think the crux would be to reliably get the number of
blocks information. So, probably relation size cache stuff might help.

-- 
With Regards,
Amit Kapila.



Commits

  1. Fix size overflow in calculation introduced by commits d6ad34f3 and bea449c6.

  2. Optimize DropRelFileNodesAllBuffers() for recovery.

  3. Optimize DropRelFileNodeBuffers() for recovery.

  4. Cache smgrnblocks() results in recovery.

  5. Add a check to prevent overwriting valid data if smgrnblocks() gives a