Horton

Re: [Patch] Optimize dropping of relation buffers using dlist

Amit Kapila <amit.kapila16@gmail.com>

View thread View on postgr.es
From: Amit Kapila <amit.kapila16@gmail.com>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: "k.jamison@fujitsu.com" <k.jamison@fujitsu.com>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>, "tanghy.fnst@cn.fujitsu.com" <tanghy.fnst@cn.fujitsu.com>, "tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com>, Andres Freund <andres@anarazel.de>, Tom Lane <tgl@sss.pgh.pa.us>, Robert Haas <robertmhaas@gmail.com>, Tomas Vondra <tomas.vondra@2ndquadrant.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, "jankirk.jamison@gmail.com" <jankirk.jamison@gmail.com>
Date: 2021-03-12T04:20:27Z
Lists: pgsql-hackers 
On Fri, Mar 12, 2021 at 4:58 AM Thomas Munro <thomas.munro@gmail.com> wrote:
>
> While rebasing CF #2933 (which drops the _cached stuff and makes this
> optimisation always available, woo), I happened to notice that we're
> summing the size of many relations and forks into a variable
> nBlocksToInvalidate of type BlockNumber.  That could overflow.
>

I also think so. I think we have two ways to address that: (a) check
immediately after each time we add blocks to nBlocksToInvalidate to
see if it crosses the threshold value BUF_DROP_FULL_SCAN_THRESHOLD and
if so, then just break the loop; (b) change the variable type to
uint64.

Any better ideas?

-- 
With Regards,
Amit Kapila.

Commits

  1. Fix size overflow in calculation introduced by commits d6ad34f3 and bea449c6.

  2. Optimize DropRelFileNodesAllBuffers() for recovery.

  3. Optimize DropRelFileNodeBuffers() for recovery.

  4. Cache smgrnblocks() results in recovery.

  5. Add a check to prevent overwriting valid data if smgrnblocks() gives a