Horton

Re: [Patch] Optimize dropping of relation buffers using dlist

Kyotaro Horiguchi <horikyota.ntt@gmail.com>

View thread View on postgr.es
From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
To: tsunakawa.takay@fujitsu.com
Cc: thomas.munro@gmail.com, amit.kapila16@gmail.com, k.jamison@fujitsu.com, tanghy.fnst@fujitsu.com, andres@anarazel.de, tgl@sss.pgh.pa.us, robertmhaas@gmail.com, tomas.vondra@2ndquadrant.com, pgsql-hackers@postgresql.org, jankirk.jamison@gmail.com
Date: 2021-03-12T06:10:07Z
Lists: pgsql-hackers 
At Fri, 12 Mar 2021 05:26:02 +0000, "tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com> wrote in 
> From: Thomas Munro <thomas.munro@gmail.com>
> > > uint64
> > 
> > +1
> 
> Thank you, the patch is attached (we tend to forget how large our world is... 64-bit)  We're sorry to cause you trouble.

BUF_DROP_FULL_SCAN_THRESHOLD cannot be larger than the size of int
since Nbuffer is an int. but nBlocksToInvalidate being uint32 looks
somewhat too tight. So +1 for changing it to uint64.

We need fill all block[file][fork] array in DropRelFileNodesAllBuffers
so we cannot bailing out from the counting loop. We could do that
DropRelFileNodesAllBuffers but that doesn't seem effective so much.

So I vote for uint64 and not bailing out.

About the patch, it would be better to change the type of
BUF_DROP_FULL_SCAN_THRESHOLD to uint64, even though the current value
doesn't harm.

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center

Commits

  1. Fix size overflow in calculation introduced by commits d6ad34f3 and bea449c6.

  2. Optimize DropRelFileNodesAllBuffers() for recovery.

  3. Optimize DropRelFileNodeBuffers() for recovery.

  4. Cache smgrnblocks() results in recovery.

  5. Add a check to prevent overwriting valid data if smgrnblocks() gives a