Re: [Patch] Optimize dropping of relation buffers using dlist

Thomas Munro <thomas.munro@gmail.com>

From: Thomas Munro <thomas.munro@gmail.com>
To: "tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com>
Cc: Kyotaro Horiguchi <horikyota.ntt@gmail.com>, "k.jamison@fujitsu.com" <k.jamison@fujitsu.com>, Amit Kapila <amit.kapila16@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>, Andres Freund <andres@anarazel.de>, Robert Haas <robertmhaas@gmail.com>, Tomas Vondra <tomas.vondra@2ndquadrant.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2020-10-22T21:45:05Z
Lists: pgsql-hackers
On Thu, Oct 22, 2020 at 8:32 PM tsunakawa.takay@fujitsu.com
<tsunakawa.takay@fujitsu.com> wrote:
> I'm probably being silly, but can't we avoid the problem by using fstat() instead of lseek(SEEK_END)?  Would they return the same value from the i-node?

Amazingly, st_size can disagree with SEEK_END when using the Linux NFS
client, but its behaviour is worse.  Here's a sequence from a Linux
NFS client talking to a Linux NFS server with no free space.  This
time, I also replaced the fsync() with sleep(60), just to make it
clear that SEEK_END offset can move at any time due to asynchronous
activity in kernel threads:

lseek(..., SEEK_END) = 9670656
fstat(...) = 0, st_size = 9670656

write(...)  = 8192
lseek(..., SEEK_END) = 9678848
fstat(...) = 0, st_size = 9670656 (*1)

sleep(...) = 0

lseek(..., SEEK_END) = 9670656 (*2)
fstat(...) = 0, st_size = 9670656

fsync(...) = -1
lseek(..., SEEK_END) = 9670656
fstat(...) = 0, st_size = 9670656
fsync(...) = 0

However, I'm not entirely sure which phenomena visible here to blame
on which subsystems, and therefore which things to expect on local
filesystems, or on other operating systems.  I can say that with a
FreeBSD NFS client and the same Linux NFS server, I don't see
phenomenon *1 (unsurprising) but I do see phenomenon *2 (surprising to
me).

> Or, can't we just try to do BufTableLookup() one block after what smgrnblocks() returns?

Unfortunately the problem isn't limited to one block.



Commits

  1. Fix size overflow in calculation introduced by commits d6ad34f3 and bea449c6.

  2. Optimize DropRelFileNodesAllBuffers() for recovery.

  3. Optimize DropRelFileNodeBuffers() for recovery.

  4. Cache smgrnblocks() results in recovery.

  5. Add a check to prevent overwriting valid data if smgrnblocks() gives a