Re: [Patch] Optimize dropping of relation buffers using dlist
Thomas Munro <thomas.munro@gmail.com>
From: Thomas Munro <thomas.munro@gmail.com>
To: "tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com>
Cc: Kyotaro Horiguchi <horikyota.ntt@gmail.com>, "k.jamison@fujitsu.com" <k.jamison@fujitsu.com>,
Amit Kapila <amit.kapila16@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>, Andres Freund <andres@anarazel.de>, Robert Haas <robertmhaas@gmail.com>,
Tomas Vondra <tomas.vondra@2ndquadrant.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2020-10-22T21:45:05Z
Lists: pgsql-hackers
On Thu, Oct 22, 2020 at 8:32 PM tsunakawa.takay@fujitsu.com <tsunakawa.takay@fujitsu.com> wrote: > I'm probably being silly, but can't we avoid the problem by using fstat() instead of lseek(SEEK_END)? Would they return the same value from the i-node? Amazingly, st_size can disagree with SEEK_END when using the Linux NFS client, but its behaviour is worse. Here's a sequence from a Linux NFS client talking to a Linux NFS server with no free space. This time, I also replaced the fsync() with sleep(60), just to make it clear that SEEK_END offset can move at any time due to asynchronous activity in kernel threads: lseek(..., SEEK_END) = 9670656 fstat(...) = 0, st_size = 9670656 write(...) = 8192 lseek(..., SEEK_END) = 9678848 fstat(...) = 0, st_size = 9670656 (*1) sleep(...) = 0 lseek(..., SEEK_END) = 9670656 (*2) fstat(...) = 0, st_size = 9670656 fsync(...) = -1 lseek(..., SEEK_END) = 9670656 fstat(...) = 0, st_size = 9670656 fsync(...) = 0 However, I'm not entirely sure which phenomena visible here to blame on which subsystems, and therefore which things to expect on local filesystems, or on other operating systems. I can say that with a FreeBSD NFS client and the same Linux NFS server, I don't see phenomenon *1 (unsurprising) but I do see phenomenon *2 (surprising to me). > Or, can't we just try to do BufTableLookup() one block after what smgrnblocks() returns? Unfortunately the problem isn't limited to one block.
Commits
-
Fix size overflow in calculation introduced by commits d6ad34f3 and bea449c6.
- 519e4c9ee21a 14.0 landed
-
Optimize DropRelFileNodesAllBuffers() for recovery.
- bea449c635c0 14.0 landed
-
Optimize DropRelFileNodeBuffers() for recovery.
- d6ad34f3410f 14.0 landed
-
Cache smgrnblocks() results in recovery.
- c5315f4f4484 14.0 cited
-
Add a check to prevent overwriting valid data if smgrnblocks() gives a
- ffae5cc5a602 8.2.0 cited