Horton

Re: Cache relation sizes?

Thomas Munro <thomas.munro@gmail.com>

View thread View on postgr.es
From: Thomas Munro <thomas.munro@gmail.com>
To: Andres Freund <andres@anarazel.de>
Cc: Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>, "Jamison, Kirk" <k.jamison@jp.fujitsu.com>, "Tsunakawa, Takayuki" <tsunakawa.takay@jp.fujitsu.com>, "Ideriha, Takeshi" <ideriha.takeshi@jp.fujitsu.com>, David Rowley <david.rowley@2ndquadrant.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2020-02-14T00:50:27Z
Lists: pgsql-hackers 
On Thu, Feb 13, 2020 at 7:18 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> ... (1) I'm pretty sure some systems would not be happy
> about that (see claims in this thread) ...

I poked a couple of people off-list and learned that, although the
Linux and FreeBSD systems I tried could do a million lseek(SEEK_END)
calls in 60-100ms, a couple of different Windows systems took between
1.1 and 3.4 seconds (worse times when running as non-administrator),
which seems to be clearly in the territory that can put a dent in your
recovery speeds on that OS.  I also learned that GetFileSizeEx() is
"only" about twice as fast, which is useful information for that other
thread about which syscall to use for this, but it's kind of
irrelevant this thread about how we can get rid of these crazy
syscalls altogether.

Commits

  1. Optimize DropRelFileNodesAllBuffers() for recovery.

  2. Cache smgrnblocks() results in recovery.

  3. Use pg_pread() and pg_pwrite() for data files and WAL.

  4. Add a check to prevent overwriting valid data if smgrnblocks() gives a