Horton

Re: Race between KeepFileRestoredFromArchive() and restartpoint

Don Seiler <don@seiler.us>

View thread View on postgr.es
From: Don Seiler <don@seiler.us>
To: David Steele <david@pgmasters.net>
Cc: Noah Misch <noah@leadboat.com>, pgsql-hackers@postgresql.org, Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Date: 2022-08-02T21:03:42Z
Lists: pgsql-hackers 
On Tue, Aug 2, 2022 at 10:01 AM David Steele <david@pgmasters.net> wrote:

>
> > That makes sense.  Each iteration of the restartpoint recycle loop has a
> 1/N
> > chance of failing.  Recovery adds >N files between restartpoints.
> Hence, the
> > WAL directory grows without bound.  Is that roughly the theory in mind?
>
> Yes, though you have formulated it better than I had in my mind.
>
> Let's see if Don can confirm that he is seeing the "could not link file"
> messages.


During my latest incident, there was only one occurrence:

could not link file “pg_wal/xlogtemp.18799" to
> “pg_wal/000000010000D45300000010”: File exists


WAL restore/recovery seemed to continue on just fine then. And it would
continue on until the pg_wal volume ran out of space unless I was manually
rm'ing already-recovered WAL files from the side.

-- 
Don Seiler
www.seiler.us

Commits

  1. Skip WAL recycling and preallocation during archive recovery.

  2. Don't ERROR on PreallocXlogFiles() race condition.

  3. Remove XLogFileInit() ability to unlink a pre-existing file.

  4. In XLogFileInit(), fix *use_existent postcondition to suit callers.

  5. Remove XLogFileInit() ability to skip ControlFileLock.

  6. Add HINT for restartpoint race with KeepFileRestoredFromArchive().

  7. Complete TODO item: