Horton

Re: Race between KeepFileRestoredFromArchive() and restartpoint

Noah Misch <noah@leadboat.com>

View thread View on postgr.es
From: Noah Misch <noah@leadboat.com>
To: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Cc: don@seiler.us, david@pgmasters.net, pgsql-hackers@postgresql.org
Date: 2022-08-03T07:28:47Z
Lists: pgsql-hackers 
On Wed, Aug 03, 2022 at 11:24:17AM +0900, Kyotaro Horiguchi wrote:
> At Tue, 2 Aug 2022 16:03:42 -0500, Don Seiler <don@seiler.us> wrote in 
> > could not link file “pg_wal/xlogtemp.18799" to
> > > “pg_wal/000000010000D45300000010”: File exists

> Hmm.  It seems like a race condition betwen StartupXLOG() and
> RemoveXlogFIle(). We need wider extent of ContolFileLock. Concretely
> taking ControlFileLock before deciding the target xlog file name in
> RemoveXlogFile() seems to prevent this happening. (If this is correct
> this is a live issue on the master branch.)

RemoveXlogFile() calls InstallXLogFileSegment() with find_free=true.  The
intent of find_free=true is to make it okay to pass a target xlog file that
ceases to be a good target.  (InstallXLogFileSegment() searches for a good
target while holding ControlFileLock.)  Can you say more about how that proved
to be insufficient?

Commits

  1. Skip WAL recycling and preallocation during archive recovery.

  2. Don't ERROR on PreallocXlogFiles() race condition.

  3. Remove XLogFileInit() ability to unlink a pre-existing file.

  4. In XLogFileInit(), fix *use_existent postcondition to suit callers.

  5. Remove XLogFileInit() ability to skip ControlFileLock.

  6. Add HINT for restartpoint race with KeepFileRestoredFromArchive().

  7. Complete TODO item: