Horton

Re: avoid multiple hard links to same WAL file after a crash

Nathan Bossart <nathandbossart@gmail.com>

View thread View on postgr.es
From: Nathan Bossart <nathandbossart@gmail.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: Kyotaro Horiguchi <horikyota.ntt@gmail.com>, tgl@sss.pgh.pa.us, robertmhaas@gmail.com, pgsql-hackers@postgresql.org
Date: 2022-05-02T17:39:07Z
Lists: pgsql-hackers 
On Mon, May 02, 2022 at 07:48:18PM +0900, Michael Paquier wrote:
> Skimming through at the buildfarm logs, it happens that the tests are
> able to see this race from time to time.  Here is one such example on
> rorqual:
> https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=rorqual&dt=2022-04-20%2004%3A47%3A58&stg=recovery-check
> 
> And here are the relevant logs:
> 2022-04-20 05:04:19.028 UTC [3109048][startup][:0] LOG:  restored log
> file "00000002.history" from archive
> 2022-04-20 05:04:19.029 UTC [3109111][walreceiver][:0] LOG:  fetching
> timeline history file for timeline 2 from primary server
> 2022-04-20 05:04:19.048 UTC [3109111][walreceiver][:0] FATAL:  could
> not link file "pg_wal/xlogtemp.3109111" to "pg_wal/00000002.history":
> File exists
> [...]
> 2022-04-20 05:04:19.234 UTC [3109250][walreceiver][:0] LOG:  started
> streaming WAL from primary at 0/3000000 on timeline 2
> 
> The WAL receiver upgrades the ERROR to a FATAL, and restarts
> streaming shortly after.  Using durable_rename() would not be an issue
> here.

Thanks for investigating this one.  I think I agree that we should simply
switch to durable_rename() (without a file existence check beforehand).

-- 
Nathan Bossart
Amazon Web Services: https://aws.amazon.com

Commits

  1. Test restartpoints in archive recovery.

  2. Reset InstallXLogFileSegmentActive after walreceiver self-initiated exit.

  3. Skip WAL recycling and preallocation during archive recovery.

  4. Don't ERROR on PreallocXlogFiles() race condition.

  5. Revert "Add HINT for restartpoint race with KeepFileRestoredFromArchive()."

  6. Remove XLogFileInit() ability to unlink a pre-existing file.

  7. In XLogFileInit(), fix *use_existent postcondition to suit callers.

  8. Remove XLogFileInit() ability to skip ControlFileLock.

  9. Replace durable_rename_excl() by durable_rename(), take two

  10. Add HINT for restartpoint race with KeepFileRestoredFromArchive().

  11. Remove durable_rename_excl()

  12. Replace existing durable_rename_excl() calls with durable_rename()