Horton

Re: avoid multiple hard links to same WAL file after a crash

Michael Paquier <michael@paquier.xyz>

View thread View on postgr.es
From: Michael Paquier <michael@paquier.xyz>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Nathan Bossart <nathandbossart@gmail.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2022-04-18T07:48:35Z
Lists: pgsql-hackers 
On Fri, Apr 08, 2022 at 09:00:36PM -0400, Robert Haas wrote:
> On Fri, Apr 8, 2022 at 12:53 PM Nathan Bossart <nathandbossart@gmail.com> wrote:
>> I think there might be another problem.  The man page for rename() seems to
>> indicate that overwriting an existing file also introduces a window where
>> the old and new path are hard links to the same file.  This isn't a problem
>> for the WAL files because we should never be overwriting an existing one,
>> but I wonder if it's a problem for other code paths.  My guess is that many
>> code paths that overwrite an existing file are first writing changes to a
>> temporary file before atomically replacing the original.  Those paths are
>> likely okay, too, as you can usually just discard any existing temporary
>> files.
> 
> I wonder if this is really true. I thought rename() was supposed to be atomic.

Not always.  For example, some old versions of MacOS have a non-atomic
implementation of rename(), like prairiedog with 10.4.  Even 10.5 does
not handle atomicity as far as I call.  In short, it looks like a bad
idea to me to rely on this idea at all.  Some FSes have their own way
of handling things, as well, but I am not much into this world.

Saying that, it would be nice to see durable_rename_excl() gone as it
has created quite a bit of pain for us in the past years.
--
Michael

Commits

  1. Test restartpoints in archive recovery.

  2. Reset InstallXLogFileSegmentActive after walreceiver self-initiated exit.

  3. Skip WAL recycling and preallocation during archive recovery.

  4. Don't ERROR on PreallocXlogFiles() race condition.

  5. Revert "Add HINT for restartpoint race with KeepFileRestoredFromArchive()."

  6. Remove XLogFileInit() ability to unlink a pre-existing file.

  7. In XLogFileInit(), fix *use_existent postcondition to suit callers.

  8. Remove XLogFileInit() ability to skip ControlFileLock.

  9. Replace durable_rename_excl() by durable_rename(), take two

  10. Add HINT for restartpoint race with KeepFileRestoredFromArchive().

  11. Remove durable_rename_excl()

  12. Replace existing durable_rename_excl() calls with durable_rename()