Horton

Re: Back-patch of: avoid multiple hard links to same WAL file after a crash

Michael Paquier <michael@paquier.xyz>

View thread View on postgr.es
From: Michael Paquier <michael@paquier.xyz>
To: Andres Freund <andres@anarazel.de>
Cc: Nathan Bossart <nathandbossart@gmail.com>, Robert Pang <robertpang@google.com>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>, robertmhaas@gmail.com, pgsql-hackers@postgresql.org
Date: 2024-12-19T05:44:53Z
Lists: pgsql-hackers

Attachments

On Wed, Dec 18, 2024 at 08:51:20PM -0500, Andres Freund wrote:
> I don't think the issue is actually quite as unlikely to be hit as reasoned in
> the commit message.  The crash has indeed to happen between the link() and
> unlink() - but at the end of a checkpoint we do that operations hundreds of
> times in a row on a busy server.  And that's just after potentially doing lots
> of write IO during a checkpoint, filling up drive write caches / eating up
> IOPS/bandwidth disk quots.

Looks so, yep.  Your timing and the report's timing are interesting.

I've been double-checking the code to refresh myself with the problem,
and I don't see a reason to not apply something like the attached set
down to v13 for all these remaining branches (minus an edit of the
commit message).

Thoughts?
--
Michael

Commits

  1. Test restartpoints in archive recovery.

  2. Reset InstallXLogFileSegmentActive after walreceiver self-initiated exit.

  3. Skip WAL recycling and preallocation during archive recovery.

  4. Don't ERROR on PreallocXlogFiles() race condition.

  5. Revert "Add HINT for restartpoint race with KeepFileRestoredFromArchive()."

  6. Remove XLogFileInit() ability to unlink a pre-existing file.

  7. In XLogFileInit(), fix *use_existent postcondition to suit callers.

  8. Remove XLogFileInit() ability to skip ControlFileLock.

  9. Replace durable_rename_excl() by durable_rename(), take two

  10. Add HINT for restartpoint race with KeepFileRestoredFromArchive().

  11. Remove durable_rename_excl()

  12. Replace existing durable_rename_excl() calls with durable_rename()