Horton

Re: Back-patch of: avoid multiple hard links to same WAL file after a crash

Noah Misch <noah@leadboat.com>

View thread View on postgr.es
From: Noah Misch <noah@leadboat.com>
To: Andres Freund <andres@anarazel.de>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Nathan Bossart <nathandbossart@gmail.com>, Michael Paquier <michael@paquier.xyz>, Robert Pang <robertpang@google.com>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>, robertmhaas@gmail.com, pgsql-hackers@postgresql.org
Date: 2025-04-26T23:03:15Z
Lists: pgsql-hackers 
On Fri, Apr 25, 2025 at 03:35:06PM -0400, Andres Freund wrote:
> On 2025-04-20 14:53:39 -0700, Noah Misch wrote:
> > The checkpoints and WAL creation took 30s, but archiving was only 20% done
> > (based on file name 00000001000000000000006D) at the 360s PGCTLTIMEOUT.
> 
> Huh.  That seems surprisingly slow, even for valgrind.  I guess it's one more
> example for why the single-threaded archiving approach sucks so badly :)

Yes!  I also didn't expect that v14/v13 would run much faster.

> I just changed the config to --trace-children=no. There already is a valgrind
> run in progress, so it won't be in effect for the next run.

Works for me.  I see that resolved things.

Commits

  1. Test restartpoints in archive recovery.

  2. Reset InstallXLogFileSegmentActive after walreceiver self-initiated exit.

  3. Skip WAL recycling and preallocation during archive recovery.

  4. Don't ERROR on PreallocXlogFiles() race condition.

  5. Revert "Add HINT for restartpoint race with KeepFileRestoredFromArchive()."

  6. Remove XLogFileInit() ability to unlink a pre-existing file.

  7. In XLogFileInit(), fix *use_existent postcondition to suit callers.

  8. Remove XLogFileInit() ability to skip ControlFileLock.

  9. Replace durable_rename_excl() by durable_rename(), take two

  10. Add HINT for restartpoint race with KeepFileRestoredFromArchive().

  11. Remove durable_rename_excl()

  12. Replace existing durable_rename_excl() calls with durable_rename()