Weird failure with latches in curculio on v15

Michael Paquier <michael@paquier.xyz>

From: Michael Paquier <michael@paquier.xyz>
To: Postgres hackers <pgsql-hackers@lists.postgresql.org>
Date: 2023-02-01T01:53:17Z
Lists: pgsql-hackers
Hi all,

While browsing the buildfarm, I have noticed this failure on curcilio:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=curculio&dt=2023-02-01%2001%3A05%3A17

The test that has reported a failure is the check on the archive
module callback:
#   Failed test 'check shutdown callback of shell archive module'
#   at t/020_archive_status.pl line 248.
# Looks like you failed 1 test of 17.
[02:28:06] t/020_archive_status.pl .............. 
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/17 subtests 

Looking closer, this is a result of an assertion failure in the latch
code:
2023-02-01 02:28:05.615 CET [6961:8] LOG:  received fast shutdown request
2023-02-01 02:28:05.615 CET [6961:9] LOG:  aborting any active transactions
2023-02-01 02:28:05.616 CET [30681:9] LOG:  process 30681 releasing ProcSignal slot 33, but it contains 0
TRAP: FailedAssertion("latch->owner_pid == MyProcPid", File: "latch.c", Line: 451, PID: 30681)

The information available in standby2.log shows that 30681 is the
startup process.  I am not sure what all that means, yet.

Thoughts or comments welcome.
--
Michael

Commits

  1. Avoid calling proc_exit() in processes forked by system().

  2. Move extra code out of the Pre/PostRestoreCommand() section.

  3. Revert refactoring of restore command code to shell_restore.c

  4. Refactor code in charge of running shell-based recovery commands

  5. Clean up inconsistent use of fflush().

  6. Report wait events for local shell commands like archive_command.