Re: hang during shutdown

Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>

From: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
To: Srinath Reddy Sadipiralla <srinath2133@gmail.com>
Cc: Jeremy Schneider <schneider@ardentperf.com>, PostgreSQL mailing lists <pgsql-hackers@postgresql.org>
Date: 2026-07-06T06:30:06Z
Lists: pgsql-hackers
Hi,

On Sun, Jul 5, 2026 at 11:55 PM Jeremy Schneider
<schneider@ardentperf.com> wrote:
> so far, it all started with this postgres hang during a shutdown. we didnt capture enough diagnostics to root-cause the domino before the hang, but the data i do have about this postgres hang is very peculiar. i'm working on some automation to gather more diagnostics if there is a recurrence. in the meantime i wanted to share what i do know, in case anyone else has seen something similar.
>
> postgres version 16.13
> Ubuntu 22.04.5 LTS
> kernel 5.15.0-1110-azure
>
> between log messages and wal contents, it appears PostmasterStateMachine transitioned into PM_WAIT_BACKENDS but never progressed beyond this for a full 30 minutes until external automation finally killed the processes.

Did you have an active logical replication stream at the time of the
shutdown? If so, it's very likely due to a bug in the logical
walsender that can be stuck in an infinite loop when the WAL is in a
specific state (last record is a rollback crossing the WAL page
boundary).

A fix for that was released in the latest 16.14 minor release[0].

[0]: https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=82935467a

Regards,
Anthonin Bonnefoy



Commits

  1. Add wal_sender_shutdown_timeout GUC to limit shutdown wait for replication