walsender-cannot-exit.txt
text/plain
Filename: walsender-cannot-exit.txt
Type: text/plain
Part: 0
With pg_usleep(10000L) added in pqReadData() you can see several tests
failed when running make check-world/meson test.
For example:
meson test recovery/019_replslot_limit
results in:
[16:25:18.510](6.771s) ok 16 - check if checkpoint command is not blocked
### Stopping node "primary2" using mode fast
# Running: pg_ctl -D .../build/testrun/recovery/019_replslot_limit/data/t_019_replslot_limit_primary2_data/pgdata -m fast stop
waiting for server to shut down............................................................... failed
pg_ctl: server does not shut down
# pg_ctl stop failed: 256
# Postmaster PID for node "primary2" is 209253
[16:26:18.567](60.057s) Bail out! pg_ctl stop failed
As I understand, it happens because WalSndLoop() performs this in a loop:
...
if (got_SIGUSR2)
WalSndDone(send_data);
...
/* Sleep until something happens or we time out */
WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_MAIN);
...
where WalSndDone() does this:
replicatedPtr = XLogRecPtrIsInvalid(MyWalSnd->flush) ?
MyWalSnd->write : MyWalSnd->flush;
if (WalSndCaughtUp && sentPtr == replicatedPtr &&
!pq_is_send_pending())
{
...
proc_exit(0);
}
if (!waiting_for_ping_response)
WalSndKeepalive(true, InvalidXLogRecPtr);
(WalSndWait's call chain is:
WalSndWait -> WaitEventSetWait -> WaitEventSetWaitBlock -> epoll_wait)
So, WalSndDone waits here for replicatedPtr to become equal to sentPtr.
In other words, walsender sends keepalive to walreceiver and waits for a
new flush position from it.
And the standby loop is as follows:
WalReceiverMain:
for (;;)
{
if (len > 0)
{
...
XLogWalRcvProcessMsg(buf[0], &buf[1], len - 1,
startpointTLI);
...
}
else if (len == 0)
break;
}
...
XLogWalRcvFlush(false, startpointTLI);
where XLogWalRcvProcessMsg() calls XLogWalRcvSendReply() if a keepalive
message received.
So if walreceiver constantly receives some messages (keepalive messages in
this case), it just can't get out of the loop and perform XLogWalRcvFlush()
/move Flush position what primary is waiting for.
But if something delays walsender for more than 10 ms between epoll_wait()
and WalSndDone() on a next iteration, standby manages to get len = 0 and to
perform Flush.