Horton

Re: Race conditions in 019_replslot_limit.pl

Andres Freund <andres@anarazel.de>

View thread View on postgr.es
From: Andres Freund <andres@anarazel.de>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Masahiko Sawada <sawada.mshk@gmail.com>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>, hlinnaka@iki.fi, pgsql-hackers@postgresql.org
Date: 2022-02-18T22:42:48Z
Lists: pgsql-hackers 
Hi,

On 2022-02-17 21:55:21 -0800, Andres Freund wrote:
> Isn't it pretty bonkers that we allow error processing to get stuck behind
> network traffic, *before* we have have released resources (locks etc)?

This is particularly likely to be a problem for walsenders, because they often
have a large output buffer filled, because walsender uses
pq_putmessage_noblock() to send WAL data. Which obviously can be large.

In the stacktrace upthread you can see:
#3  0x00007faf4b70f48b in secure_write (port=0x7faf4c22da50, ptr=0x7faf4c2f1210, len=21470) at /home/andres/src/postgresql/src/backend/libpq/be-secure.c:29

which certainly is more than in most other cases of error messages being
sent. And it obviously might not be the first to have gone out.


> I wonder if we should try to send, but do it in a nonblocking way.

I think we should probably do so at least during FATAL error processing. But
also consider doing so for ERROR, because not releasing resources after
getting cancelled / terminated is pretty nasty imo.

Greetings,

Andres Freund

Commits

  1. Revert 019_replslot_limit.pl related debugging aids.

  2. Don't fail for > 1 walsenders in 019_replslot_limit, add debug messages.

  3. Add retries for further investigation of 019_replslot_limit.pl failures.

  4. Add further debug info to help debug 019_replslot_limit.pl failures.

  5. Add temporary debug info to help debug 019_replslot_limit.pl failures.

  6. Move replication slot release to before_shmem_exit().

  7. Server-side fix for delayed NOTIFY and SIGTERM processing.