Re: LISTEN/NOTIFY bug: VACUUM sets frozenxid past a xid in async queue

Arseniy Mukhin <arseniy.mukhin.dev@gmail.com>

From: Arseniy Mukhin <arseniy.mukhin.dev@gmail.com>

To: Matheus Alcantara <matheusssilv97@gmail.com>

Cc: Masahiko Sawada <sawada.mshk@gmail.com>, Rishu Bagga <rishu.postgres@gmail.com>, Yura Sokolov <y.sokolov@postgrespro.ru>, Daniil Davydov <3danissimo@gmail.com>, Álvaro Herrera <alvherre@kurilemu.de>, Alexandra Wang <alexandra.wang.oss@gmail.com>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>, Tom Lane <tgl@sss.pgh.pa.us>, Joel Jacobson <joel@compiler.org>

Date: 2025-09-28T13:17:48Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Clear 'xid' in dummy async notify entries written to fill up pages
- 84f1bf4afa5e 14.21 landed
- 21a9014cf00a 15.16 landed
- 0e8eaa2181d4 16.12 landed
- d80d5f099502 17.8 landed
- 82fa6b78dba1 18.2 landed
- 0bdc777e8007 19 (unreleased) landed
Fix remaining race condition with CLOG truncation and LISTEN/NOTIFY
- c2e58c0711fe 14.21 landed
- 0c862646cf2a 15.16 landed
- 44e8c60be66c 16.12 landed
- c2682810ab7d 17.8 landed
- 7b069a1876e4 18.2 landed
- 797e9ea6e54b 19 (unreleased) landed
Fix bug where we truncated CLOG that was still needed by LISTEN/NOTIFY
- eba917d360e7 14.21 landed
- 1a469d7b5b7d 15.16 landed
- 053e1868b7ee 16.12 landed
- d02c03ddc5e3 17.8 landed
- 321ec54625fd 18.2 landed
- 8eeb4a0f7c06 19 (unreleased) landed
Escalate ERRORs during async notify processing to FATAL
- 7cb05dd2d198 14.21 landed
- b1da37de21d4 15.16 landed
- c1a5bde003b8 16.12 landed
- b821c92920f0 17.8 landed
- aab4a84bb070 18.2 landed
- 1b4699090eaf 19 (unreleased) landed
Limit the size of TID lists during parallel GIN build
- c98dffcb7c70 19 (unreleased) cited

Attachments

v5-0001-Make-AsyncQueueEntry-s-self-contained.patch (text/x-patch) patch v5-0001
v5-0002-tap-tests.patch (text/x-patch) patch v5-0002

Hi,

On Wed, Sep 24, 2025 at 4:59 PM Arseniy Mukhin
<arseniy.mukhin.dev@gmail.com> wrote:
>
> On Wed, Sep 24, 2025 at 1:40 AM Matheus Alcantara
> <matheusssilv97@gmail.com> wrote:
> > ...
> > > This way the listener reads the head that includes all writer's
> > > notifications and a snapshot where the writer is not in progress, so
> > > nothing stops the listener from sending these notifications and it's
> > > even possible to have the listener's position that is after the queue
> > > head, so yes, it's bad :( Sorry about that.
> > >
> > Yeah, this is bad. I'm wondering if we could reproduce such race
> > conditions scenarios with some TAP tests.
> >
>
> I agree it would be great to have more tests for such cases. As for
> the 'committed field' patch, I think we can add a TAP test that shows
> that listeners postpone processing of notifications until
> notifications were marked as 'committed=false' in case of aborted
> transactions. I tried to write one, but have not succeeded yet. Hope
> to finish it soon.

I finally managed to write a TAP test for it, so there is a new
version with the tap test.

I also realized that we can increase test coverage in
002_aborted_tx_notifies.pl if notifications of the aborted transaction
span several pages. This way we can better test
asyncQueueRollbackNotifications(). So I changed
002_aborted_tx_notifies.pl TAP test a bit.

And there is a small indentation change in lmgr.h that should fix this
git am warning.

I added all changes as a separate patch file (0002) so it was more
clear what changed. Please feel free to merge into the main patch file
/ drop any part of it that makes sense to you.

Best regards,
Arseniy Mukhin