Re: LISTEN/NOTIFY bug: VACUUM sets frozenxid past a xid in async queue
Masahiko Sawada <sawada.mshk@gmail.com>
From: Masahiko Sawada <sawada.mshk@gmail.com>
To: Matheus Alcantara <matheusssilv97@gmail.com>
Cc: Álvaro Herrera <alvherre@kurilemu.de>, Joel Jacobson <joel@compiler.org>, Arseniy Mukhin <arseniy.mukhin.dev@gmail.com>, Rishu Bagga <rishu.postgres@gmail.com>, Yura Sokolov <y.sokolov@postgrespro.ru>, Daniil Davydov <3danissimo@gmail.com>, Alexandra Wang <alexandra.wang.oss@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, Tom Lane <tgl@sss.pgh.pa.us>
Date: 2025-10-21T21:42:18Z
Lists: pgsql-hackers
Commits
Same data as JSON:
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Clear 'xid' in dummy async notify entries written to fill up pages
- 84f1bf4afa5e 14.21 landed
- 21a9014cf00a 15.16 landed
- 0e8eaa2181d4 16.12 landed
- d80d5f099502 17.8 landed
- 82fa6b78dba1 18.2 landed
- 0bdc777e8007 19 (unreleased) landed
-
Fix remaining race condition with CLOG truncation and LISTEN/NOTIFY
- c2e58c0711fe 14.21 landed
- 0c862646cf2a 15.16 landed
- 44e8c60be66c 16.12 landed
- c2682810ab7d 17.8 landed
- 7b069a1876e4 18.2 landed
- 797e9ea6e54b 19 (unreleased) landed
-
Fix bug where we truncated CLOG that was still needed by LISTEN/NOTIFY
- eba917d360e7 14.21 landed
- 1a469d7b5b7d 15.16 landed
- 053e1868b7ee 16.12 landed
- d02c03ddc5e3 17.8 landed
- 321ec54625fd 18.2 landed
- 8eeb4a0f7c06 19 (unreleased) landed
-
Escalate ERRORs during async notify processing to FATAL
- 7cb05dd2d198 14.21 landed
- b1da37de21d4 15.16 landed
- c1a5bde003b8 16.12 landed
- b821c92920f0 17.8 landed
- aab4a84bb070 18.2 landed
- 1b4699090eaf 19 (unreleased) landed
-
Limit the size of TID lists during parallel GIN build
- c98dffcb7c70 19 (unreleased) cited
On Mon, Oct 20, 2025 at 11:19 AM Matheus Alcantara <matheusssilv97@gmail.com> wrote: > > On Mon Oct 20, 2025 at 11:18 AM -03, Álvaro Herrera wrote: > > On 2025-Oct-20, Matheus Alcantara wrote: > > > >> This is similar to what was already proposed at [1]. This approach was > >> abandoned because a notification on the queue may block datfrozenxid > >> advance and clog truncation which can cause other issues for the users [2]. > > > > Well, I think that this is the right solution for backpatching, and that > > you were wrong to abandon it. You can continue to design a better > > mechanism for the master branch, but in old branches we cannot really do > > all those things you're proposing to do. > > > I actually would prefer this approach TBH, but since this can cause > other issues like transaction wraparound due to not consumed > notifications we would need other mechanisms to prevent that and I'm not > sure if users should expect this kind of behavior changes on minor > version updates? True, unconsumed notifications could cause transaction wraparound by preventing datfrozenxid from advancing. However, this risk only applies when users have long-term unconsumed notifications, which is uncommon. That said, we should note that, as I mentioned previously[1], a process can accumulate unconsumed notifications simply by being in idle-in-transaction state, even without backend_xmin and backend_xid, which prevents datfrozenxid from advancing. While this might not be problematic in practice if it's rare, I find it concerning that we have no way to check the age of unconsumed notifications. > > I think that to go with this solution we would need some way to drop too > old notifications from the queue to advance the datfrozenxid, so I > imagine that we would need some GUC to make this configurable and we can > configure a default value of course but some use cases may not be the > best configuration, this is something that users should expected to deal > on minor version updates? I think adding a new GUC would be overkill for this fix. As for dropping old notifications from the queue, we probably don't need to make it configurable - we could simply drop notifications whose commit status is no longer available (instead of raising an error). > > Going with the "self contained" idea sound more easier to backpatch > actually, so this is the main reason that I abandoned this other > approach. Could you please point what make the v8 version not visible > for bachpatching? Regarding the v8 patch, it introduces a fundamentally new way of managing notification entries (adding entries with 'committed' state and marking them 'aborted' in abort paths). This affects all use cases, not just those involving very old unconsumed notifications, and could introduce more serious bugs like PANIC or SEGV. For backpatching, I prefer targeting just the problematic behavior while leaving unrelated parts unchanged. Though Álvaro might have a different perspective on this. Regards, [1] https://www.postgresql.org/message-id/CAD21AoCD%2BHXoc2QZCAS9d8ahDeikNqbnU0i6cQzpMFOEurkPPg%40mail.gmail.com -- Masahiko Sawada Amazon Web Services: https://aws.amazon.com