Re: LISTEN/NOTIFY bug: VACUUM sets frozenxid past a xid in async queue

Joel Jacobson <joel@compiler.org>

From: "Joel Jacobson" <joel@compiler.org>

To: pgsql-hackers <pgsql-hackers@postgresql.org>

Cc: "Heikki Linnakangas" <hlinnaka@iki.fi>

Date: 2025-10-25T18:01:56Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Clear 'xid' in dummy async notify entries written to fill up pages
- 84f1bf4afa5e 14.21 landed
- 21a9014cf00a 15.16 landed
- 0e8eaa2181d4 16.12 landed
- d80d5f099502 17.8 landed
- 82fa6b78dba1 18.2 landed
- 0bdc777e8007 19 (unreleased) landed
Fix remaining race condition with CLOG truncation and LISTEN/NOTIFY
- c2e58c0711fe 14.21 landed
- 0c862646cf2a 15.16 landed
- 44e8c60be66c 16.12 landed
- c2682810ab7d 17.8 landed
- 7b069a1876e4 18.2 landed
- 797e9ea6e54b 19 (unreleased) landed
Fix bug where we truncated CLOG that was still needed by LISTEN/NOTIFY
- eba917d360e7 14.21 landed
- 1a469d7b5b7d 15.16 landed
- 053e1868b7ee 16.12 landed
- d02c03ddc5e3 17.8 landed
- 321ec54625fd 18.2 landed
- 8eeb4a0f7c06 19 (unreleased) landed
Escalate ERRORs during async notify processing to FATAL
- 7cb05dd2d198 14.21 landed
- b1da37de21d4 15.16 landed
- c1a5bde003b8 16.12 landed
- b821c92920f0 17.8 landed
- aab4a84bb070 18.2 landed
- 1b4699090eaf 19 (unreleased) landed
Limit the size of TID lists during parallel GIN build
- c98dffcb7c70 19 (unreleased) cited

Attachments

v12-vacuum_notify_queue_cleanup-without-code-dup.txt (text/plain)
v12-vacuum_notify_queue_cleanup-with-code-dup.txt (text/plain)
async_notify_freeze_xids.txt (text/plain)

On Sat, Oct 25, 2025, at 15:08, Arseniy Mukhin wrote:
> I reimplemented the test in 0002 as an isolation test and added the
> commit message. PFA the new version.

I've rebased the patches so they can be applied on top of v12:
- v12-vacuum_notify_queue_cleanup-without-code-dup.txt
- v12-vacuum_notify_queue_cleanup-with-code-dup.txt

I also want to share a new idea that Heikki Linnakangas came up with
during PgConf25 in Riga.

The idea is basically to scan the notification queue from tail to head,
and update xids that precedes newFrozenXid, to either
FrozenTransactionId if committed, or InvalidTransactionId if not.

I've implemented this by adding a new extern function
AsyncNotifyFreezeXids to async.h, called from vac_update_datfrozenxid.

This way, we don't need to hold back the advancement of newFrozenXid in
vac_update_datfrozenxid.

Attempt of implementing Heikki's idea:
- async_notify_freeze_xids.txt

This idea has the benefit of never holding back newFrozenXid,
however, I see some problems with it:

- If there is a bug in the code, we could accidentally overwrite
  xid values we didn't intend to overwrite, and maybe we would never
  find out that we did.

- We wouldn't know for sure that we've understood the cause of
  this bug.

With v12-vacuum_notify_queue_cleanup, we instead have the downside of
possibly holding back newFrozenXid, but with the advantage of giving us
higher confidence in that we've correctly identified the cause of the
bug.

/Joel