Re: LISTEN/NOTIFY bug: VACUUM sets frozenxid past a xid in async queue

Matheus Alcantara <matheusssilv97@gmail.com>

From: "Matheus Alcantara" <matheusssilv97@gmail.com>

To: "Joel Jacobson" <joel@compiler.org>, Álvaro Herrera <alvherre@kurilemu.de>, "Arseniy Mukhin" <arseniy.mukhin.dev@gmail.com>

Cc: "Matheus Alcantara" <matheusssilv97@gmail.com>, "Masahiko Sawada" <sawada.mshk@gmail.com>, "Rishu Bagga" <rishu.postgres@gmail.com>, "Yura Sokolov" <y.sokolov@postgrespro.ru>, "Daniil Davydov" <3danissimo@gmail.com>, "Alexandra Wang" <alexandra.wang.oss@gmail.com>, "pgsql-hackers" <pgsql-hackers@postgresql.org>, "Tom Lane" <tgl@sss.pgh.pa.us>

Date: 2025-10-20T12:41:43Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Clear 'xid' in dummy async notify entries written to fill up pages
- 84f1bf4afa5e 14.21 landed
- 21a9014cf00a 15.16 landed
- 0e8eaa2181d4 16.12 landed
- d80d5f099502 17.8 landed
- 82fa6b78dba1 18.2 landed
- 0bdc777e8007 19 (unreleased) landed
Fix remaining race condition with CLOG truncation and LISTEN/NOTIFY
- c2e58c0711fe 14.21 landed
- 0c862646cf2a 15.16 landed
- 44e8c60be66c 16.12 landed
- c2682810ab7d 17.8 landed
- 7b069a1876e4 18.2 landed
- 797e9ea6e54b 19 (unreleased) landed
Fix bug where we truncated CLOG that was still needed by LISTEN/NOTIFY
- eba917d360e7 14.21 landed
- 1a469d7b5b7d 15.16 landed
- 053e1868b7ee 16.12 landed
- d02c03ddc5e3 17.8 landed
- 321ec54625fd 18.2 landed
- 8eeb4a0f7c06 19 (unreleased) landed
Escalate ERRORs during async notify processing to FATAL
- 7cb05dd2d198 14.21 landed
- b1da37de21d4 15.16 landed
- c1a5bde003b8 16.12 landed
- b821c92920f0 17.8 landed
- aab4a84bb070 18.2 landed
- 1b4699090eaf 19 (unreleased) landed
Limit the size of TID lists during parallel GIN build
- c98dffcb7c70 19 (unreleased) cited

On Sun Oct 19, 2025 at 2:14 PM -03, Joel Jacobson wrote:
> On Fri, Oct 17, 2025, at 15:51, Álvaro Herrera wrote:
>> I have the impression that this thread has lost focus on the idea of
>> producing a backpatchable bugfix.  The last proposed patch has a lot of
>> new mechanism that doesn't seem suitable for backpatch.  I could be
>> wrong of course.
>
> I've tried to create a minimal isolated fix, hopefully suitable for
> backpatching, with no new mechanisms, other than the added
> GetOldestQueuedNotifyXid used by vac_update_datfrozenxid.
>
> It's based on the approach discussed earlier in this thread, that just
> goes through the notification queue from QUEUE_TAIL to QUEUE_HEAD, to
> find the oldestXid in the current database.
>
> Implementation:
>
> * Break out SLRU read page code from asyncQueueReadAllNotifications into
>   new helper-function asyncQueueReadPageToBuffer.
>
> * Add GetOldestQueuedNotifyXid which uses the new helper-function
>   asyncQueueReadPageToBuffer.
>
> It passes the 001_xid_freeze.pl test, not included in this patch.
>
This is similar to what was already proposed at [1]. This approach was
abandoned because a notification on the queue may block datfrozenxid
advance and clog truncation which can cause other issues for the users [2].


[1] https://www.postgresql.org/message-id/CAFY6G8cJm73_MM9SuynZUqtqcaTuepUDgDuvS661oLW7U0dgsg%40mail.gmail.com
[2] https://www.postgresql.org/message-id/d186fba0-dc65-4274-aa96-3906bbb2e530%40postgrespro.ru

--
Matheus Alcantara