Horton

Re: Optimize LISTEN/NOTIFY

Arseniy Mukhin <arseniy.mukhin.dev@gmail.com>

View thread
From: Arseniy Mukhin <arseniy.mukhin.dev@gmail.com>
To: Joel Jacobson <joel@compiler.org>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2025-11-13T06:36:22Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Optimize LISTEN/NOTIFY via shared channel map and direct advancement.

  2. Fix incorrect logic for caching ResultRelInfos for triggers

On Thu, Nov 13, 2025 at 9:28 AM Joel Jacobson <joel@compiler.org> wrote:
>
> On Wed, Nov 12, 2025, at 17:57, Arseniy Mukhin wrote:
> > IIUC it's impossible for the listener to stop somewhere in between
> > queueHeadBeforeWrite and queueHeadAfterWrite. If the listener has
> > managed to read the first notification from the notifier, it means the
> > notifier transaction is complete and the listener should stop only
> > after reading all notifications (so we should always see pos =
> > queueHeadAfterWrite or further).
> >
> > So If I haven't missed anything, I think we can use QUEUE_POS_EQUAL as
> > direct advancement condition:
> >
> >     if (!QUEUE_BACKEND_IS_ADVANCING(i) && QUEUE_POS_EQUAL(pos,
> > queueHeadBeforeWrite))
> >     {
> >            QUEUE_BACKEND_POS(i) = queueHeadAfterWrite;
> >     }
>
> I added some logging just to test the hypothesis:
>
> @@ -2072,6 +2082,12 @@ SignalBackends(void)
>             {
>                 Assert(!QUEUE_POS_PRECEDES(pos, queueHeadBeforeWrite));
>
> +               if (!QUEUE_POS_EQUAL(pos, queueHeadBeforeWrite))
> +                   elog(LOG, "Direct advancement: PID %d from pos (%lld,%d) to queueHeadAfterWrite (%lld,%d)",
> +                        pid,
> +                        (long long) QUEUE_POS_PAGE(pos), QUEUE_POS_OFFSET(pos),
> +                        (long long) QUEUE_POS_PAGE(queueHeadAfterWrite), QUEUE_POS_OFFSET(queueHeadAfterWrite));
> +
>                 QUEUE_BACKEND_POS(i) = queueHeadAfterWrite;
>             }
>         }
>
> And I'm getting a lot of such log entries when benchmarking
> `./pg_async_notify_test --listeners 1 --notifiers 1 --channels 50`
>
> I think this confirms that listeners can actually stop somewhere in between
> queueHeadBeforeWrite and queueHeadAfterWrite.


Ahh, yes, I think you are right. I missed that notifiers update the
head when they move to the next page. Thank you for the detailed
example and sorry for taking your time with it. I agree that
QUEUE_POS_PRECEDES(pos, queueHeadAfterWrite) is correct and covers
more cases where we can do direct advancement.


Best regards,
Arseniy Mukhin