Horton

Re: Optimize LISTEN/NOTIFY

Chao Li <li.evan.chao@gmail.com>

View thread
From: Chao Li <li.evan.chao@gmail.com>
To: Joel Jacobson <joel@compiler.org>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Thomas Munro <thomas.munro@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, Heikki Linnakangas <hlinnaka@iki.fi>, Rishu Bagga <rishu.postgres@gmail.com>
Date: 2025-09-29T02:33:54Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Optimize LISTEN/NOTIFY via shared channel map and direct advancement.

  2. Fix incorrect logic for caching ResultRelInfos for triggers

> On Sep 28, 2025, at 18:24, Joel Jacobson <joel@compiler.org> wrote:
> 
>> 
>> I might miss the factor of holding an exclusive lock. I will revisit 
>> that part again.
> 
> I've re-read this entire thread, and I actually think my original
> approaches are more promising, that is, the
> 0001-optimize_listen_notify-v4.patch patch, doing multicast targeted
> signaling.
> 
> Therefore, merely consider the latest patch as PoC with some possible
> interesting ideas.
> 
> Before this patch, I had never used PostgreSQL's timeout mechanism
> before, so I didn't consider it when thinking about how to solve the
> remaining problems with 0001-optimize_listen_notify-v4.patch, which
> currently can't guarantee that all listening backends will eventually
> catch up, since it just kicks one of the most lagging ones, for each
> notification. This could be a problem in practise if there is a long
> period of time with no notifications coming in. Then some listening
> backends could end up not being signaled and would stay behind,
> preventing the queue tail from advancing.
> 
> I'm thinking maybe somehow we can use the timeout mechanism here, but
> I'm not sure how yet. Any ideas?
> 
> /Joel


Hi Joel,

I never had a concern about using the timeout mechanism. My comment was about enabling timeout duplicately.

I just revisited the code, now I agree that I was over-worried because I missed considering NotifyQueueLock. With the lock protection, a backend process’ QUEUE_BACKEND_WAKEUP_PENDING_FLAG won’t have race condition, then it should have no duplicate signals sending to the same backend process. Then in the backend process, you have “last_wakeup_start_time” that avoids duplicate timeout within a configured period, and you reset last_wakeup_start_time in asyncQueueReadAllNotifications() together with cleaning the QUEUE_BACKEND_WAKEUP_PENDING_FLAG.

So, overall v2 looks good to me.

One last tiny comment is about naming of last_wakeup_start_time. I think it can be renamed to “last_wakeup_time”. Because the variable just records when asyncQueueReadAllNotifications() last time called, there seems not a meaning of “start” involved.

Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/