Horton

Thread

  1. LISTEN/NOTIFY enhancement: Portable signal handling?

    Sean Chittenden <sean@chittenden.org> — 2004-12-27T09:15:23Z

    Howdy.  I'm starting some work on our favorite LISTEN/NOTIFY subsystem 
in the hopes of more advanced functionality.  Right now I'm using a 
bastardized (RAISE NOTICE + tailing error logs) + NOTIFY to get what 
should just be built in to the LISTEN/NOTIFY interface.  Here's the 
syntax for the proposed functionality.  I've updated the grammar to 
support the following:

   LISTEN name [BLOCK] [WITH] [TIMEOUT [=] ''::INTERVAL];
   NOTIFY name a_expr;

But am having something of a think-o when it comes to the semantics of 
the BLOCK keyword for the LISTEN command.  Let me first explain my 
goals for this exercise:

* Allow passing of a data type in a NOTIFY.  Something like:

   NOTIFY 'relname' a_expr;

   Should be able to pass a text, INET, etc... shouldn't matter... not 
sure if this is possible though given that OIDs don't travel with data 
types... I may have to settle for a TEXT arg, which is acceptable.

* Allow LISTEN to block until a value changes.  LISTEN [BLOCK|WAIT] 
'relname'

* Allow LISTEN to have a timeout

   LISTEN name [BLOCK] [WITH] [TIMEOUT [=] ''::INTERVAL];

* Allow blocking LISTEN queries to update the status of the proc title
   while blocking.


Basically I want to introduce formal support for turning PostgreSQL 
into a message bus.  To start with, it won't be a scalable message bus 
that can scale to thousands of concurrent connections (that's something 
I'd like to do long term(tm), but I digress).  The problem is with the 
BLOCK'ing semantics or implementation.  There are two ways that I can 
do this, IMHO, but I'm out to solicit alternatives as I'm still getting 
a handle on what the best interfaces/APIs are internally to get 
something done.

Option 1) Use sleep(3) for the given timeout and wake up on some 
interruptible a signal (USR2?).  This is the simplest solution, but 
likely the least portable to win32.  Given the new world order of 8.0 
and it's portability headaches, it's something I'm aware of.

Option 2) block on an exclusive lock.  Check to see if relname has been 
registered.  If it has, block on the existing exclusive lock (can I 
block on a lock for a given number of sec/ms?).  If it hasn't, create 
an exclusive lock, but give the lock away (to the parent postmaster, a 
lockmgr proc, etc) so that a notify can remove the remove and unlock 
the exclusive lock so that all of the blocking children wake up.

The async interface is nice, but not really useful to me as it requires 
polling, instead of unblocking when an event comes through, which would 
create a vastly more real time interface that should be easier on the 
database.  Does anyone think there would be any conflicts with the use 
of the existing alarm code from storage/lmgr/proc.c for the 
LISTEN/NOTIFY interface?  It looks like SIGALRM has a reserved purpose. 
  I haven't found any global alarm handling interface that can be used 
to assign different meanings when an SIGALRM is received.  Any other 
thoughts on the best way to proceed?

-sc

-- 
Sean Chittenden

  2. Re: LISTEN/NOTIFY enhancement: Portable signal handling?

    Tom Lane <tgl@sss.pgh.pa.us> — 2004-12-27T17:04:35Z

    Sean Chittenden <sean@chittenden.org> writes:
> * Allow LISTEN to block until a value changes.  LISTEN [BLOCK|WAIT] 
> 'relname'

> * Allow LISTEN to have a timeout

>    LISTEN name [BLOCK] [WITH] [TIMEOUT [=] ''::INTERVAL];

> * Allow blocking LISTEN queries to update the status of the proc title
>    while blocking.

I don't believe in any of these things, at least not on the server side.
You can get the same effect on the client side without cluttering the
LISTEN semantics and implementation.

			regards, tom lane

  3. Re: LISTEN/NOTIFY enhancement: Portable signal handling?

    Tom Lane <tgl@sss.pgh.pa.us> — 2004-12-27T17:41:07Z

    Sean Chittenden <sean@chittenden.org> writes:
> The async interface is nice, but not really useful to me as it requires 
> polling, instead of unblocking when an event comes through, which would 
> create a vastly more real time interface that should be easier on the 
> database.

BTW, this is nonsense; the backend sends a message exactly when the
NOTIFY occurs.  It may well be that libpq needs some work to make it
easier to wait for a NOTIFY without polling, but again there isn't a
reason to clutter the server-side code with it.

			regards, tom lane

  4. Re: LISTEN/NOTIFY enhancement: Portable signal handling?

    Sean Chittenden <sean@chittenden.org> — 2004-12-27T19:09:37Z

    >> The async interface is nice, but not really useful to me as it 
>> requires
>> polling, instead of unblocking when an event comes through, which 
>> would
>> create a vastly more real time interface that should be easier on the
>> database.
>
> BTW, this is nonsense; the backend sends a message exactly when the
> NOTIFY occurs.  It may well be that libpq needs some work to make it
> easier to wait for a NOTIFY without polling, but again there isn't a
> reason to clutter the server-side code with it.

It's asynchronous with regards to client delivery of the message.  
Sending of the NOTIFY message is synchronous, but delivery to the 
client is not.  I don't see how it could be any other way in 
PostgreSQL.  libpq(3) actually has a reasonable interface that relies 
on the developer to block on the fd as described here:

http://developer.postgresql.org/docs/postgres/libpq-notify.html

The problem is that this doesn't work in pl/*, which is the problem I 
was trying to address.  *shrug*  -sc

-- 
Sean Chittenden

  5. Re: LISTEN/NOTIFY enhancement: Portable signal handling?

    Zygo Blaxell <yswnhdxi@umail.furryterror.org> — 2005-10-24T16:02:11Z

    In case anyone's wondering why I'm replying to an article that's nearly a
year old, it came up in a search while I was looking for the more recent
LISTEN/NOTIFY thread...

In article <7373.1104169267@sss.pgh.pa.us>,
Tom Lane  <tgl@sss.pgh.pa.us> wrote:
>Sean Chittenden <sean@chittenden.org> writes:
>> The async interface is nice, but not really useful to me as it requires 
>> polling, instead of unblocking when an event comes through, which would 
>> create a vastly more real time interface that should be easier on the 
>> database.
>
>BTW, this is nonsense; the backend sends a message exactly when the
>NOTIFY occurs.  It may well be that libpq needs some work to make it
>easier to wait for a NOTIFY without polling, but again there isn't a
>reason to clutter the server-side code with it.

One thing that persistently annoys me about using LISTEN in clients
is that much of the time I'm not using libpq directly, but some layer
above libpq that implements a generic SQL interface.  These interfaces
generally assume that SQL calls are synchronous and blocking unless
you use some kind of escape to the lower-level driver, and in a few
environments there is no such escape at all.

I'd really like to have something that looks as much like an SQL statement
as possible, which blocks until a NOTIFY event or a timeout occurs.  If not
a stand-alone SQL command then at least a function.

In one case where I really needed this, I implemented a really *ugly*
PL/PerlU function that did the following:

	open a new client connection (with caching) to the server from
	the server's own backend

	set up appropriate LISTEN commands,

	go to sleep on a poll() from the file descriptor,

	then return immediately after poll() does.

Thankfully this particular application was not required to scale beyond
a half-dozen clients!
-- 
Zygo Blaxell (Laptop) <zblaxell@feedme.hungrycats.org>
GPG = B203 9402 B0E7 0F20 3B74  B13C DFBC C916 4395 FD03