Thread
-
Re: [HACKERS] Proposal for async support in libpq
Tom Lane <tgl@sss.pgh.pa.us> — 1998-04-17T19:51:17Z
Bruce Momjian <maillist@candle.pha.pa.us> writes: > This all looks good. Another thing we really need it to be able to > cancel queries. This would be a big win, and looks like it could fit > into the scheme here. I thought about proposing a PQcancelAsync that would cancel the active query-in-progress. But that would require support on the backend side, and I am far from competent to make it happen. (libpq is simple enough that I'm not afraid to rewrite it, but making major mods to the backend is another story. I just got here this week...) If anyone who does know what they're doing is willing to make the necessary backend mods, I'm all for it. The libpq side would be easy enough. How would such cancellation interact with transactions, btw? Would you expect it to roll back only the current command, or abort the whole transaction? We'd also have to consider corner cases, like when the backend has already finished the query by the time it gets the cancel request. regards, tom lane
-
Re: [HACKERS] Proposal for async support in libpq
Bruce Momjian <maillist@candle.pha.pa.us> — 1998-04-17T20:26:22Z
> > Bruce Momjian <maillist@candle.pha.pa.us> writes: > > This all looks good. Another thing we really need it to be able to > > cancel queries. This would be a big win, and looks like it could fit > > into the scheme here. > > I thought about proposing a PQcancelAsync that would cancel the active > query-in-progress. But that would require support on the backend side, > and I am far from competent to make it happen. (libpq is simple enough > that I'm not afraid to rewrite it, but making major mods to the backend > is another story. I just got here this week...) > > If anyone who does know what they're doing is willing to make the > necessary backend mods, I'm all for it. The libpq side would be > easy enough. > > How would such cancellation interact with transactions, btw? Would > you expect it to roll back only the current command, or abort the > whole transaction? We'd also have to consider corner cases, like > when the backend has already finished the query by the time it gets > the cancel request. It is pretty easy, just an elog(ERROR) would do it. The issue is allowing the backend to see the request. We can put some checks in tcop/postgres.c as it moves from module to module, and something in the executor to check for the cancel, and do an elog(ERROR). It would be nice if it arrived as out-of-band data, so we could check for input quickly without having to actually process it if it is not a cancel notification. The out-of-band data will send a SIGURG signal to the backend, and we can set a global variable, and check the variable at various places. To do all this, we need to be able to send a query, and not have it block, and it seems you are giving us this capability. You supply the indication to the backend, and I will see that the backend processes it properly. -- Bruce Momjian | 830 Blythe Avenue maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026 + If your life is a hard drive, | (610) 353-9879(w) + Christ can be your backup. | (610) 853-3000(h)
-
Re: [HACKERS] Proposal for async support in libpq
Bruce Momjian <maillist@candle.pha.pa.us> — 1998-04-18T05:13:52Z
> > Bruce Momjian <maillist@candle.pha.pa.us> writes: > > This all looks good. Another thing we really need it to be able to > > cancel queries. This would be a big win, and looks like it could fit > > into the scheme here. > > I thought about proposing a PQcancelAsync that would cancel the active > query-in-progress. But that would require support on the backend side, > and I am far from competent to make it happen. (libpq is simple enough > that I'm not afraid to rewrite it, but making major mods to the backend > is another story. I just got here this week...) In backend/libpq/pqcomm.c, I see pg_sendoob() which sends out-of-band data FROM the backend TO the client, but it is not called from anywhere. This could be a method of signaling that a notification was pending, and sending out-of-band data FROm the client TO the backend could be used for cancelling a query. out-of-band data causes a convenient signal to the process on the other end, which can easily be used to handle these cases. -- Bruce Momjian | 830 Blythe Avenue maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026 + If your life is a hard drive, | (610) 353-9879(w) + Christ can be your backup. | (610) 853-3000(h)
-
Re: [HACKERS] Proposal for async support in libpq
Maarten Boekhold <maartenb@dutepp0.et.tudelft.nl> — 1998-04-18T10:10:47Z
> In backend/libpq/pqcomm.c, I see pg_sendoob() which sends out-of-band > data FROM the backend TO the client, but it is not called from anywhere. > > This could be a method of signaling that a notification was pending, and > sending out-of-band data FROm the client TO the backend could be used > for cancelling a query. > > out-of-band data causes a convenient signal to the process on the other > end, which can easily be used to handle these cases. Wasn't the problem with OOB data that java doesn't support this? I remember that OOB data has come up before on this list a long time ago, and that at that time some java bloke (peter?) started to complain :) Maarten _____________________________________________________________________________ | TU Delft, The Netherlands, Faculty of Information Technology and Systems | | Department of Electrical Engineering | | Computer Architecture and Digital Technique section | | M.Boekhold@et.tudelft.nl | -----------------------------------------------------------------------------
-
Re: [HACKERS] Proposal for async support in libpq
Peter T Mount <psqlhack@maidast.demon.co.uk> — 1998-04-18T10:34:25Z
On Sat, 18 Apr 1998, Bruce Momjian wrote: > In backend/libpq/pqcomm.c, I see pg_sendoob() which sends out-of-band > data FROM the backend TO the client, but it is not called from anywhere. > > This could be a method of signaling that a notification was pending, and > sending out-of-band data FROm the client TO the backend could be used > for cancelling a query. > > out-of-band data causes a convenient signal to the process on the other > end, which can easily be used to handle these cases. Just a quick question: If you have an OOB packet sent to the backend, how would we handle the case where a row is being sent to the backend, but the OOB packet comes in the middle of it? It may sound like a silly question, but I'm thinking if a client is on the end of a slow network connection, then the packet containing the row could become fragmented, and the OOB packet could get in the way. Anyhow, I'm trying to find out how to implement OOB in Java. I know it's there, as I've seen it in the past. Just can't find it at the moment. -- Peter T Mount petermount@earthling.net or pmount@maidast.demon.co.uk Main Homepage: http://www.demon.co.uk/finder Work Homepage: http://www.maidstone.gov.uk Work EMail: peter@maidstone.gov.uk
-
Re: [HACKERS] Proposal for async support in libpq
Peter T Mount <psqlhack@maidast.demon.co.uk> — 1998-04-18T10:40:45Z
On Sat, 18 Apr 1998, Maarten Boekhold wrote: > Wasn't the problem with OOB data that java doesn't support this? I > remember that OOB data has come up before on this list a long time ago, > and that at that time some java bloke (peter?) started to complain :) I said at the time, that I wasn't certain that Java did or didn't support it. Since then, I have noticed references to it, but I've lost them since. I'm delving into the docs as I type, looking for this. -- Peter T Mount petermount@earthling.net or pmount@maidast.demon.co.uk Main Homepage: http://www.demon.co.uk/finder Work Homepage: http://www.maidstone.gov.uk Work EMail: peter@maidstone.gov.uk
-
Re: [HACKERS] Proposal for async support in libpq
Bruce Momjian <maillist@candle.pha.pa.us> — 1998-04-19T01:16:59Z
> > > In backend/libpq/pqcomm.c, I see pg_sendoob() which sends out-of-band > > data FROM the backend TO the client, but it is not called from anywhere. > > > > This could be a method of signaling that a notification was pending, and > > sending out-of-band data FROm the client TO the backend could be used > > for cancelling a query. > > > > out-of-band data causes a convenient signal to the process on the other > > end, which can easily be used to handle these cases. > > Wasn't the problem with OOB data that java doesn't support this? I > remember that OOB data has come up before on this list a long time ago, > and that at that time some java bloke (peter?) started to complain :) I sure don't remember this topic, or anyone complaining about OOB. -- Bruce Momjian | 830 Blythe Avenue maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026 + If your life is a hard drive, | (610) 353-9879(w) + Christ can be your backup. | (610) 853-3000(h)
-
Re: [HACKERS] Proposal for async support in libpq
Bruce Momjian <maillist@candle.pha.pa.us> — 1998-04-19T01:44:02Z
> Just a quick question: If you have an OOB packet sent to the backend, how > would we handle the case where a row is being sent to the backend, but the > OOB packet comes in the middle of it? > > It may sound like a silly question, but I'm thinking if a client is on the > end of a slow network connection, then the packet containing the row could > become fragmented, and the OOB packet could get in the way. > > Anyhow, I'm trying to find out how to implement OOB in Java. I know it's > there, as I've seen it in the past. Just can't find it at the moment. > Because it is TCP/IP, the packets are re-assembled, so you can't get the OOB inside a normal packet. It is not like UDP. Second, the OOB data does not arrive in the normal data stream, but must be read by specifiying the MSG_OOB flag to the the recv() system. One issue raised by Stevens' "Unix Network Programming"(p. 333) is that the OOB signal(SIGURG) can arrive before the data is ready to be read. I have the Stevens' book, and it will probably be required to get this working properly. -- Bruce Momjian | 830 Blythe Avenue maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026 + If your life is a hard drive, | (610) 353-9879(w) + Christ can be your backup. | (610) 853-3000(h)
-
Re: [HACKERS] Proposal for async support in libpq
David Gould <dg@illustra.com> — 1998-05-16T07:09:05Z
> > Bruce Momjian <maillist@candle.pha.pa.us> writes: > > > This all looks good. Another thing we really need it to be able to > > > cancel queries. This would be a big win, and looks like it could fit > > > into the scheme here. > > > > I thought about proposing a PQcancelAsync that would cancel the active > > query-in-progress. But that would require support on the backend side, > > and I am far from competent to make it happen. (libpq is simple enough > > that I'm not afraid to rewrite it, but making major mods to the backend > > is another story. I just got here this week...) > > > > If anyone who does know what they're doing is willing to make the > > necessary backend mods, I'm all for it. The libpq side would be > > easy enough. > > > > How would such cancellation interact with transactions, btw? Would > > you expect it to roll back only the current command, or abort the > > whole transaction? We'd also have to consider corner cases, like > > when the backend has already finished the query by the time it gets > > the cancel request. > > It is pretty easy, just an elog(ERROR) would do it. The issue is > allowing the backend to see the request. We can put some checks in > tcop/postgres.c as it moves from module to module, and something in the > executor to check for the cancel, and do an elog(ERROR). It would be > nice if it arrived as out-of-band data, so we could check for input > quickly without having to actually process it if it is not a cancel > notification. > > The out-of-band data will send a SIGURG signal to the backend, and we > can set a global variable, and check the variable at various places. > > To do all this, we need to be able to send a query, and not have it > block, and it seems you are giving us this capability. > > You supply the indication to the backend, and I will see that the > backend processes it properly. Old news I know, but I was saving it to follow up and then ... I agree completely with Bruces proposal for handling this in the back-end. I have recently done something very similar for another database product. The important points are: - the signal handler can only set a single global variable. No other action is to be done in the handler. - the variable is to be tested only at well defined times where the recovery from an elog() can be handled correctly. It is nice if this test is "frequent, but not too frequent". At scan begin time is fairly good, and for large scans perhaps every few pages. Every row is too often. When stepping to a new plan node is also good. - There should be a further global flag to disable recognition of the cancel. This is used for example during an in progress elog() -> cleanup sequence. The cleanup code is not really reentrant so an elog() in the middle of an elog is likely to leave an inconsistant state. -dg David Gould dg@illustra.com 510.628.3783 or 510.305.9468 Informix Software (No, really) 300 Lakeside Drive Oakland, CA 94612 "Of course, someone who knows more about this will correct me if I'm wrong, and someone who knows less will correct me if I'm right." --David Palmer (palmer@tybalt.caltech.edu)