Thread

1. Re: [HACKERS] Re: [INTERFACES] retrieving varchar size

   Tom Lane <tgl@sss.pgh.pa.us> — 1998-04-24T14:50:51Z

   Bruce Momjian <maillist@candle.pha.pa.us> writes:
   > Added to TODO:
   > 	* Add pg_attribute.atttypmod/Resdom->restypmod to PGresult structure
   > This is a good suggestion.
   
   This will require a frontend/backend protocol change, no?
   
   If so, right now would be a great time to address it; I'm about halfway
   through rewriting libpq for the asynchronous-query support we discussed
   last week, and would be happy to make the client-side mods while I still
   have the code in my head.
   
   As long as we are opening up the protocol, there is an incredibly grotty
   hack in libpq that I'd like to get rid of.  It's hard for me to be
   sure whether it's even necessary, but: when libpq gets a 'C' response
   (which the documentation says is a "completed response") it assumes that
   this is *not* the end of the transaction, and that the only way to be
   sure that everything's been read is to send an empty query and wait for
   the empty query's 'I' response to be returned.
   
   	case 'C':		/* portal query command, no rows returned */
   		/*
   		 * since backend may produce more than one result
   		 * for some commands need to poll until clear.
   		 * Send an empty query down, and keep reading out of
   		 * the pipe until an 'I' is received.
   		 */
   
   Does this ring a bell with anyone?  I'm prepared to believe that it's
   useless code, but have no easy way to be sure.
   
   Needless to say, if there really is an ambiguity then the *right* answer
   is to fix the protocol so that the end of a query/response cycle is
   unambiguously determinable.  It looks to me like this hack is costing us
   an extra round trip to the server for every ordinary query.  That sucks.
   
   			regards, tom lane
   
   

2. Re: [HACKERS] Re: [INTERFACES] retrieving varchar size

   Byron Nikolaidis <byronn@insightdist.com> — 1998-04-24T15:12:26Z

   Yes,  it rings a bell alright,
   
   When you execute a multiple query (denoted by semicolans) like "set geqo to
   'off'; show datestyle; select * from table", you get that multiple returns and
   MUST read until you get the 'I'.  If you don't, your screwed the next time you
   try and read anything cause all that stuff is still in the pipe.
   
   Question though, I didnt think my request would have caused a major protocol
   change.  I though that the '-1' would simply be replaced by the correct size?
   
   Byron
   
   
   Tom Lane wrote:
   
   > Bruce Momjian <maillist@candle.pha.pa.us> writes:
   > > Added to TODO:
   > >       * Add pg_attribute.atttypmod/Resdom->restypmod to PGresult structure
   > > This is a good suggestion.
   >
   > This will require a frontend/backend protocol change, no?
   >
   > If so, right now would be a great time to address it; I'm about halfway
   > through rewriting libpq for the asynchronous-query support we discussed
   > last week, and would be happy to make the client-side mods while I still
   > have the code in my head.
   >
   > As long as we are opening up the protocol, there is an incredibly grotty
   > hack in libpq that I'd like to get rid of.  It's hard for me to be
   > sure whether it's even necessary, but: when libpq gets a 'C' response
   > (which the documentation says is a "completed response") it assumes that
   > this is *not* the end of the transaction, and that the only way to be
   > sure that everything's been read is to send an empty query and wait for
   > the empty query's 'I' response to be returned.
   >
   >         case 'C':               /* portal query command, no rows returned */
   >                 /*
   >                  * since backend may produce more than one result
   >                  * for some commands need to poll until clear.
   >                  * Send an empty query down, and keep reading out of
   >                  * the pipe until an 'I' is received.
   >                  */
   >
   > Does this ring a bell with anyone?  I'm prepared to believe that it's
   > useless code, but have no easy way to be sure.
   >
   > Needless to say, if there really is an ambiguity then the *right* answer
   > is to fix the protocol so that the end of a query/response cycle is
   > unambiguously determinable.  It looks to me like this hack is costing us
   > an extra round trip to the server for every ordinary query.  That sucks.
   >
   >                         regards, tom lane
   
   
   
   
   

3. Re: [HACKERS] Re: [INTERFACES] retrieving varchar size

   Bruce Momjian <maillist@candle.pha.pa.us> — 1998-04-26T02:30:25Z

   > 
   > Yes,  it rings a bell alright,
   > 
   > When you execute a multiple query (denoted by semicolans) like "set geqo to
   > 'off'; show datestyle; select * from table", you get that multiple returns and
   > MUST read until you get the 'I'.  If you don't, your screwed the next time you
   > try and read anything cause all that stuff is still in the pipe.
   
   Good point.  If we don't send the empty query, the queued up results get
   out of sync with the requests.
   
   One solution is to handle it the way psql does.  It keeps track of the
   quotes, backslashes, and semicolons in the input string, and sends just
   one query each time to the backend, and prints the results.
   
   Now, with libpq, I think the proper solution would be to scan the input
   string, and count the number of queries being send, send the whole
   strings (with the multiple queries) and retrieve that many answers from
   the backend, discarding all but the last result.  If you do that, I can
   remove the stuff from psql.c.
   
   > 
   > Question though, I didnt think my request would have caused a major protocol
   > change.  I though that the '-1' would simply be replaced by the correct size?
   
   Well, the -1 is in attlen, which is the type length.  text, char,
   varchar are all varlena(variable length)/-1.  atttypmod is the length
   specified at attribute creation time.  It is similar, but not the same
   as the length, and trying to put the typmod in the length field really
   messes up the clarity of what is going on.  We added atttypmod to
   clarify the code in the backend, and it should be sent to the front end.
   Soon, maybe will have atttypmod specifiying the precision of DECIMAL, or
   currency of MONEY.
   
   As far as adding atttypmod to libpq, I say do it.  If you look in the
   backend's BeginCommand(), under the Remote case label, you will see it
   sending the atttypid to the front end, using the TupleDesc that was
   passed to it.  Just after sending the atttyplen, I can send the
   atttypmod value, which is an int16.  I can do all the backend changes. 
   There are a few places where this would have to be changed in the
   backend.
   
   Other front-end libraries reading this protocol will have to change to
   to accept this field.
   
   -- 
   Bruce Momjian                          |  830 Blythe Avenue
   maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
     +  If your life is a hard drive,     |  (610) 353-9879(w)
     +  Christ can be your backup.        |  (610) 853-3000(h)
   
   

4. Re: [HACKERS] Re: [INTERFACES] retrieving varchar size

   Peter T Mount <psqlhack@retep.org.uk> — 1998-04-26T10:25:23Z

   On Sat, 25 Apr 1998, Bruce Momjian wrote:
   
   > > 
   > > Yes,  it rings a bell alright,
   > > 
   > > When you execute a multiple query (denoted by semicolans) like "set geqo to
   > > 'off'; show datestyle; select * from table", you get that multiple returns and
   > > MUST read until you get the 'I'.  If you don't, your screwed the next time you
   > > try and read anything cause all that stuff is still in the pipe.
   > 
   > Good point.  If we don't send the empty query, the queued up results get
   > out of sync with the requests.
   > 
   > One solution is to handle it the way psql does.  It keeps track of the
   > quotes, backslashes, and semicolons in the input string, and sends just
   > one query each time to the backend, and prints the results.
   > 
   > Now, with libpq, I think the proper solution would be to scan the input
   > string, and count the number of queries being send, send the whole
   > strings (with the multiple queries) and retrieve that many answers from
   > the backend, discarding all but the last result.  If you do that, I can
   > remove the stuff from psql.c.
   
   I think for libpq, that would be a good idea, but it would mean that there
   is a difference in behaviour between the interfaces.
   
   The JDBC spec allows for multiple ResultSet's to be returned from a query,
   and our driver handles this already.
   
   Now is this the client libpq, or the backend libpq you are thinking of
   changing? If it's the backend one, then this will break JDBC with multiple
   result sets.
   
   > > Question though, I didnt think my request would have caused a major protocol
   > > change.  I though that the '-1' would simply be replaced by the correct size?
   > 
   > Well, the -1 is in attlen, which is the type length.  text, char,
   > varchar are all varlena(variable length)/-1.  atttypmod is the length
   > specified at attribute creation time.  It is similar, but not the same
   > as the length, and trying to put the typmod in the length field really
   > messes up the clarity of what is going on.  We added atttypmod to
   > clarify the code in the backend, and it should be sent to the front end.
   > Soon, maybe will have atttypmod specifiying the precision of DECIMAL, or
   > currency of MONEY.
   
   That would be useful.
   
   > As far as adding atttypmod to libpq, I say do it.  If you look in the
   > backend's BeginCommand(), under the Remote case label, you will see it
   > sending the atttypid to the front end, using the TupleDesc that was
   > passed to it.  Just after sending the atttyplen, I can send the
   > atttypmod value, which is an int16.  I can do all the backend changes. 
   > There are a few places where this would have to be changed in the
   > backend.
   > 
   > Other front-end libraries reading this protocol will have to change to
   > to accept this field.
   
   As soon as you do it, I'll convert JDBC.
   
   -- 
   Peter T Mount  peter@retep.org.uk or petermount@earthling.net
   Main Homepage: http://www.demon.co.uk/finder (moving soon to www.retep.org.uk)
   ************ Someday I may rebuild this signature completely ;-) ************
   Work Homepage: http://www.maidstone.gov.uk Work EMail: peter@maidstone.gov.uk
   
   
   

5. Re: [HACKERS] Re: [INTERFACES] retrieving varchar size

   Bruce Momjian <maillist@candle.pha.pa.us> — 1998-04-26T14:24:29Z

   > > One solution is to handle it the way psql does.  It keeps track of the
   > > quotes, backslashes, and semicolons in the input string, and sends just
   > > one query each time to the backend, and prints the results.
   > > 
   > > Now, with libpq, I think the proper solution would be to scan the input
   > > string, and count the number of queries being send, send the whole
   > > strings (with the multiple queries) and retrieve that many answers from
   > > the backend, discarding all but the last result.  If you do that, I can
   > > remove the stuff from psql.c.
   > 
   > I think for libpq, that would be a good idea, but it would mean that there
   > is a difference in behaviour between the interfaces.
   > 
   > The JDBC spec allows for multiple ResultSet's to be returned from a query,
   > and our driver handles this already.
   
   Oh.  That prevents us from changing the backend to ignore returning more
   than one result for multiple queries in a PQexec.  Perhaps we need a new
   return query protocol character like 'J' to denote query returns that
   are not the LAST return, so libpq can throw them away, and jdbc and
   process them as normal, but also figure out when it gets the last one.
   
   
   > 
   > Now is this the client libpq, or the backend libpq you are thinking of
   > changing? If it's the backend one, then this will break JDBC with multiple
   > result sets.
   > 
   > > > Question though, I didnt think my request would have caused a major protocol
   > > > change.  I though that the '-1' would simply be replaced by the correct size?
   > > 
   > > Well, the -1 is in attlen, which is the type length.  text, char,
   > > varchar are all varlena(variable length)/-1.  atttypmod is the length
   > > specified at attribute creation time.  It is similar, but not the same
   > > as the length, and trying to put the typmod in the length field really
   > > messes up the clarity of what is going on.  We added atttypmod to
   > > clarify the code in the backend, and it should be sent to the front end.
   > > Soon, maybe will have atttypmod specifiying the precision of DECIMAL, or
   > > currency of MONEY.
   > 
   > That would be useful.
   > 
   > > As far as adding atttypmod to libpq, I say do it.  If you look in the
   > > backend's BeginCommand(), under the Remote case label, you will see it
   > > sending the atttypid to the front end, using the TupleDesc that was
   > > passed to it.  Just after sending the atttyplen, I can send the
   > > atttypmod value, which is an int16.  I can do all the backend changes. 
   > > There are a few places where this would have to be changed in the
   > > backend.
   > > 
   > > Other front-end libraries reading this protocol will have to change to
   > > to accept this field.
   > 
   > As soon as you do it, I'll convert JDBC.
   > 
   > -- 
   > Peter T Mount  peter@retep.org.uk or petermount@earthling.net
   > Main Homepage: http://www.demon.co.uk/finder (moving soon to www.retep.org.uk)
   > ************ Someday I may rebuild this signature completely ;-) ************
   > Work Homepage: http://www.maidstone.gov.uk Work EMail: peter@maidstone.gov.uk
   > 
   > 
   
   
   -- 
   Bruce Momjian                          |  830 Blythe Avenue
   maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
     +  If your life is a hard drive,     |  (610) 353-9879(w)
     +  Christ can be your backup.        |  (610) 853-3000(h)
   
   

6. Re: [HACKERS] Re: [INTERFACES] retrieving varchar size

   Peter T Mount <psqlhack@retep.org.uk> — 1998-04-26T15:59:33Z

   On Sun, 26 Apr 1998, Bruce Momjian wrote:
   
   [snip]
   
   > > I think for libpq, that would be a good idea, but it would mean that there
   > > is a difference in behaviour between the interfaces.
   > > 
   > > The JDBC spec allows for multiple ResultSet's to be returned from a query,
   > > and our driver handles this already.
   > 
   > Oh.  That prevents us from changing the backend to ignore returning more
   > than one result for multiple queries in a PQexec.  Perhaps we need a new
   > return query protocol character like 'J' to denote query returns that
   > are not the LAST return, so libpq can throw them away, and jdbc and
   > process them as normal, but also figure out when it gets the last one.
   
   That should be easy enough to implement.
   
   -- 
   Peter T Mount  peter@retep.org.uk or petermount@earthling.net
   Main Homepage: http://www.demon.co.uk/finder (moving soon to www.retep.org.uk)
   ************ Someday I may rebuild this signature completely ;-) ************
   Work Homepage: http://www.maidstone.gov.uk Work EMail: peter@maidstone.gov.uk