Thread

1. User control over psql error stream

   Karl O. Pinc <kop@meme.com> — 2012-11-15T20:53:24Z

   Hi,
   
   This patch gives the end user control over psql's
   error stream.  This allows a single psql session
   to use \o to send both errors and table output
   to multiple files.  Useful when capturing test output, etc.
   
   Control is provided via a new "estream" \pset.  Here's
   the docs.
   
   -------------------------<snip>------------------------
   estream
   
       Controls the output stream(s) used to report error messages. Value 
   must be one of: stderr (the default), which sends errors to the 
   standard error stream; query, which injects error messages into the 
   query result output stream; or both, which sends errors to both output 
   streams. "Error messages" are comprised of errors from psql and notice 
   messages and errors from the database server. 
   -------------------------<snip>------------------------
   
   Against head.
   
   psql-estream.patch       The patch.
   psql-estream_test.patch  Adds a regression test to test the patch.
   
   There's a number of problems with psql-estream_test.patch,
   the most notable is that it probably won't work on
   MS Windows because it uses /dev/null to avoid touching the
   host filesystem.   I'm not sure whether this should have
   a regression test and if so what the right way is to do it.
   
   Note that psql-stream.patch includes some re-writing of
   the docs for the psql \o option that goes slightly beyond
   the minimum change required to explain \pset estream's effects.
   
   Regards,
   
   Karl <kop@meme.com>
   Free Software:  "You don't pay back, you pay forward."
                    -- Robert A. Heinlein
   
   

2. Re: User control over psql error stream

   Peter Eisentraut <peter_e@gmx.net> — 2012-11-21T19:41:56Z

   On 11/15/12 3:53 PM, Karl O. Pinc wrote:
   > This patch gives the end user control over psql's
   > error stream.  This allows a single psql session
   > to use \o to send both errors and table output
   > to multiple files.  Useful when capturing test output, etc.
   
   What does this do that cannot in practice be achieved using shell
   redirection operators?
   
   
   

3. Re: User control over psql error stream

   Karl O. Pinc <kop@meme.com> — 2012-11-22T03:51:47Z

   On 11/21/2012 01:41:56 PM, Peter Eisentraut wrote:
   > On 11/15/12 3:53 PM, Karl O. Pinc wrote:
   > > This patch gives the end user control over psql's
   > > error stream.  This allows a single psql session
   > > to use \o to send both errors and table output
   > > to multiple files.  Useful when capturing test output, etc.
   > 
   > What does this do that cannot in practice be achieved using shell
   > redirection operators?
   
   I've 3 answers to this.  The short one, the long one,
   and the practical one.  Sorry to go on about it.
   
   The brief generic answer is:
   
   Easily capture error output associated with disparate
   input streams, interspersed with the data output associated with
   the disparate input streams, in separate files
   correlated with the input, in the case where transactions
   span multiple input streams.   That latter
   bit about transactions being the important part.
   
   Of course I could be wrong/not clever enough.
   
   
   The long generic answer is:
   
   Shell redirection works fine if the transactions do not span 
   input streams; a separate connection/psql instance can
   be used to redirect stderr intermixed with stdout as desired.  
   However when transactions span input streams then a single
   psql instance/db connection is required to maintain
   transaction state.  In such a case it is difficult
   to re-redirect stderr so that it's output appears
   intermixed with table data output (stdout) in
   time wise order while at the same time redirecting
   all output into files that correlate with the
   various input streams.  
   
   It's doable with something like:
   
     psql 2>&1 < myfifo | mydemultiplexer &
   
   where "myfifo" is a fifo fed the various input streams
   and "mydemultiplexer" is a some process that
   splits stdin into various output files as
   various bits of input are sent to psql.  But there
   must be a way to communicate with the mydemultiplexer,
   and you need to worry about synchronization of
   input with output.  And it's not clear to me how to
   write mydemultiplexer in shell either.
   
   Note that I've not actually tried the example above.
   
   
   The practical answer is that I want my dependency
   based test framework to work:
   
   http://gombemi.ccas.gwu.edu/cgi-bin/darcsweb.cgi?r=gombemi;a=tree;f=/
   db/test
   
   For an intro see:
   
   http://gombemi.ccas.gwu.edu/cgi-bin/darcsweb.cgi?
   r=gombemi;a=headblob;f=/db/test/test.mk
   
   I'm using this patch in my test framework.
   If I had to I think I could get away
   without it, but it would involve a bit of
   work, and more than a bit of additional
   complication.
   
   Although I've got (much) further work to do 
   if I'm to follow through on my notion to
   release this as a generic test framework,
   what's documented in test.mk works.  Probably.
   (Not bad for under 300 lines of actual code,
   comments excluded.  Perhaps under 200 if you
   eliminate blank lines and configuration for
   my real-world use.)
   
   Comments/suggestions are welcome.
   
   Regards,
   
   Karl <kop@meme.com>
   Free Software:  "You don't pay back, you pay forward."
                    -- Robert A. Heinlein
   
   
   
   

4. Re: User control over psql error stream

   Karl O. Pinc <kop@meme.com> — 2012-11-22T05:33:18Z

   On 11/21/2012 01:41:56 PM, Peter Eisentraut wrote:
   > On 11/15/12 3:53 PM, Karl O. Pinc wrote:
   > > This patch gives the end user control over psql's
   > > error stream.  This allows a single psql session
   > > to use \o to send both errors and table output
   > > to multiple files.  Useful when capturing test output, etc.
   > 
   > What does this do that cannot in practice be achieved using shell
   > redirection operators?
   
   To look at it from another angle, you need it for the
   same reason you need \o -- to redirect output
   to multiple places from within a single psql process.
   \o redirects stdout, but I'm interested in
   redirecting stderr.
   
   Which makes me think that instead of a \pset
   option a, say, \e, command is called for.
   This would do the same thing \o does only
   for stderr instead of stdout.  
   
   The problem comes
   when you want \e to send to the same place \o
   sends to, or vice versa.  You can't just compare textual paths
   because there's more than one way to write
   a path to an object in the filesystem.
   You need either a "special" argument
   to \e and \o -- very ungood as it breaks
   backwards compatibility with \o --
   or you need, perhaps, yet another \
   command to send stderr to where \o
   is going or, conversely, send \o
   to where stderr is going.  Perhaps\oe
   could send \o output to where stderr
   is going at the moment and \eo could
   send stderr output to where \o output
   is going at the moment?
   
   Sorry to ramble on.  I'm tired.
   
   Regards,
   
   Karl <kop@meme.com>
   Free Software:  "You don't pay back, you pay forward."
                    -- Robert A. Heinlein
   
   
   
   

5. Re: User control over psql error stream

   Karl O. Pinc <kop@meme.com> — 2012-12-07T20:50:48Z

   On 11/21/2012 11:33:18 PM, Karl O. Pinc wrote:
   > On 11/21/2012 01:41:56 PM, Peter Eisentraut wrote:
   > > On 11/15/12 3:53 PM, Karl O. Pinc wrote:
   > > > This patch gives the end user control over psql's
   > > > error stream.  This allows a single psql session
   > > > to use \o to send both errors and table output
   > > > to multiple files.  Useful when capturing test output, etc.
   > > 
   > > What does this do that cannot in practice be achieved using shell
   > > redirection operators?
   > 
   > To look at it from another angle, you need it for the
   > same reason you need \o -- to redirect output
   > to multiple places from within a single psql process.
   > \o redirects stdout, but I'm interested in
   > redirecting stderr.
   
   There is an interaction here with \set EXIT_ON_ERROR
   that may need additional documentation.  As the patch is,
   when \set EXIT_ON_ERROR	is used the error stream is sent
   to stderr in addition to the \pset estream setting.
   At least I think this is what is happening.
   This behavior seems correct to me.
   
   Before proceeding further I would like to wait
   for feedback regarding whether a \e command,
   per above and previous, is preferable to the existing patch's
   \pset estream.
   
   Regards,
   
   
   Karl <kop@meme.com>
   Free Software:  "You don't pay back, you pay forward."
                    -- Robert A. Heinlein