Thread

1. libpq URI and regression testing

   Alvaro Herrera <alvherre@alvh.no-ip.org> — 2012-04-17T13:47:26Z

   Hi,
   
   When I committed Alex Shulgin's patch to add URI support to libpq, I
   included the test harness as well.  However, due to it being in a
   separate subdirectory that did not previously had tests, it's not being
   run by buildfarm.  
   
   It's not considered in "make installcheck-world" either.
   
   What's the preferred way to make it automatically tested as much as
   possible?  I know the buildfarm does not run "installcheck-world", so if
   we want it there, it'd need a bit more code on the client side.  I think
   it would be wise to have it also run on installcheck-world.
   
   Hmm.  Just had this thought: not all platform support the same socket
   types.  Maybe we should have separated the .in file in three parts:
   IPv4, IPv6, unix-domain socket.  That way each platform would only run
   tests that pertain to it.  Right now there's a single "regress.in" file
   that lists all the tests.
   
   Opinions?
   
   -- 
   Álvaro Herrera <alvherre@alvh.no-ip.org>
   
   

2. Re: libpq URI and regression testing

   Peter Eisentraut <peter_e@gmx.net> — 2012-04-17T18:41:04Z

   On tis, 2012-04-17 at 10:47 -0300, Alvaro Herrera wrote:
   > What's the preferred way to make it automatically tested as much as
   > possible?  I know the buildfarm does not run "installcheck-world", so if
   > we want it there, it'd need a bit more code on the client side.  I think
   > it would be wise to have it also run on installcheck-world.
   
   It was agreed during the patch discussion that it shouldn't be run
   automatically.
   
   > Hmm.  Just had this thought: not all platform support the same socket
   > types.  Maybe we should have separated the .in file in three parts:
   > IPv4, IPv6, unix-domain socket.  That way each platform would only run
   > tests that pertain to it.  Right now there's a single "regress.in" file
   > that lists all the tests.
   
   That's one reason for that, but there are probably others in the way of
   making this fully portable and automatable.
   
   
   
   

3. Re: libpq URI and regression testing

   Alvaro Herrera <alvherre@commandprompt.com> — 2012-04-17T18:47:55Z

   Excerpts from Peter Eisentraut's message of mar abr 17 15:41:04 -0300 2012:
   > On tis, 2012-04-17 at 10:47 -0300, Alvaro Herrera wrote:
   > > What's the preferred way to make it automatically tested as much as
   > > possible?  I know the buildfarm does not run "installcheck-world", so if
   > > we want it there, it'd need a bit more code on the client side.  I think
   > > it would be wise to have it also run on installcheck-world.
   > 
   > It was agreed during the patch discussion that it shouldn't be run
   > automatically.
   
   Oh, okay.  I didn't notice that.  I guess we should issue a
   call-for-testing, then, so that we ensure it works (FSVO works) in all
   (FSVO all) platforms.
   
   > > Hmm.  Just had this thought: not all platform support the same socket
   > > types.  Maybe we should have separated the .in file in three parts:
   > > IPv4, IPv6, unix-domain socket.  That way each platform would only run
   > > tests that pertain to it.  Right now there's a single "regress.in" file
   > > that lists all the tests.
   > 
   > That's one reason for that, but there are probably others in the way of
   > making this fully portable and automatable.
   
   Hmm.
   
   -- 
   Álvaro Herrera <alvherre@commandprompt.com>
   The PostgreSQL Company - Command Prompt, Inc.
   PostgreSQL Replication, Consulting, Custom Development, 24x7 support
   
   

4. Re: libpq URI and regression testing

   Andrew Dunstan <andrew@dunslane.net> — 2012-04-17T19:03:50Z

   
   On 04/17/2012 02:47 PM, Alvaro Herrera wrote:
   > Excerpts from Peter Eisentraut's message of mar abr 17 15:41:04 -0300 2012:
   >> On tis, 2012-04-17 at 10:47 -0300, Alvaro Herrera wrote:
   >>> What's the preferred way to make it automatically tested as much as
   >>> possible?  I know the buildfarm does not run "installcheck-world", so if
   >>> we want it there, it'd need a bit more code on the client side.  I think
   >>> it would be wise to have it also run on installcheck-world.
   >> It was agreed during the patch discussion that it shouldn't be run
   >> automatically.
   > Oh, okay.  I didn't notice that.  I guess we should issue a
   > call-for-testing, then, so that we ensure it works (FSVO works) in all
   > (FSVO all) platforms.
   >
   >>> Hmm.  Just had this thought: not all platform support the same socket
   >>> types.  Maybe we should have separated the .in file in three parts:
   >>> IPv4, IPv6, unix-domain socket.  That way each platform would only run
   >>> tests that pertain to it.  Right now there's a single "regress.in" file
   >>> that lists all the tests.
   >> That's one reason for that, but there are probably others in the way of
   >> making this fully portable and automatable.
   
   
   This test setup also appears to labor under the illusion that we live in 
   a Unix-only world. And for no good reason that I can tell. The shell 
   script should be ripped out and replaced by a perl script which could 
   actually be used on any windows build host. The MSVC build system also 
   needs adjusting to make it build the test driver, at least.
   
   cheers
   
   andrew
   
   
   

5. Re: libpq URI and regression testing

   Alvaro Herrera <alvherre@commandprompt.com> — 2012-04-17T19:50:23Z

   Excerpts from Andrew Dunstan's message of mar abr 17 16:03:50 -0300 2012:
   
   > >> That's one reason for that, but there are probably others in the way of
   > >> making this fully portable and automatable.
   > 
   > This test setup also appears to labor under the illusion that we live in 
   > a Unix-only world. And for no good reason that I can tell. The shell 
   > script should be ripped out and replaced by a perl script which could 
   > actually be used on any windows build host. The MSVC build system also 
   > needs adjusting to make it build the test driver, at least.
   
   I'll see about it.
   
   -- 
   Álvaro Herrera <alvherre@commandprompt.com>
   The PostgreSQL Company - Command Prompt, Inc.
   PostgreSQL Replication, Consulting, Custom Development, 24x7 support
   
   

6. Re: libpq URI and regression testing

   Alex Shulgin <ash@commandprompt.com> — 2012-04-18T21:08:24Z

   Peter Eisentraut <peter_e@gmx.net> writes:
   
   > On tis, 2012-04-17 at 10:47 -0300, Alvaro Herrera wrote:
   >> What's the preferred way to make it automatically tested as much as
   >> possible?  I know the buildfarm does not run "installcheck-world", so if
   >> we want it there, it'd need a bit more code on the client side.  I think
   >> it would be wise to have it also run on installcheck-world.
   >
   > It was agreed during the patch discussion that it shouldn't be run
   > automatically.
   
   Hm... the testing program we've actually committed avoids any connection
   attempts and only deals with testing the parser routine.  Does it change
   that?
   
   --
   Regards,
   Alex
   
   

7. Re: libpq URI and regression testing

   Alex Shulgin <ash@commandprompt.com> — 2012-04-18T21:13:17Z

   Andrew Dunstan <andrew@dunslane.net> writes:
   
   >>> That's one reason for that, but there are probably others in the way of
   >>> making this fully portable and automatable.
   >
   > This test setup also appears to labor under the illusion that we live
   > in a Unix-only world. And for no good reason that I can tell. The
   > shell script should be ripped out and replaced by a perl script which
   > could actually be used on any windows build host. The MSVC build
   > system also needs adjusting to make it build the test driver, at
   > least.
   
   Good catch.  Attached is my first take at writing Perl: replaces the
   shell script, adds $libpq_uri_regress project to Mkvcbuild.pm.
   
   I don't have access to a win32 box unfortunately, so if anyone who does
   could try this out that'd be great.
   
   --
   Regards,
   Alex
   
   

8. Re: libpq URI and regression testing

   Peter Eisentraut <peter_e@gmx.net> — 2012-04-19T18:40:22Z

   On tor, 2012-04-19 at 00:13 +0300, Alex wrote:
   > +#!/usr/bin/env perl
   
   Don't do that.  Call the script using $(PERL) from the makefile.
   
   
   

9. Re: libpq URI and regression testing

   Alex Shulgin <ash@commandprompt.com> — 2012-04-19T20:06:05Z

   Peter Eisentraut <peter_e@gmx.net> writes:
   
   > On tor, 2012-04-19 at 00:13 +0300, Alex wrote:
   >> +#!/usr/bin/env perl
   >
   > Don't do that.  Call the script using $(PERL) from the makefile.
   
   Thank you for the suggestion.  Attached v2 does just this (while keeping
   a more commonly found shebang line in the perl script for running it w/o
   the makefile.)
   
   I figure src/tools/msvc/vcregress.pl will need to be updated too, but
   trying to model all this after ecpg regression tests I'm stuck at
   replicating ecpg_regression.proj for libpq's uri-regress.  I'd
   appreciate any help from the Windows guys at this point.
   
   --
   Alex
   
   

10. Re: libpq URI and regression testing

    Alvaro Herrera <alvherre@commandprompt.com> — 2012-07-06T21:03:11Z

    Excerpts from Alex's message of jue abr 19 17:06:05 -0300 2012:
    > Peter Eisentraut <peter_e@gmx.net> writes:
    > 
    > > On tor, 2012-04-19 at 00:13 +0300, Alex wrote:
    > >> +#!/usr/bin/env perl
    > >
    > > Don't do that.  Call the script using $(PERL) from the makefile.
    > 
    > Thank you for the suggestion.  Attached v2 does just this (while keeping
    > a more commonly found shebang line in the perl script for running it w/o
    > the makefile.)
    
    I've applied this to 9.3.  Andrew, can we have the non-MSVC buildfarm
    members running on the master branch also run "make installcheck" in
    src/interfaces/libpq?  If there are platform dependencies here, it would
    be good to know what they are; if we don't have the tests run
    automatically we will never know.
    
    We'll need to figure out some way for MSVC animals to run the tests as
    well.  Any takers?
    
    I think this is necessary, because otherwise it is quite clear that the
    tests are going to be kept failing forever.  They were already failing
    because of message style changes that didn't update the expected output.
    
    (I am unsure about pushing the sh to perl test harness conversion in
    9.2.  If anybody thinks it should be done, please discuss.)
    
    -- 
    Álvaro Herrera <alvherre@commandprompt.com>
    The PostgreSQL Company - Command Prompt, Inc.
    PostgreSQL Replication, Consulting, Custom Development, 24x7 support