Thread

1. Re: [mail] Re: Windows Build System

   Dave Page <dpage@vale-housing.co.uk> — 2003-01-29T16:58:57Z

   
   > -----Original Message-----
   > From: Vince Vielhaber [mailto:vev@michvhf.com] 
   > Sent: 29 January 2003 16:57
   > To: Katie Ward
   > Cc: Dave Page; Tom Lane; Curtis Faith; pgsql-hackers@postgresql.org
   > Subject: RE: [mail] Re: [HACKERS] Windows Build System
   > 
   > 
   > The code's been available for what a week or two?  Do you 
   > actually think that can be considered conclusive by any standard?
   
   Public beta testing (but closed source) has been going on for some
   months.
   
   Regards, Dave.
   
   

2. Re: [mail] Re: Windows Build System

   Vince Vielhaber <vev@michvhf.com> — 2003-01-29T17:12:34Z

   On Wed, 29 Jan 2003, Dave Page wrote:
   
   > > The code's been available for what a week or two?  Do you
   > > actually think that can be considered conclusive by any standard?
   >
   > Public beta testing (but closed source) has been going on for some
   > months.
   
   So you've been running these unscientific tests you're telling us
   about being so successful for "some months"?
   
   Vince.
   -- 
    Fast, inexpensive internet service 56k and beyond!  http://www.pop4.net/
      http://www.meanstreamradio.com       http://www.unknown-artists.com
            Internet radio: It's not file sharing, it's just radio.
   
   
   

3. Re: [mail] Re: Windows Build System

   James Hubbard <jhubbard@mcs.uvawise.edu> — 2003-01-29T17:36:36Z

   Vince Vielhaber wrote:
   > On Wed, 29 Jan 2003, Dave Page wrote:
   > 
   > 
   >>>The code's been available for what a week or two?  Do you
   >>>actually think that can be considered conclusive by any standard?
   >>
   >>Public beta testing (but closed source) has been going on for some
   >>months.
   > 
   > 
   > So you've been running these unscientific tests you're telling us
   > about being so successful for "some months"?
   > 
   > Vince.
   
   I open my mouth and insert foot:  Where do I get any of these scientific 
   tests to determine if the latest and greatest 7.3.x will not fall down on my 
   favorite Unix?
   
   James Hubbard
   
   
   

4. Re: [mail] Re: Windows Build System

   Vince Vielhaber <vev@michvhf.com> — 2003-01-29T18:04:11Z

   On Wed, 29 Jan 2003, James Hubbard wrote:
   
   > Vince Vielhaber wrote:
   > > On Wed, 29 Jan 2003, Dave Page wrote:
   > >
   > >
   > >>>The code's been available for what a week or two?  Do you
   > >>>actually think that can be considered conclusive by any standard?
   > >>
   > >>Public beta testing (but closed source) has been going on for some
   > >>months.
   > >
   > >
   > > So you've been running these unscientific tests you're telling us
   > > about being so successful for "some months"?
   > >
   > > Vince.
   >
   > I open my mouth and insert foot:  Where do I get any of these scientific
   > tests to determine if the latest and greatest 7.3.x will not fall down on my
   > favorite Unix?
   
   If you're looking for a tool to test with, there was an announcement here
   not too long ago for one.  But it goes beyond just running a test suite
   against it.  Many of the available tools are designed to test what works
   and how well it works.  Testing goes beyond that.  You want to know what
   doesn't work, does the database return to a normal state if the unthinkable
   happens (eg. Tom's suggestion of yanking the plug), how about loss of
   network communications or sudden intermittant communication?  Or the
   function that may not be checking its input that well - when it fails is
   everything ok or did that transaction someone else was in the middle of
   get blown away?
   
   A gal that used to do MSDOS testing for MS (Jen something, don't recall
   her last name) would pull a floppy out in the middle of read or write
   and found a certain sequence would either hose the floppy, get the system
   to reboot (don't recall the exact details, it's been YEARS).
   
   Vince.
   -- 
    Fast, inexpensive internet service 56k and beyond!  http://www.pop4.net/
      http://www.meanstreamradio.com       http://www.unknown-artists.com
            Internet radio: It's not file sharing, it's just radio.
   
   
   

5. Re: [mail] Re: Windows Build System

   Justin Clift <justin@postgresql.org> — 2003-01-29T18:14:33Z

   James Hubbard wrote:
   <snip>
   > I open my mouth and insert foot:  Where do I get any of these scientific 
   > tests to determine if the latest and greatest 7.3.x will not fall down 
   > on my favorite Unix?
   
   For Open Source benchmarks, there is:
   
   Open Source Database Benchmark:
   http://osdb.sf.net
   
   With this, you *want* to use the latest CVS version, as that can 
   generate it's own datasets of any size.  The older, released versions 
   couldn't and you had to download databases of limited size.
   
   
   Database Opensource Test Suite:
   http://ltp.sourceforge.net/dotshowto.php
   
   This works with DB2, Oracle, Sybase, MySQL, and PostgreSQL, and looks to 
   have been developed by IBM.  Haven't yet used this, but did notice that 
   the configuration instructions make no reference to upping the memory 
   buffers.  i.e. all of the tests they've done were probably with the 
   defaults (yuck!)
   
   Emailed this group yesterday asking if they're open to suggestions for 
   improvement, and they said they definitely are.  If anyone has specific 
   they'd like to let them know, they do seem open to it.
   
   
   A commercial solution that people often mention is Benchmark Factory:
   
   http://www.benchmarkfactory.com
   
   Haven't personally used it, although it's apparently the software that 
   Great Bridge used for all of their testing.
   
   
   Hope this helps.
   
   Regards and best wishes,
   
   Justin Clift
   
   
   > James Hubbard
   
   
   -- 
   "My grandfather once told me that there are two kinds of people: those
   who work and those who take the credit. He told me to try to be in the
   first group; there was less competition there."
   - Indira Gandhi
   
   
   

6. Re: [mail] Re: Windows Build System

   James Hubbard <jhubbard@mcs.uvawise.edu> — 2003-01-29T18:53:53Z

   Vince Vielhaber wrote:
   >>><snip>
   >>>
   >>>So you've been running these unscientific tests you're telling us
   >>>about being so successful for "some months"?
   >>>
   >>>Vince.
   >>
   >>I open my mouth and insert foot:  Where do I get any of these scientific
   >>tests to determine if the latest and greatest 7.3.x will not fall down on my
   >>favorite Unix?
   > 
   > 
   > If you're looking for a tool to test with, there was an announcement here
   > not too long ago for one.  But it goes beyond just running a test suite
   > against it.  Many of the available tools are designed to test what works
   > and how well it works.  Testing goes beyond that.  You want to know what
   > doesn't work, does the database return to a normal state if the unthinkable
   > happens (eg. Tom's suggestion of yanking the plug), how about loss of
   > network communications or sudden intermittant communication?  Or the
   > function that may not be checking its input that well - when it fails is
   > everything ok or did that transaction someone else was in the middle of
   > get blown away?
   > 
   > A gal that used to do MSDOS testing for MS (Jen something, don't recall
   > her last name) would pull a floppy out in the middle of read or write
   > and found a certain sequence would either hose the floppy, get the system
   > to reboot (don't recall the exact details, it's been YEARS).
   > 
   
   I'm not disagreeing with you on testing.  I've seen the announcments. 
   Justin Clift just posted them again.  But, as far as I've seen there are no 
   real scientific tests that anyone here has posted.  I've seen the occasional 
   post with db_bench.  You asked "To what standards?"  I've not seen any 
   standards that are meaningful.  Maybe I'm just not looking.
   
   Any benchmarks/tests that someone posts are going to be subjective anyway. 
     No one seems to be using the same tool.  The osdb is step in the right 
   direction, but I've not really seen anyone using it.  The regressions are 
   the only thing that I can see and run. It would be nice if there were a few 
   people that had test setups that could post benchmarks/tests, so that we 
   could see how things look for each release.
   (i.e.:  on the 5GB test, it did this; when I cut the power and turned it 
   back onn it did this and this.)
   
   When I download, install, and use postgresql, I take it on faith that it 
   will perform as the developers say that it does. Maybe this is a bad thing, 
   but I don't think so    My use of it is very meager at the best so I don't 
   have a lot to worry about.  If I had loads of data and mission critical apps 
   I would probably test a lot, but I don't.
   
   All I'm saying is to cut them some slack and give them some ideas to test 
   until there is a really good testing/benchmarking tool that everyone can use 
   that won't be as subjective.
   
   I personally want this to succeed.  After having to use MySQL for a class 
   project, I don't really want to use it again. I had to use because it was 
   the only cross platform tool. Not everyone in the class was running linux or 
   xBSD, so I had to go with MySQL.   From what I've seen, It looks like I'll 
   have to anyhow because that's what many job ads are looking for.
   
   I believe Oracle used the excuse that PostgreSQL was unproven, when they 
   complained about its use for the .org registry. What we may think about 
   Windows being fragile and being a piece of crap doesn't really matter. 
   People are using it and it's at least doing they want.
   
   I've probably not said this before, but I appreciate all the hard work that 
   everyone puts into this project.
   
   James Hubbard