Horton

Thread

  1. Possible make_oidjoins_check Security Issue

    Rod Taylor <pg@rbt.ca> — 2004-10-19T20:18:35Z

    http://secunia.com/advisories/12860/

  2. Re: Possible make_oidjoins_check Security Issue

    Neil Conway <neilc@samurai.com> — 2004-10-20T02:52:57Z

    On Wed, 2004-10-20 at 06:18, Rod Taylor wrote:
> http://secunia.com/advisories/12860/

This seems like a rather inconsequential problem, but it should be
fixed. The first two ideas that come to mind: use temporary files in
$PWD rather than /tmp, or create a subdirectory in /tmp to use for the
temporary files.

-Neil

  3. Re: Possible make_oidjoins_check Security Issue

    Alvaro Herrera <alvherre@dcc.uchile.cl> — 2004-10-20T03:55:29Z

    On Wed, Oct 20, 2004 at 12:52:57PM +1000, Neil Conway wrote:
> On Wed, 2004-10-20 at 06:18, Rod Taylor wrote:
> > http://secunia.com/advisories/12860/
> 
> This seems like a rather inconsequential problem, but it should be
> fixed. The first two ideas that come to mind: use temporary files in
> $PWD rather than /tmp, or create a subdirectory in /tmp to use for the
> temporary files.

Better, use mktemp(1).  The thread testing script already does it IIRC.

-- 
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
"Un poeta es un mundo encerrado en un hombre" (Victor Hugo)

  4. Re: Possible make_oidjoins_check Security Issue

    Tom Lane <tgl@sss.pgh.pa.us> — 2004-10-20T04:11:12Z

    Neil Conway <neilc@samurai.com> writes:
> On Wed, 2004-10-20 at 06:18, Rod Taylor wrote:
>> http://secunia.com/advisories/12860/

> This seems like a rather inconsequential problem,

Indeed, since ordinary users have no use for make_oidjoins_check.
It's surely very implausible that anyone would run it as root; and
even if someone did, the attacker cannot control what gets written.

> but it should be fixed. The first two ideas that come to mind: use
> temporary files in $PWD rather than /tmp, or create a subdirectory in
> /tmp to use for the temporary files.

I believe that the subdirectory idea is also vulnerable without great
care.

My inclination so far as the Red Hat packages are concerned is simply to
omit the contrib/findoidjoins files from the installed RPMs.

The patch originally proposed by trustix involved using mktemp(1), which
would be a great fix if mktemp(1) weren't so laughably unportable :-(
But in any case it's hard to see why we are expending RPM distro space
on this script in the first place.  I suspect that no one on the planet
except Bruce and myself have ever actually run this script.

			regards, tom lane

  5. Re: Possible make_oidjoins_check Security Issue

    Tom Lane <tgl@sss.pgh.pa.us> — 2004-10-20T04:31:11Z

    Alvaro Herrera <alvherre@dcc.uchile.cl> writes:
> Better, use mktemp(1).  The thread testing script already does it IIRC.

There are only two uses of mktemp(1) in our source tree: configure and
config.guess.  Both were gotten from elsewhere, and both jump through
some seriously unreadable hoops in order to achieve allegedly-portable
behavior.  mktemp(1) is simply not portable :-( ... the Single Unix Spec
refuses to touch it at all ...

			regards, tom lane

  6. Re: Possible make_oidjoins_check Security Issue

    Alvaro Herrera <alvherre@dcc.uchile.cl> — 2004-10-20T05:42:46Z

    On Wed, Oct 20, 2004 at 12:31:11AM -0400, Tom Lane wrote:
> Alvaro Herrera <alvherre@dcc.uchile.cl> writes:
> > Better, use mktemp(1).  The thread testing script already does it IIRC.
> 
> There are only two uses of mktemp(1) in our source tree: configure and
> config.guess.  Both were gotten from elsewhere, and both jump through
> some seriously unreadable hoops in order to achieve allegedly-portable
> behavior.

Huh, right.  I was remembering mkstemp(3), which is used in the thread
test (which is not a script after all ...)

config.guess usage surely is ugly ...

-- 
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
"Porque francamente, si para saber manejarse a uno mismo hubiera que
rendir examen... ¿Quién es el machito que tendría carnet?"  (Mafalda)

  7. Re: Possible make_oidjoins_check Security Issue

    Andrew Dunstan <andrew@dunslane.net> — 2004-10-20T11:54:27Z

    
Tom Lane wrote:

>I suspect that no one on the planet
>except Bruce and myself have ever actually run this script.
>
>
>  
>

Then why don't we just remove it? Problem solved ...

cheers

andrew

  8. Re: Possible make_oidjoins_check Security Issue

    Tom Lane <tgl@sss.pgh.pa.us> — 2004-10-20T14:38:25Z

    Andrew Dunstan <andrew@dunslane.net> writes:
> Tom Lane wrote:
>> I suspect that no one on the planet
>> except Bruce and myself have ever actually run this script.

> Then why don't we just remove it? Problem solved ...

Because it's a needed maintenance tool.  There isn't any particularly
good reason for it to get installed as though it were an interesting
program for users, though, so I think that this is mostly a matter of
poor packaging choices.  I am in fact intending to remove the
contrib/findoidjoins files from the set of stuff installed by Red Hat's
RPMs.

I suppose you could make an argument for moving this directory out of
contrib and putting it under src/tools instead, but that seems like more
work (and loss of CVS history) than it's worth.

			regards, tom lane

  9. Re: Possible make_oidjoins_check Security Issue

    Andrew Dunstan <andrew@dunslane.net> — 2004-10-20T15:23:43Z

    
Tom Lane wrote:

>Andrew Dunstan <andrew@dunslane.net> writes:
>  
>
>>Tom Lane wrote:
>>    
>>
>>>I suspect that no one on the planet
>>>except Bruce and myself have ever actually run this script.
>>>      
>>>
>
>  
>
>>Then why don't we just remove it? Problem solved ...
>>    
>>
>
>Because it's a needed maintenance tool.  There isn't any particularly
>good reason for it to get installed as though it were an interesting
>program for users, though, so I think that this is mostly a matter of
>poor packaging choices.  I am in fact intending to remove the
>contrib/findoidjoins files from the set of stuff installed by Red Hat's
>RPMs.
>
>I suppose you could make an argument for moving this directory out of
>contrib and putting it under src/tools instead, but that seems like more
>work (and loss of CVS history) than it's worth.
>
>
>  
>

Then maybe there's a case for removing findoidjoins from WANTED_DIRS in 
contrib/Makefile? I agree this issue is so trifling that it's not worth 
spending much energy on.

On a very slightly related note, I see that ipcclean (which is a shell 
script) is installed on Windows by "make install".  Do we want to fix 
that or trust the binary packagers to remove it?

cheers

andrew

  10. Re: Possible make_oidjoins_check Security Issue

    Bruce Momjian <pgman@candle.pha.pa.us> — 2004-10-26T14:47:40Z

    Tom Lane wrote:
> Neil Conway <neilc@samurai.com> writes:
> > On Wed, 2004-10-20 at 06:18, Rod Taylor wrote:
> >> http://secunia.com/advisories/12860/
> 
> > This seems like a rather inconsequential problem,
> 
> Indeed, since ordinary users have no use for make_oidjoins_check.
> It's surely very implausible that anyone would run it as root; and
> even if someone did, the attacker cannot control what gets written.
> 
> > but it should be fixed. The first two ideas that come to mind: use
> > temporary files in $PWD rather than /tmp, or create a subdirectory in
> > /tmp to use for the temporary files.
> 
> I believe that the subdirectory idea is also vulnerable without great
> care.

I believe the proper way to handle this is a new directory under /tmp. 
I use this in my local scripts:

	TMP=/tmp/$$
	OMASK="`umask`"
	umask 077
	if ! mkdir "$TMP"
	then    echo "Can't create temporary directory $TMP." 1>&2
	        exit 1
	fi
	umask "$OMASK"
	unset OMASK

It basically makes sure it creates a new directory under /tmp with a
umask that guarantees no one else can create a file in that directory. 
All temp files are accessed as $TMP/XXX.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

  11. Re: Possible make_oidjoins_check Security Issue

    Tom Lane <tgl@sss.pgh.pa.us> — 2004-10-26T14:58:47Z

    Bruce Momjian <pgman@candle.pha.pa.us> writes:
> I believe the proper way to handle this is a new directory under /tmp. 

It's definitely not worth the trouble.  I looked at what configure does
to make /tmp subdirectories portably, and it is spectacularly ugly
(not to mention long).  If make_oidjoins_check were a user-facing tool
that would be one thing, but it isn't ...

			regards, tom lane

  12. Re: [HACKERS] Possible make_oidjoins_check Security Issue

    Bruce Momjian <pgman@candle.pha.pa.us> — 2004-11-03T22:45:59Z

    Tom Lane wrote:
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > I believe the proper way to handle this is a new directory under /tmp. 
> 
> It's definitely not worth the trouble.  I looked at what configure does
> to make /tmp subdirectories portably, and it is spectacularly ugly
> (not to mention long).  If make_oidjoins_check were a user-facing tool
> that would be one thing, but it isn't ...

From a public relations perspective and a code reuse perspective I think
we should create temporary tables securely.  The attached applied patch
fixes contrib/findoidjoins/make_oidjoins_check.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

  13. Re: [HACKERS] Possible make_oidjoins_check Security Issue

    Bruce Momjian <pgman@candle.pha.pa.us> — 2004-11-03T22:53:22Z

    Bruce Momjian wrote:
> Tom Lane wrote:
> > Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > > I believe the proper way to handle this is a new directory under /tmp. 
> > 
> > It's definitely not worth the trouble.  I looked at what configure does
> > to make /tmp subdirectories portably, and it is spectacularly ugly
> > (not to mention long).  If make_oidjoins_check were a user-facing tool
> > that would be one thing, but it isn't ...
> 
> >From a public relations perspective and a code reuse perspective I think
> we should create temporary tables securely.  The attached applied patch

                             ^^^^^^
                              files

> fixes contrib/findoidjoins/make_oidjoins_check.

Sorry, meant temporary files.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

  14. Re: [HACKERS] Possible make_oidjoins_check Security Issue

    Tom Lane <tgl@sss.pgh.pa.us> — 2004-11-03T22:55:33Z

    Bruce Momjian <pgman@candle.pha.pa.us> writes:
> From a public relations perspective and a code reuse perspective I think
> we should create temporary tables securely.  The attached applied patch
> fixes contrib/findoidjoins/make_oidjoins_check.

... and creates issues of its own, such as attempting an rm -rf on
something that it shouldn't.  At the very least don't install the trap
until after creating the directory successfully.

I really think this is a waste of time though.  The current code creates
the temp files in the current directory, and if the bad guy has write
access on that directory you are already screwed (for instance, what's
to stop him from altering the script file itself to do anything at all
when you run it?).  I do not think that putting stuff back into /tmp is
an improvement; that just adds risks where none exist now.

			regards, tom lane

  15. Re: [HACKERS] Possible make_oidjoins_check Security Issue

    Bruce Momjian <pgman@candle.pha.pa.us> — 2004-11-03T23:07:19Z

    Tom Lane wrote:
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > From a public relations perspective and a code reuse perspective I think
> > we should create temporary tables securely.  The attached applied patch
> > fixes contrib/findoidjoins/make_oidjoins_check.
> 
> ... and creates issues of its own, such as attempting an rm -rf on
> something that it shouldn't.  At the very least don't install the trap
> until after creating the directory successfully.

OK, moved.

> I really think this is a waste of time though.  The current code creates
> the temp files in the current directory, and if the bad guy has write
> access on that directory you are already screwed (for instance, what's
> to stop him from altering the script file itself to do anything at all
> when you run it?).  I do not think that putting stuff back into /tmp is
> an improvement; that just adds risks where none exist now.

My method is secure, and I think we do have to handle this in a way that
addresses the security concerns.  It is easy to say no one would run
this under normal use but that isn't really a safe answer for the
security folks, I think.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

  16. Re: [HACKERS] Possible make_oidjoins_check Security Issue

    Neil Conway <neilc@samurai.com> — 2004-11-03T23:21:04Z

    On Thu, 2004-11-04 at 10:07, Bruce Momjian wrote:
> My method is secure, and I think we do have to handle this in a way that
> addresses the security concerns.

I think Tom's fix adequately addresses the security concerns. Exactly
what is wrong with writing to the current working directory?

> It is easy to say no one would run
> this under normal use but that isn't really a safe answer for the
> security folks, I think.

This is a non-sequitor -- I don't think Tom or anyone else has argued
this.

-Neil

  17. Re: [HACKERS] Possible make_oidjoins_check Security Issue

    Bruce Momjian <pgman@candle.pha.pa.us> — 2004-11-03T23:28:24Z

    Neil Conway wrote:
> On Thu, 2004-11-04 at 10:07, Bruce Momjian wrote:
> > My method is secure, and I think we do have to handle this in a way that
> > addresses the security concerns.
> 
> I think Tom's fix adequately addresses the security concerns. Exactly
> what is wrong with writing to the current working directory?

Because it could be run from a directory where others have write
permission.

> > It is easy to say no one would run
> > this under normal use but that isn't really a safe answer for the
> > security folks, I think.
> 
> This is a non-sequitor -- I don't think Tom or anyone else has argued
> this.

I remember hearing that from someone.  I thought it was Tom.

Bottom line is that the only secure way I have ever heard of for
creating temp files is to create a 077 directory in /tmp and write in
there.


-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

  18. Re: [HACKERS] Possible make_oidjoins_check Security Issue

    Tom Lane <tgl@sss.pgh.pa.us> — 2004-11-03T23:34:19Z

    Bruce Momjian <pgman@candle.pha.pa.us> writes:
>> I think Tom's fix adequately addresses the security concerns. Exactly
>> what is wrong with writing to the current working directory?

> Because it could be run from a directory where others have write
> permission.

In which case, they could also change the findoidjoins script itself.
I think your fix is *less* secure than what you replaced.

However, I've already wasted more than enough time on this issue...
I'm done arguing about it.

			regards, tom lane

  19. Re: [HACKERS] Possible make_oidjoins_check Security Issue

    Bruce Momjian <pgman@candle.pha.pa.us> — 2004-11-03T23:42:25Z

    Tom Lane wrote:
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> >> I think Tom's fix adequately addresses the security concerns. Exactly
> >> what is wrong with writing to the current working directory?
> 
> > Because it could be run from a directory where others have write
> > permission.
> 
> In which case, they could also change the findoidjoins script itself.
> I think your fix is *less* secure than what you replaced.
> 
> However, I've already wasted more than enough time on this issue...
> I'm done arguing about it.

As far as I know, my method is the only secure method.  If I am wrong I
would like to know.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

  20. Re: [HACKERS] Possible make_oidjoins_check Security Issue

    Gavin Sherry <swm@linuxworld.com.au> — 2004-11-04T00:21:27Z

    On Wed, 3 Nov 2004, Bruce Momjian wrote:

> Tom Lane wrote:
> > Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > >> I think Tom's fix adequately addresses the security concerns. Exactly
> > >> what is wrong with writing to the current working directory?
> >
> > > Because it could be run from a directory where others have write
> > > permission.
> >
> > In which case, they could also change the findoidjoins script itself.
> > I think your fix is *less* secure than what you replaced.
> >
> > However, I've already wasted more than enough time on this issue...
> > I'm done arguing about it.
>
> As far as I know, my method is the only secure method.  If I am wrong I
> would like to know.

I think the problem can really be solved by just removing it from the
distribution. However, one thing I noticed with Bruce's script is that it
does not respect $TMPDIR -- which security conscious admins may be
setting. Solution would be to set TMP=${TMPDIR:-/tmp} before defining the
path to the temporary sub directory.

Thanks,

Gavin

  21. Re: [HACKERS] Possible make_oidjoins_check Security Issue

    Tom Lane <tgl@sss.pgh.pa.us> — 2004-11-04T01:40:01Z

    Gavin Sherry <swm@linuxworld.com.au> writes:
> I think the problem can really be solved by just removing it from the
> distribution.

Just FYI, I've already done that in Red Hat's RPMs (not sure if Devrim
followed suit).  I can't think of a good reason for "make install" to
install that script, either.

			regards, tom lane

  22. Re: [HACKERS] Possible make_oidjoins_check Security Issue

    Bruce Momjian <pgman@candle.pha.pa.us> — 2004-11-04T02:05:08Z

    Gavin Sherry wrote:
> On Wed, 3 Nov 2004, Bruce Momjian wrote:
> 
> > Tom Lane wrote:
> > > Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > > >> I think Tom's fix adequately addresses the security concerns. Exactly
> > > >> what is wrong with writing to the current working directory?
> > >
> > > > Because it could be run from a directory where others have write
> > > > permission.
> > >
> > > In which case, they could also change the findoidjoins script itself.
> > > I think your fix is *less* secure than what you replaced.
> > >
> > > However, I've already wasted more than enough time on this issue...
> > > I'm done arguing about it.
> >
> > As far as I know, my method is the only secure method.  If I am wrong I
> > would like to know.
> 
> I think the problem can really be solved by just removing it from the
> distribution. However, one thing I noticed with Bruce's script is that it
> does not respect $TMPDIR -- which security conscious admins may be
> setting. Solution would be to set TMP=${TMPDIR:-/tmp} before defining the
> path to the temporary sub directory.

OK, TMPDIR honored.  Thanks.

I am fine with removing it but if we don't I would like to have it
secure, mostly from a public relations perspective.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

  23. Re: [HACKERS] Possible make_oidjoins_check Security Issue

    Neil Conway <neilc@samurai.com> — 2004-11-04T05:46:18Z

    On Thu, 2004-11-04 at 13:05, Bruce Momjian wrote:
> I am fine with removing it but if we don't I would like to have it
> secure, mostly from a public relations perspective.

A change which introduced two regressions and fails to materially
improve the security of the script is a curious definition of "secure"
if you ask me...

Attached is a patch that removes the make_oidjoins_check script from
"make install". Barring any objections, I'll apply it to HEAD later
today.

-Neil

  24. Re: [HACKERS] Possible make_oidjoins_check Security Issue

    Tom Lane <tgl@sss.pgh.pa.us> — 2004-11-04T05:55:42Z

    Neil Conway <neilc@samurai.com> writes:
> Attached is a patch that removes the make_oidjoins_check script from
> "make install". Barring any objections, I'll apply it to HEAD later
> today.

If we are going in that direction, all the files installed by this
subdirectory should be suppressed (ie, findoidjoins and
README.findoidjoins too).

			regards, tom lane

  25. Re: [HACKERS] Possible make_oidjoins_check Security Issue

    Peter Eisentraut <peter_e@gmx.net> — 2004-11-04T10:06:45Z

    Tom Lane wrote:
> If we are going in that direction, all the files installed by this
> subdirectory should be suppressed (ie, findoidjoins and
> README.findoidjoins too).

Why not move it to src/tools, so no one gets the impression that it is 
user code?

-- 
Peter Eisentraut
http://developer.postgresql.org/~petere/

  26. Re: [HACKERS] Possible make_oidjoins_check Security Issue

    Tom Lane <tgl@sss.pgh.pa.us> — 2004-11-04T14:47:46Z

    Peter Eisentraut <peter_e@gmx.net> writes:
> Why not move it to src/tools, so no one gets the impression that it is 
> user code?

I thought about that earlier, but concluded it wasn't worth the loss of
CVS history.

			regards, tom lane

  27. CVS should die (was: Possible make_oidjoins_check ...)

    Alvaro Herrera <alvherre@dcc.uchile.cl> — 2004-11-04T18:34:39Z

    On Thu, Nov 04, 2004 at 09:47:46AM -0500, Tom Lane wrote:
> Peter Eisentraut <peter_e@gmx.net> writes:
> > Why not move it to src/tools, so no one gets the impression that it is 
> > user code?
> 
> I thought about that earlier, but concluded it wasn't worth the loss of
> CVS history.

I have counted three times you have said that in the recent past.  IMHO
this really screams of changing the SCM tool.

Can this be discussed for 8.1?

-- 
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
"La felicidad no es mañana. La felicidad es ahora"

  28. Re: [PATCHES] CVS should die (was: Possible make_oidjoins_check ...)

    Tom Lane <tgl@sss.pgh.pa.us> — 2004-11-04T19:21:45Z

    Alvaro Herrera <alvherre@dcc.uchile.cl> writes:
> Can this be discussed for 8.1?

It's been discussed, and rejected, several times already.  There aren't
any alternatives that are enough better than CVS to be worth the
changeover effort.

			regards, tom lane

  29. Re: CVS should die (was: Possible make_oidjoins_check ...)

    Marc G. Fournier <scrappy@postgresql.org> — 2004-11-04T19:32:30Z

    On Thu, 4 Nov 2004, Alvaro Herrera wrote:

> On Thu, Nov 04, 2004 at 09:47:46AM -0500, Tom Lane wrote:
>> Peter Eisentraut <peter_e@gmx.net> writes:
>>> Why not move it to src/tools, so no one gets the impression that it is
>>> user code?
>>
>> I thought about that earlier, but concluded it wasn't worth the loss of
>> CVS history.

why would we lose CVS history?  I can physically move the files in 
/cvsroot to accomplish this ... just tell me what needs to move, and to 
where ...


----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664

  30. Re: CVS should die (was: Possible make_oidjoins_check ...)

    Tom Lane <tgl@sss.pgh.pa.us> — 2004-11-04T19:41:08Z

    "Marc G. Fournier" <scrappy@postgresql.org> writes:
> why would we lose CVS history?  I can physically move the files in 
> /cvsroot to accomplish this ... just tell me what needs to move, and to 
> where ...

If you physically move the files, that would retroactively change their
placement in back versions, no?  ie, it would appear that all previous
releases had had 'em under src/tools as well.

AFAICS the only nondestructive way to do this is to cvs delete and cvs
add, with a commit comment saying where the files were moved from.  Then
when you are looking at them in CVS, you'd have to navigate over to the
previous location (by hand, probably; the commit comment isn't going to
automate this for you) and look in the Attic to read the prior CVS history.
It's not impossible, certainly, but it discourages moving files for less
than the very best of reasons.

(I'm rather interested to know whether any other SCMs have a better
solution to this problem, and if so what it is.  It's not obvious how
to do better.)

			regards, tom lane

  31. Re: CVS should die (was: Possible make_oidjoins_check ...)

    Marc G. Fournier <scrappy@postgresql.org> — 2004-11-04T20:02:13Z

    On Thu, 4 Nov 2004, Tom Lane wrote:

> "Marc G. Fournier" <scrappy@postgresql.org> writes:
>> why would we lose CVS history?  I can physically move the files in
>> /cvsroot to accomplish this ... just tell me what needs to move, and to
>> where ...
>
> If you physically move the files, that would retroactively change their
> placement in back versions, no?  ie, it would appear that all previous
> releases had had 'em under src/tools as well.

Erk, yes, good point ...

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664

  32. Re: CVS should die (was: Possible make_oidjoins_check ...)

    Andrew Sullivan <ajs@crankycanuck.ca> — 2004-11-04T20:54:11Z

    On Thu, Nov 04, 2004 at 02:41:08PM -0500, Tom Lane wrote:
> 
> (I'm rather interested to know whether any other SCMs have a better
> solution to this problem, and if so what it is.  It's not obvious how
> to do better.)

I understood that the whole point of subversion was mostly to make
moving files easier.  It's number two in the feature list at the
subversion home page.  They say they version meta-data.

A

-- 
Andrew Sullivan  | ajs@crankycanuck.ca
In the future this spectacle of the middle classes shocking the avant-
garde will probably become the textbook definition of Postmodernism. 
                --Brad Holland

  33. Re: CVS should die

    Oliver Jowett <oliver@opencloud.com> — 2004-11-04T21:01:05Z

    Tom Lane wrote:

> AFAICS the only nondestructive way to do this is to cvs delete and cvs
> add, with a commit comment saying where the files were moved from.  Then
> when you are looking at them in CVS, you'd have to navigate over to the
> previous location (by hand, probably; the commit comment isn't going to
> automate this for you) and look in the Attic to read the prior CVS history.
> It's not impossible, certainly, but it discourages moving files for less
> than the very best of reasons.

You can also do a repository-side copy of the ,v file to the new 
location, remove old tags & branches from that new copy, and 'cvs 
delete' the old copy. That preserves history but the file should still 
show up in the old location (and not also in the new location) when 
older versions are checked out. In theory. It's all very hairy..

> (I'm rather interested to know whether any other SCMs have a better
> solution to this problem, and if so what it is.  It's not obvious how
> to do better.)

Subversion deals with this reasonably well. The main difference to CVS 
is that it does not try to track multiple lines of development in a 
particular file; instead, you make (internally cheap) copies *within* 
the repository tree when you branch or tag.

Once you have that, it's much easier to track file copies and deletions, 
as each path in the repository effectively has a linear history. A 
rename is just a copy and delete.

See http://svnbook.red-bean.com/svnbook-1.0/ch04s02.html for some more 
detail.

-O

  34. Re: CVS should die (was: Possible make_oidjoins_check ...)

    Joerg Hessdoerfer <joerg.hessdoerfer@sea-gmbh.com> — 2004-11-04T21:20:11Z

    Hi,

On Thursday 04 November 2004 20:41, Tom Lane wrote:
> "Marc G. Fournier" <scrappy@postgresql.org> writes:
> > why would we lose CVS history?  I can physically move the files in
> > /cvsroot to accomplish this ... just tell me what needs to move, and to
> > where ...
>
> If you physically move the files, that would retroactively change their
> placement in back versions, no?  ie, it would appear that all previous
> releases had had 'em under src/tools as well.
>
> AFAICS the only nondestructive way to do this is to cvs delete and cvs
> add, with a commit comment saying where the files were moved from.  Then
> when you are looking at them in CVS, you'd have to navigate over to the
> previous location (by hand, probably; the commit comment isn't going to
> automate this for you) and look in the Attic to read the prior CVS history.
> It's not impossible, certainly, but it discourages moving files for less
> than the very best of reasons.
>
> (I'm rather interested to know whether any other SCMs have a better
> solution to this problem, and if so what it is.  It's not obvious how
> to do better.)
>
>    regards, tom lane
>
> ---------------------------(end of broadcast)---------------------------
> TIP 8: explain analyze is your friend

<Advocacy>
Yes, some do. At least SVN (Subversion) can handle moves very well, and 
especially it doesn't loose history on moves/renames.
SVN holds it's repo entries in a database like 'filesystem', which can be 
backed by BDB4 or flat files (as of 1.1).
SVN has proven to be stable in many OSS projects, and vastly superior over CVS 
especially in handling multi-GB projects containing binary files in our 
company.

I refrain from listing all the advantages, if interested, have a look for 
yourself at http://subversion.tigris.org

</Advocacy>

Having one file in the repo per working copy file like with CVS is an obvious, 
but also obviously limited approach.  

Greetings,
 Jörg
-- 
Leading SW developer  - S.E.A GmbH
Mail: joerg.hessdoerfer@sea-gmbh.com
WWW:  http://www.sea-gmbh.com

  35. Re: [HACKERS] CVS should die

    Thomas Hallgren <thhal@mailblocks.com> — 2004-11-04T21:24:49Z

    Tom,
> (I'm rather interested to know whether any other SCMs have a better
> solution to this problem, and if so what it is.  It's not obvious how
> to do better.)
> 
I've been working with a few SCM's and IMHO only one of them really 
handles this really well. That's ClearCase. I'm well aware that 
ClearCase is not an option but I though it could still be interesting to 
know how this can be managed when done right.

In ClearCase everything (both files and directories) are "elements". A 
directory is a version of an element and it contains versions of other 
elements. It's not very different from Unix and I-nodes although 
everything is of course versioned.

Subversion claims they handle moves pretty well. When I checked it out, 
it turns out that a move is a copy (versions and all) followed by a 
delete, thus maintaining version history at both locations. I'd 
recommend anyone who think CVS is insufficient due to file moves to 
investigate subversion.

Regards,
Thomas Hallgren

  36. Re: CVS should die

    Thomas Hallgren <thhal@mailblocks.com> — 2004-11-04T21:24:49Z

    Tom,
> (I'm rather interested to know whether any other SCMs have a better
> solution to this problem, and if so what it is.  It's not obvious how
> to do better.)
> 
I've been working with a few SCM's and IMHO only one of them really 
handles this really well. That's ClearCase. I'm well aware that 
ClearCase is not an option but I though it could still be interesting to 
know how this can be managed when done right.

In ClearCase everything (both files and directories) are "elements". A 
directory is a version of an element and it contains versions of other 
elements. It's not very different from Unix and I-nodes although 
everything is of course versioned.

Subversion claims they handle moves pretty well. When I checked it out, 
it turns out that a move is a copy (versions and all) followed by a 
delete, thus maintaining version history at both locations. I'd 
recommend anyone who think CVS is insufficient due to file moves to 
investigate subversion.

Regards,
Thomas Hallgren

  37. Re: CVS should die (was: Possible make_oidjoins_check ...)

    Russell Smith <mr-russ@pws.com.au> — 2004-11-04T21:25:07Z

    On Fri, 5 Nov 2004 07:02 am, Marc G. Fournier wrote:
> On Thu, 4 Nov 2004, Tom Lane wrote:
> 
> > "Marc G. Fournier" <scrappy@postgresql.org> writes:
> >> why would we lose CVS history?  I can physically move the files in
> >> /cvsroot to accomplish this ... just tell me what needs to move, and to
> >> where ...
> >
> > If you physically move the files, that would retroactively change their
> > placement in back versions, no?  ie, it would appear that all previous
> > releases had had 'em under src/tools as well.
> 
> Erk, yes, good point ...
You could always, physically copy the file to the new location. Giving you all the history in the new location
and run CVS delete on the only location.  I can't see how this is too different from the cvs remove/cvs add.
However you get to keep the history as well as keeping the old version.

The second problem still exists where it's in 2 locations in previous releases. unless you cvs remove the new copy from
those branches as well.  As always CVS is a bit messy with these things, but just throwing ideas on the pile that might work.

Regards

Russell Smith

  38. Re: CVS should die

    Gaetano Mendola <mendola@bigfoot.com> — 2004-11-04T21:29:41Z

    Tom Lane wrote:
 > Alvaro Herrera <alvherre@dcc.uchile.cl> writes:
 >
 >>Can this be discussed for 8.1?
 >
 >
 > It's been discussed, and rejected, several times already.  There aren't
 > any alternatives that are enough better than CVS to be worth the
 > changeover effort.

The effort is not so big:  http://cvs2svn.tigris.org

Do not rename or move around a file because your SCM limits, is insane.


Regards
Gaetano Mendola

  39. Re: CVS should die (was: Possible make_oidjoins_check

    Neil Conway <neilc@samurai.com> — 2004-11-04T22:20:17Z

    [CC list trimmed]

On Fri, 2004-11-05 at 06:41, Tom Lane wrote:
> (I'm rather interested to know whether any other SCMs have a better
> solution to this problem, and if so what it is.  It's not obvious how
> to do better.)

Sure -- just about every "next generation" OSS version control tool gets
this right, or at least does a lot better than CVS -- i.e. svn,
monotone, arch, darchs, and possibly others I haven't heard of.

Gavin and I have been using Monotone for a few weeks now to manage some
development we're doing. I've been really impressed with it --
conceptually, it just "makes sense". Unfortunately the implementation is
currently too immature to consider moving the main development tree
onto, at least for the moment.

-Neil

  40. Re: CVS should die

    Gaetano Mendola <mendola@bigfoot.com> — 2004-11-04T22:54:19Z

    Neil Conway wrote:
> [CC list trimmed]
> 
> On Fri, 2004-11-05 at 06:41, Tom Lane wrote:
> 
>>(I'm rather interested to know whether any other SCMs have a better
>>solution to this problem, and if so what it is.  It's not obvious how
>>to do better.)
> 
> 
> Sure -- just about every "next generation" OSS version control tool gets
> this right, or at least does a lot better than CVS -- i.e. svn,
> monotone, arch, darchs, and possibly others I haven't heard of.
> 
> Gavin and I have been using Monotone for a few weeks now to manage some
> development we're doing. I've been really impressed with it --
> conceptually, it just "makes sense". Unfortunately the implementation is
> currently too immature to consider moving the main development tree
> onto, at least for the moment.

In SVN there is no concept of Branch, Label and so on, all these operations
are performed by "copy" if you want LABEL your source is enough create a
directory and copy all the source in that directory.
In order to emulate the way to work of CVS is enough follow these guidelines:
http://docs.codehaus.org/display/HAUS/How+to+Organize+a+Subversion+Repository

We are migrating from Clearcase to SVN and the only think we are going
to miss is the UCM process.


Regards
Gaetano Mendola

  41. Re: CVS should die (was: Possible make_oidjoins_check ...)

    David Helgason <david@uti.is> — 2004-11-05T00:12:27Z

    The intuitive understanding of a file is certainly something like "a 
file called 'baz.c' residing at 'foo/bar/', which contains the BAZ 
subsystem". Now, when renaming/moving a file such an intuitive 
understanding is partially lost. UI-wise that's a problem which I 
haven't ever seen solved well.

However, other SCM systems such as Subversion and Continuus (and our 
to-be-released system Maint, and certainly others) treat files as 
unique entities unrelated to their path, and thus don't have problems 
with moves.

With regards to modes of working this, it boils down to two methods. 
One is treating directories as first class entities (opposed to CVS 
which treats dirs as semi-relevant appendices to real files), versioned 
to contain a list of children, or simpler yet, to store the parent 
directory as an meaningful attribute of an object. Both methods have 
their pros and cons, the latter is somehow simpler to intuitively grasp 
for people.

This doesn't really answer the question of what tool Postgres might 
change to, but it seems that Subversion is a good tool one should 
consider. And by golly, CVS is bad. Just consider the cons – having to 
forbid renames in all but the most necessary cases – it just invites 
cruft into any project.

d.
-- 
David Helgason,
Business Development et al.,
Over the Edge I/S (http://otee.dk)
Direct line +45 2620 0663
Main line +45 3264 5049

On 4. nov 2004, at 20:41, Tom Lane wrote:

> "Marc G. Fournier" <scrappy@postgresql.org> writes:
>> why would we lose CVS history?  I can physically move the files in
>> /cvsroot to accomplish this ... just tell me what needs to move, and 
>> to
>> where ...
>
> If you physically move the files, that would retroactively change their
> placement in back versions, no?  ie, it would appear that all previous
> releases had had 'em under src/tools as well.
>
> AFAICS the only nondestructive way to do this is to cvs delete and cvs
> add, with a commit comment saying where the files were moved from.  
> Then
> when you are looking at them in CVS, you'd have to navigate over to the
> previous location (by hand, probably; the commit comment isn't going to
> automate this for you) and look in the Attic to read the prior CVS 
> history.
> It's not impossible, certainly, but it discourages moving files for 
> less
> than the very best of reasons.
>
> (I'm rather interested to know whether any other SCMs have a better
> solution to this problem, and if so what it is.  It's not obvious how
> to do better.)
>
> 			regards, tom lane
>
> ---------------------------(end of 
> broadcast)---------------------------
> TIP 8: explain analyze is your friend
>

  42. Re: CVS should die

    Chris Browne <cbbrowne@acm.org> — 2004-11-05T01:57:34Z

    In an attempt to throw the authorities off his trail, tgl@sss.pgh.pa.us (Tom Lane) transmitted:
> Alvaro Herrera <alvherre@dcc.uchile.cl> writes:
>> Can this be discussed for 8.1?
>
> It's been discussed, and rejected, several times already.  There
> aren't any alternatives that are enough better than CVS to be worth
> the changeover effort.

Subversion may be getting close to the point where it may be worth
thinking of, and there is a pretty full-featured conversion scheme,
cvs2svn, allowing considerable choice as to what aspects of the CVS
branches will be included.

The one traditional _enormous_ problem with it was that while
much-lauded, it suffered interoperability issues.  People running
different versions of {Debian|RHAT|FreeBSD|...} could have versions
that couldn't talk to one another.  That appears to have been
alleviated:

   "Now that subversion has reached 1.0.0 our compatibility guarantees
   require forward and backward compatible repository formats for all
   patch releases and backward compatible for minor releases.  So
   until 2.0.0 comes out there will be no change that should require a
   dump for upgrading to newer versions."

I'll buy the argument that it'll take some work for people familiar
with CVS to get familiar with SVN.  Of course...

   "Generally, Subversion's interface to a particular feature is
    similar to CVS's, except where there's a compelling reason to do
    otherwise."

I have been watching Subversion develop for quite some time, and have
always felt it the right idea to put usage off because it did not
appear mature enough.  I have always thought "in another year, it may
be ready."  As far as maturity is concerned, it looks like it's there
now.  The formerly compelling reasons for instant rejection are no
longer there.

If it's plausible to run a SVN archive, in parallel, that can accept
patches coming out of the present CVS, it must surely be time for some
intrepid fan of Subversion to put up an an archive and start showing
off how much better it is.  Proving it's viable by demonstration is a
pretty ideal methodology, no?

By the way, one of the longer term goals is for SVN to support a SQL
repository backend; there's probably merit to some "common dogfood
usage" ;-).
-- 
(reverse (concatenate 'string "gro.gultn" "@" "enworbbc"))
http://linuxfinances.info/info/unix.html
"Are  we  worried about  Linux?  ... Sure  we  are  worried." 
-- Steve Ballmer, VP of MICROS~1 at Seybold publishing conference

  43. Re: CVS should die

    Thomas Hallgren <thhal@mailblocks.com> — 2004-11-05T12:28:25Z

    Joerg Hessdoerfer wrote:
> <Advocacy>
> Yes, some do. At least SVN (Subversion) can handle moves very well, and 
> especially it doesn't loose history on moves/renames.
> SVN holds it's repo entries in a database like 'filesystem', which can be 
> backed by BDB4 or flat files (as of 1.1).
> SVN has proven to be stable in many OSS projects, and vastly superior over CVS 
> especially in handling multi-GB projects containing binary files in our 
> company.
> 
> I refrain from listing all the advantages, if interested, have a look for 
> yourself at http://subversion.tigris.org
> 
> </Advocacy>
<MoreAdvocacy>
Another compelling reason to use SVN is that one of their long term 
goals is to use an SQL backend. PostreSQL must be the absolute best 
choice for that, right? So knowledge of SVN and some future 
collaboration could perhaps be beneficial for both parties.

SVN is also targeted as a CVS replacement and a CVS user will feel very 
much at home.
</MoreAdvocacy>

Regards,
Thomas Hallgren

  44. Re: CVS should die

    Thomas Hallgren <thhal@mailblocks.com> — 2004-11-05T12:28:25Z

    Joerg Hessdoerfer wrote:
> <Advocacy>
> Yes, some do. At least SVN (Subversion) can handle moves very well, and 
> especially it doesn't loose history on moves/renames.
> SVN holds it's repo entries in a database like 'filesystem', which can be 
> backed by BDB4 or flat files (as of 1.1).
> SVN has proven to be stable in many OSS projects, and vastly superior over CVS 
> especially in handling multi-GB projects containing binary files in our 
> company.
> 
> I refrain from listing all the advantages, if interested, have a look for 
> yourself at http://subversion.tigris.org
> 
> </Advocacy>
<MoreAdvocacy>
Another compelling reason to use SVN is that one of their long term 
goals is to use an SQL backend. PostreSQL must be the absolute best 
choice for that, right? So knowledge of SVN and some future 
collaboration could perhaps be beneficial for both parties.

SVN is also targeted as a CVS replacement and a CVS user will feel very 
much at home.
</MoreAdvocacy>

Regards,
Thomas Hallgren

  45. Re: [PATCHES] CVS should die

    Neil Conway <neilc@samurai.com> — 2004-11-05T12:36:32Z

    Thomas Hallgren wrote:
> Another compelling reason to use SVN is that one of their long term 
> goals is to use an SQL backend.

That is about as far from a "compelling reason" to use a particular 
version control system as I can imagine.

-Neil

  46. Re: [PATCHES] CVS should die

    Andrew Dunstan <andrew@dunslane.net> — 2004-11-05T13:13:18Z

    
Neil Conway wrote:

> Thomas Hallgren wrote:
>
>> Another compelling reason to use SVN is that one of their long term 
>> goals is to use an SQL backend.
>
>
> That is about as far from a "compelling reason" to use a particular 
> version control system as I can imagine.
>
>

Yeah.

I see these considerations as being important:

. does tool x do what we need?
. is tool x FOSS software?
. is the benefit to be gained from moving to tool x worth the pain involved?

I'll repeat an observation I made (more or less) last time we had this 
discussion: the loudest voice in it belongs to those who actually use 
the repository most. When Tom or Bruce or Peter (for example) tell us we 
need to change I'll take lots more notice.

I have little doubt that we will one day move away from CVS. What we 
will move to is still open - and I don't yet see a reason to rush into 
the arms of Subversion.

cheers

andrew

  47. Re: [PATCHES] CVS should die

    Thomas Hallgren <thhal@mailblocks.com> — 2004-11-05T14:24:31Z

    Andrew Dunstan wrote:
> 
> 
> Neil Conway wrote:
> 
>> Thomas Hallgren wrote:
>>
>>> Another compelling reason to use SVN is that one of their long term 
>>> goals is to use an SQL backend.
>>
>>
>>
>> That is about as far from a "compelling reason" to use a particular 
>> version control system as I can imagine.
>>
>>
> 
> Yeah.
> 
> I see these considerations as being important:
> 
> . does tool x do what we need?
> . is tool x FOSS software?
> . is the benefit to be gained from moving to tool x worth the pain 
> involved?
>
Duh! Bad wording on my part. I didn't mean that their future use of SQL 
backend should be a criteria. So "compelling reason" was a bad choice of 
words.

What I meant was, that at some point, we might be able to help the SVN 
people reach their SQL objective and at the same time push for 
PostgreSQL as the best choice. If PostgreSQL uses SVN (for the reasons 
mentioned by Andrew), then some knowledge will be gained and 
relationships established that might make such a collaboration very natural.

Once a PostgreSQL based backend is well tested and very stable, 
PostgreSQL can make the switch to use it for their own production. From 
an SVN standpoint, it must be a perfect reference to be able to say 
"Look, PostgreSQL uses SVN with their own database to store their own 
code". A better proof of concept doesn't exist! From a PostgreSQL 
standpoint? Well SVN already have a high amount of users and it is 
growing rapidly, i.e. the visibility is great.

It also struck me that the requirements for an SCM backend store must be 
especially well handled by an MVCC type system. New data is added 
frequently but very rarely updated or deleted.

Regards,
Thomas Hallgren

  48. Re: [PATCHES] CVS should die

    Peter Eisentraut <peter_e@gmx.net> — 2004-11-05T15:22:55Z

    Am Freitag, 5. November 2004 14:13 schrieb Andrew Dunstan:
> I'll repeat an observation I made (more or less) last time we had this
> discussion: the loudest voice in it belongs to those who actually use
> the repository most. When Tom or Bruce or Peter (for example) tell us we
> need to change I'll take lots more notice.

I'm certainly open to considering subversion, although I have a certain 
traumatic experience with it that may or may not be related to the BDB 
backend that it uses.

I think for a start it would be nice if pgfoundry could optionally offer 
subversion (and/or arch) for source control, so that some developer groups 
and also our system administrators could get some experience with it.

-- 
Peter Eisentraut
http://developer.postgresql.org/~petere/

  49. Re: [PATCHES] CVS should die

    Gaetano Mendola <mendola@bigfoot.com> — 2004-11-05T15:44:18Z

    Peter Eisentraut wrote:
> Am Freitag, 5. November 2004 14:13 schrieb Andrew Dunstan:
> 
>>I'll repeat an observation I made (more or less) last time we had this
>>discussion: the loudest voice in it belongs to those who actually use
>>the repository most. When Tom or Bruce or Peter (for example) tell us we
>>need to change I'll take lots more notice.
> 
> 
> I'm certainly open to considering subversion, although I have a certain 
> traumatic experience with it that may or may not be related to the BDB 
> backend that it uses.
> 
> I think for a start it would be nice if pgfoundry could optionally offer 
> subversion (and/or arch) for source control, so that some developer groups 
> and also our system administrators could get some experience with it.

I good very start point is see if cvs2svn can handle the postgresql CVS without
errors.


Regards
Gaetano Mendola

  50. Re: [PATCHES] CVS should die

    Andrew Dunstan <andrew@dunslane.net> — 2004-11-05T15:46:38Z

    
Peter Eisentraut wrote:

>I think for a start it would be nice if pgfoundry could optionally offer 
>subversion (and/or arch) for source control, so that some developer groups 
>and also our system administrators could get some experience with it.
>
>  
>

I agree. We (the pgfoundry admins) will see what can be done - no 
promises at this stage.

cheers

andrew

  51. Re: [PATCHES] CVS should die

    Heikki Linnakangas <hlinnaka@iki.fi> — 2004-11-05T17:38:12Z

    On Fri, 5 Nov 2004, Gaetano Mendola wrote:

> Peter Eisentraut wrote:
>
>> I'm certainly open to considering subversion, although I have a certain 
>> traumatic experience with it that may or may not be related to the BDB 
>> backend that it uses.
>> 
>> I think for a start it would be nice if pgfoundry could optionally offer 
>> subversion (and/or arch) for source control, so that some developer groups 
>> and also our system administrators could get some experience with it.
>
> I good very start point is see if cvs2svn can handle the postgresql CVS 
> without
> errors.

I tried that yesterday out of curiosity. It had problems with 3 files 
which I removed manually:

/pgsql/src/interfaces/perl5/Attic/ApachePg.pl,v
/pgsql/src/interfaces/perl5/Attic/test.pl.newstyle,v
/pgsql/src/interfaces/perl5/Attic/test.pl.oldstyle.pl,v

Otherwise, no problems.

Interestingly, the subversion repository is 585MB, and the CVS repository 
is only 260MB, so apparently Subversion is not very good at compressing 
the repository. Not that it matters, though.

FWIW, I think Peter's idea of offering Subversion as an alternative in 
pgfoundry is very good.

I also agree with Andrew's observation that it's really up to the 
committers since they are the ones that have to work with whatever system 
we have.

I like subversion very much, but one thing that troubles me a bit is the 
number of extra libraries required to compile and run it. Also, is there 
pre-compiled binaries for all the platforms that PostgreSQL supports?

- Heikki

  52. Re: [PATCHES] CVS should die

    Joshua D. Drake <jd@commandprompt.com> — 2004-11-05T17:45:17Z

    Heikki Linnakangas wrote:

> On Fri, 5 Nov 2004, Gaetano Mendola wrote:
>
>> Peter Eisentraut wrote:
>>
>>> I'm certainly open to considering subversion, although I have a 
>>> certain traumatic experience with it that may or may not be related 
>>> to the BDB backend that it uses.
>>>
>>> I think for a start it would be nice if pgfoundry could optionally 
>>> offer subversion (and/or arch) for source control, so that some 
>>> developer groups and also our system administrators could get some 
>>> experience with it.
>>
>>
>> I good very start point is see if cvs2svn can handle the postgresql 
>> CVS without
>> errors.
>
>
> I tried that yesterday out of curiosity. It had problems with 3 files 
> which I removed manually:
>
> /pgsql/src/interfaces/perl5/Attic/ApachePg.pl,v
> /pgsql/src/interfaces/perl5/Attic/test.pl.newstyle,v
> /pgsql/src/interfaces/perl5/Attic/test.pl.oldstyle.pl,v
>
> Otherwise, no problems.
>
> Interestingly, the subversion repository is 585MB, and the CVS 
> repository is only 260MB, so apparently Subversion is not very good at 
> compressing the repository. Not that it matters, though.
>
Subversion, I believe uses SleepycatDb (eg Db4). Thus it isn't flat 
files like CVS.


>
> I also agree with Andrew's observation that it's really up to the 
> committers since they are the ones that have to work with whatever 
> system we have.
>
> I like subversion very much, but one thing that troubles me a bit is 
> the number of extra libraries required to compile and run it. Also, is 
> there pre-compiled binaries for all the platforms that PostgreSQL 
> supports?

Doubtful. Also there are known issues with Subversion on >FC1 if you are 
running newer versions of it. You have to compile specially with 
--without-posix-mutexes (I think that was the flag).

Sincerely,

Joshua D. Drake


>
> - Heikki
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster



-- 
Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com
PostgreSQL Replicator -- production quality replication for PostgreSQL

  53. Re: [PATCHES] CVS should die

    Gaetano Mendola <mendola@bigfoot.com> — 2004-11-05T17:58:40Z

    Heikki Linnakangas wrote:

> I tried that yesterday out of curiosity. It had problems with 3 files 
> which I removed manually:
> 
> /pgsql/src/interfaces/perl5/Attic/ApachePg.pl,v
> /pgsql/src/interfaces/perl5/Attic/test.pl.newstyle,v
> /pgsql/src/interfaces/perl5/Attic/test.pl.oldstyle.pl,v
> 
> Otherwise, no problems.
> 
> Interestingly, the subversion repository is 585MB, and the CVS 
> repository is only 260MB, so apparently Subversion is not very good at 
> compressing the repository. Not that it matters, though.
> 
> FWIW, I think Peter's idea of offering Subversion as an alternative in 
> pgfoundry is very good.

Mmm, do you mean createing periodically "snapshot"? Yes this could be
a good idea.

> I also agree with Andrew's observation that it's really up to the 
> committers since they are the ones that have to work with whatever 
> system we have.

That's true, but is really sad see Tom Lane think twice to move a file or
not because CVS.

> I like subversion very much, but one thing that troubles me a bit is the 
> number of extra libraries required to compile and run it. Also, is there 
> pre-compiled binaries for all the platforms that PostgreSQL supports?

I don't know about the server, but for sure what is more important here is the
client side and now that the win environment matter more then before,  I have to
say that TortoiseSVN ( tortoisesvn.tigris.org ) is much better then WinCVS.


Regards
Gaetano Mendola

  54. Re: [PATCHES] CVS should die

    Ian Lawrence Barwick <barwick@gmail.com> — 2004-11-05T18:42:50Z

    On Fri, 5 Nov 2004 16:22:55 +0100, Peter Eisentraut <peter_e@gmx.net> wrote:
> Am Freitag, 5. November 2004 14:13 schrieb Andrew Dunstan:
> > I'll repeat an observation I made (more or less) last time we had this
> > discussion: the loudest voice in it belongs to those who actually use
> > the repository most. When Tom or Bruce or Peter (for example) tell us we
> > need to change I'll take lots more notice.
> 
> I'm certainly open to considering subversion, although I have a certain
> traumatic experience with it that may or may not be related to the BDB
> backend that it uses.

Aha, glad I'm not the only one. Version 1.1 has a flat-file based
backend which is not prone to BDB-permission-related problems, see:
http://svnbook.red-bean.com/svnbook-1.1/ch05.html#svn-ch-5-sect-1.4 .
It's only been around a few months though and the docs mention
possible issues with scalability.

Ian Barwick
barwick@gmail.com

  55. Re: [PATCHES] CVS should die

    Heikki Linnakangas <hlinnaka@iki.fi> — 2004-11-05T19:16:24Z

    On Fri, 5 Nov 2004, Gaetano Mendola wrote:

> Heikki Linnakangas wrote:
>
>> FWIW, I think Peter's idea of offering Subversion as an alternative in 
>> pgfoundry is very good.
>
> Mmm, do you mean createing periodically "snapshot"? Yes this could be
> a good idea.

No, I mean that each project could choose to use either cvs or svn, like 
they do at Apache.

Sure, if you could have both, that would be even better.

>> I like subversion very much, but one thing that troubles me a bit is the 
>> number of extra libraries required to compile and run it. Also, is there 
>> pre-compiled binaries for all the platforms that PostgreSQL supports?
>
> I don't know about the server, but for sure what is more important here is 
> the
> client side and now that the win environment matter more then before,  I have 
> to
> say that TortoiseSVN ( tortoisesvn.tigris.org ) is much better then WinCVS.

True. Looking at the Subversion downloads page, they seem to have binaries 
for various Linux distributions, FreeBSD, NetBSD and OpenBSD, Solaris, 
Mac OS X and Win32. According to the supported platforms chapter in 
pgsql documentation, we also support AIX, BSD/OS, HP-UX, IRIX, Tru64 
UNIX, UnixWare, and Linux on Alpha, arm41, m64, MIPS, PPC, S/390 and 
Sparc.

Developers on those platforms would have to compile subversion themselves, 
or compile pgsql from source tarballs.

Have you looked at TortoiseCVS (www.tortoisecvs.org)? I think TortoiseSVN 
is a fork of that.

- Heikki

  56. Re: [PATCHES] CVS should die

    Gaetano Mendola <mendola@bigfoot.com> — 2004-11-05T19:38:51Z

    -----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Heikki Linnakangas wrote:

| Have you looked at TortoiseCVS (www.tortoisecvs.org)? I think
| TortoiseSVN is a fork of that.

I didn't know the existence, is not even listed in the subproject
on CVS home page, I discovered TortoiseSVN on the Subversion home
page.


Regards
Gaetano Mendola



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (MingW32)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFBi9bJ7UpzwH2SGd4RAgraAKCcNLaMJPPjVxfqRQ1yGG2+GssiAACeJFg3
zULofgK2ouUum3wNSjUmG3U=
=Bq/a
-----END PGP SIGNATURE-----

  57. Re: [PATCHES] CVS should die

    Tom Lane <tgl@sss.pgh.pa.us> — 2004-11-05T20:37:54Z

    Ian Barwick <barwick@gmail.com> writes:
> Aha, glad I'm not the only one. Version 1.1 has a flat-file based
> backend which is not prone to BDB-permission-related problems, see:
> http://svnbook.red-bean.com/svnbook-1.1/ch05.html#svn-ch-5-sect-1.4 .
> It's only been around a few months though and the docs mention
> possible issues with scalability.

One of the reasons I'm disinclined to move is that none of the proposed
alternatives seem especially, um, mature.  AFAIK this project has never
had CVS lose any data in the eight years we've used it.  I'd want a
comparable level of trust in any replacement SCM, and I haven't got it.

			regards, tom lane

  58. Re: [PATCHES] CVS should die

    Andrew McMillan <andrew@catalyst.net.nz> — 2004-11-06T09:36:25Z

    On Fri, 2004-11-05 at 15:37 -0500, Tom Lane wrote:
> 
> One of the reasons I'm disinclined to move is that none of the proposed
> alternatives seem especially, um, mature.  AFAIK this project has never
> had CVS lose any data in the eight years we've used it.  I'd want a
> comparable level of trust in any replacement SCM, and I haven't got it.

A very sane reason.  I've lost my share of stuff with SVN in trialling
it, but we are switching our company over to Arch, which seems to offer
significantly more benefits.  From our trialling of it, I think it has a
more robust and mature repository structure too.

Watching the PostgreSQL team developing I would think that Arch would
provide much better support for the developers than SVN would. 

Switching to Arch is more work, but it also offers a lot more benefits -
including the opportunity for individuals to maintain their own trees,
and be able to work out which patchsets from someone else's tree have
not been applied.  If anything is going to become the open-source
BitKeeper it will be this, I think.

Cheers,
					Andrew.

-------------------------------------------------------------------------
Andrew @ Catalyst .Net .NZ  Ltd,  PO Box 11-053, Manners St,  Wellington
WEB: http://catalyst.net.nz/            PHYS: Level 2, 150-154 Willis St
DDI: +64(4)803-2201      MOB: +64(272)DEBIAN      OFFICE: +64(4)499-2267
                     Planning an election?  Call us!
-------------------------------------------------------------------------

  59. Re: [PATCHES] CVS should die

    Thomas Hallgren <thhal@mailblocks.com> — 2004-11-06T10:53:13Z

    Andrew McMillan wrote:
> Switching to Arch is more work, but it also offers a lot more benefits -
> including the opportunity for individuals to maintain their own trees,
> and be able to work out which patchsets from someone else's tree have
> not been applied.  If anything is going to become the open-source
> BitKeeper it will be this, I think.
> 
For those interested in SVN versus arch, I found the following from Tom 
Lord (the guy behind Arch)

http://web.mit.edu/ghudson/thoughts/diagnosing

and a reply from Greg Hudson (SVN developer)

http://web.mit.edu/ghudson/thoughts/undiagnosing.

Regards,
Thomas Hallgren

  60. Re: [PATCHES] CVS should die

    Anand Kumria <wildfire@progsoc.org> — 2004-11-08T17:18:56Z

    On Sat, 06 Nov 2004 11:53:13 +0100, Thomas Hallgren wrote:

> Andrew McMillan wrote:
>> Switching to Arch is more work, but it also offers a lot more benefits -
>> including the opportunity for individuals to maintain their own trees,
>> and be able to work out which patchsets from someone else's tree have
>> not been applied.  If anything is going to become the open-source
>> BitKeeper it will be this, I think.
>> 
> For those interested in SVN versus arch, I found the following from Tom 
> Lord (the guy behind Arch)
> 
> http://web.mit.edu/ghudson/thoughts/diagnosing
> 
> and a reply from Greg Hudson (SVN developer)
> 
> http://web.mit.edu/ghudson/thoughts/undiagnosing.
> 

There is a fairly detailed comparison in the GNU Arch wiki as well.

<URL: http://wiki.gnuarch.org/moin.cgi/SubVersionAndCvsComparison>

Note: if you're a postgres committer you may have more luck seeking out
your nearest SCM advocate -- almost all of them would regard Postgres
migrating to their preferred SCM as a 'win' -- let them work for you by
walking you through things.

Cheers,
Anand

  61. Re: CVS should die (was: Possible make_oidjoins_check ...)

    Steve Crawford <scrawford@pinpointresearch.com> — 2004-11-09T22:56:54Z

    > This doesn't really answer the question of what tool Postgres might
> change to, but it seems that Subversion is a good tool one should
> consider. And by golly, CVS is bad. Just consider the cons – having
> to forbid renames in all but the most necessary cases – it just
> invites cruft into any project.

Interesting reading:
http://better-scm.berlios.de/comparison/comparison.html
http://zooko.com/revision_control_quick_ref.html

Cheers,
Steve

  62. Re: CVS should die

    Thomas Hallgren <thhal@mailblocks.com> — 2004-11-14T00:20:03Z

    Tom Lane wrote:
> ... There aren't
> any alternatives that are enough better than CVS to be worth the
> changeover effort.
> 
I've done some research over the last couple of days for a fairly big 
project where we face the challenges of breaking up a monolith into 
modules and consequently will be forced to move a lot of files. I now 
second Tom's opinion. Here's why:

Subversion doesn't move files. They copy and delete. So if you have 
parallel work on a file that is "moved", you are headed for problems. 
See threads:

"Question about rename" on users@subversion.tigris.org
news://news.gmane.org:119/cmsqci$s9q$1@sea.gmane.org

and

"Misinforming the user on rename with local changes" 
dev@subversion.tigris.org
news://news.gmane.org:119/419379F3.5070302@ftml.net

What I find especially intriguing is that although Subversion have 
version controlled directories, they still identify the content of the 
files using the location in the repository rather than using a globally 
unique identifier. Didn't they anticipate files being moved around and 
perhaps linked?

This thread started due to CVS problems with moving files and Subversion 
will perhaps get there eventually but IMHO they are certainly not there yet.

GNU-Arch seems promising in some respects. It really can rename files 
and track them using an id, but it doesn't run on Windows without Cygwin 
(and even then not too well it seems). Personally I dislike the fact 
that the author seems somewhat religious about free software and hostile 
towards Windows instead of focusing on delivering a portable solution. 
In my case, the fact that GNU-Arch is not portable is reason enough to 
discard it as a viable alternative and I think it would be unfortunate 
if PostgreSQL locked Windows users out from repository access.

The other Open Source alternatives are, IMHO not mature enough to be 
considered for serious projects yet.

I wish ClearCase was fast, free, and suitable for distributed 
development :-) Unfortunately it's slow, expensive, and extremely 
network intensive. My approach will be to wait and perhaps contribute to 
Subversion if I get some time left. They really need a great database 
backend.

Regards,
Thomas Hallgren

  63. Re: CVS should die

    Andrew Dunstan <andrew@dunslane.net> — 2004-11-14T02:25:02Z

    
Thomas Hallgren wrote:

>
> GNU-Arch seems promising in some respects. It really can rename files 
> and track them using an id, but it doesn't run on Windows without 
> Cygwin (and even then not too well it seems). Personally I dislike the 
> fact that the author seems somewhat religious about free software and 
> hostile towards Windows instead of focusing on delivering a portable 
> solution. In my case, the fact that GNU-Arch is not portable is reason 
> enough to discard it as a viable alternative and I think it would be 
> unfortunate if PostgreSQL locked Windows users out from repository 
> access.
>
>

s/unfortunate/totally unacceptable/


cheers

andrew

  64. Re: CVS should die

    Travis P <twp@castle.fastmail.fm> — 2004-11-14T03:19:13Z

    On Nov 13, 2004, at 6:20 PM, Thomas Hallgren wrote:

> "Question about rename" on users@subversion.tigris.org
> news://news.gmane.org:119/cmsqci$s9q$1@sea.gmane.org

Thomas (Hallgren):  Unfortunately, my efforts to get Thunderbird to do 
something useful with that URL have been unsuccessful and I can't find 
the thread on the (usable) mailing list archive ( don't use the 
tigris.org archive; use http://svn.haxx.se/ ).

> "Misinforming the user on rename with local changes" 
> dev@subversion.tigris.org
> news://news.gmane.org:119/419379F3.5070302@ftml.net

Might be easier to read with a browser here:
   http://svn.haxx.se/dev/archive-2004-11/index.shtml

Yes, looks like it could be a potential problem/inconvenience if a file 
is both moved and altered simultaneously.

I see the risk of problems as similar to those that if two people edit 
the same section of the same file at the same time, complicated 
conflicts could emerge.  You could avoid this problem using 
Lock-Modify-Unlock rather than Copy-Modify-Merge.  Some people do.  
Others (many CVS users) choose the productivity gains by using 
Copy-Modify-Merge and then deal with the eventual problem when/if it 
shows up.

It is too bad the Subversion design didn't anticipate this such that 
it's not a problem at all, but in many environments, it may not be much 
of an issue.

> This thread started due to CVS problems with moving files and 
> Subversion will perhaps get there eventually but IMHO they are 
> certainly not there yet.

It is worth noting that there is already huge improvement in this area. 
  There's a baby in that bathwater. :-)  It's made my life much easier, 
particularly for Java development where refactoring requires renames 
and moves of files and directories more often than with some other 
languages like C/C++.

I'm not really advocating a switch if you don't think it's worth it.  
Just hoping to contribute constructively to the discussion where the 
worth of the costs/benefits are measured by others.

Back to playing with the 8 beta for me,  :-)
--Travis

  65. Re: CVS should die

    Thomas Hallgren <thhal@mailblocks.com> — 2004-11-14T08:47:11Z

    Travis P wrote:

> Thomas (Hallgren):  Unfortunately, my efforts to get Thunderbird to do 
> something useful with that URL have been unsuccessful and I can't find 
> the thread on the (usable) mailing list archive ( don't use the 
> tigris.org archive; use http://svn.haxx.se/ ).

Thanks Travis. I'm not at all friend with Tigris mailing list archives 
and I wasn't aware of haxx.se. Here are the relevant links:

http://svn.haxx.se/dev/archive-2004-11/0505.shtml
http://svn.haxx.se/users/archive-2004-11/0433.shtml
http://svn.haxx.se/users/archive-2004-11/0445.shtml

Regards,
Thomas Hallgren

  66. Re: CVS should die

    Kaz Kylheku <kaz@ashi.footprints.net> — 2004-11-15T07:41:21Z

    twp@castle.fastmail.fm (Travis P) wrote in message news:<F54111AA-35EB-11D9-B14F-003065F9DAF8@castle.fastmail.fm>...
> > "Misinforming the user on rename with local changes" 
> > dev@subversion.tigris.org
> > news://news.gmane.org:119/419379F3.5070302@ftml.net
> 
> Might be easier to read with a browser here:
>    http://svn.haxx.se/dev/archive-2004-11/index.shtml
> 
> Yes, looks like it could be a potential problem/inconvenience if a file 
> is both moved and altered simultaneously.

You guys should check out the software that I developed called
Meta-CVS.

It creates a version control system that has directory structure
versioning, over top of the file versioning semantics provided by CVS.

Meta-CVS does not have problems with these corner cases, by design.

> I see the risk of problems as similar to those that if two people edit 
> the same section of the same file at the same time, complicated 
> conflicts could emerge.

In Meta-CVS, conflicts in the directory structure are exactly like
these conflicts, because the directory structure is marked up as a
straightforward text document.

When conflicts occur, you can read that document and it's obvious that
one user wanted to rename foo.c to src/foo.c, whereas another one
wanted to rename it to foobar.c.

Meta-CVS completely separates the directory structure from the files,
in the classic way: just like Unix file systems, and network file
systems like NFS and others. A file is known by an 128 bit identifier
encoded as text in hexadecimal.

So for example, even picking up a deletion to a locally modified file
is safe.

Suppose you have been editing a document foo.txt, do a ``mcvs up'' and
it's deleted. Your changes are still safe and can be committed. All
that happened was that your foo.txt was unlinked from the directory
structure. The real object, still exists, and has all your changes. It
can be committed to CVS. Independently of that action, you can
re-introduce your object into the directory structure: just change the
markup document (a file called MAP in the MCVS directory at the root
of your project) and then run ``mcvs up''. Meta-CVS will notice the
change, and link the file into the appropriate place, as requested by
the new markup. You can commit that markup change, and the file will
reappear in people's sandboxes when they pick it up.

> It is too bad the Subversion design didn't anticipate this such that 
> it's not a problem at all, but in many environments, it may not be much 
> of an issue.

Fortunately, I anticipated the problem before I laid down the first
line of code.