Horton

Thread

  1. Removing derived files from CVS

    Tom Lane <tgl@sss.pgh.pa.us> — 1999-03-19T02:40:03Z

    I have been looking into what it would take to remove derived files
from the CVS repository, and it doesn't look bad at all.  I propose
we do so before 6.5 beta.

In case anyone's forgotten: the issue is derived files, such as gram.c,
which we currently keep in the CVS repository even though they are not
master source files.  Doing so causes a number of headaches, including
wasted time to check in and check out updates to both master and derived
files, unreasonable bulk of the CVS files for these derived files,
errors due to timestamp skew (after checking out, it can look like you
have an up-to-date derived file when you do not), etc etc.

The only reason for keeping these files in CVS is so that users who
obtain the source distribution don't have to have tools that can rebuild
these files.  But there's a better way to handle that: generate the derived
files while preparing tarballs.  That way we can remove the derived
files from CVS.  We'll also eliminate the other time skew problem that's
been seen in more than one past release tarball: the derived files will
be certain to have newer timestamps than their masters in the tarballs.

The most reliable way to do this is just to have a script that does
	configure
	"make" all the derived files
	make distclean
and invoke this script as part of the tarball generation procedure.
Configuring in order to find out which yacc and lex to use may seem
a tad expensive ;-) but this way will work, whereas taking shortcuts
would have a tendency to break.  Doing the make distclean also ensures
that the tarball will not contain any extraneous files, which seems like
a good idea.

I have just tested this procedure and determined that it takes less than
2 minutes on hub.org, which seems well within the realm of acceptability
for a nightly batch job.

So, a few questions for the list:

1. Does anyone object to removing these files from the CVS repository and
handling them as above:
	src/backend/parser/gram.c
	src/backend/parser/parse.h
	src/backend/parser/scan.c
	src/interfaces/ecpg/preproc/preproc.c
	src/interfaces/ecpg/preproc/preproc.h
	src/interfaces/ecpg/preproc/pgc.c

2. Should we also handle src/configure this way?  That would mean that
people who obtain the code straight from CVS would have to have autoconf
installed.  It's probably a good idea but I'm not certain.

3. src/pl/plpgsql/src/ also contains yacc and lex output files that are
checked into CVS.  We definitely should remove them from CVS, but should
we leave them to be generated by recipients of the distribution, or
should we handle them like the big grammar files?  I don't think they
are big enough to break anyone's yacc, but...

4. Currently, a recipient must have at least minimally working yacc/lex
capability anyway, because the bootstrap files in src/backend/bootstrap/
are not pre-built in the distribution.  If we used the same procedure
for the bootstrap and plpgsql files as for the bigger parsers, then it
would be possible to build Postgres without a local yacc or lex.  Is
this worth doing, or would it just bloat the distribution to no purpose?
As far as I know we have not gotten complaints about the need for
yacc/lex for these files; it's only that the parser and ecpg grammars
are too big for some vendor versions...

			regards, tom lane

  2. Re: [HACKERS] Removing derived files from CVS

    Bruce Momjian <maillist@candle.pha.pa.us> — 1999-03-19T02:44:09Z

    > I have been looking into what it would take to remove derived files
> from the CVS repository, and it doesn't look bad at all.  I propose
> we do so before 6.5 beta.
> 
> In case anyone's forgotten: the issue is derived files, such as gram.c,
> which we currently keep in the CVS repository even though they are not
> master source files.  Doing so causes a number of headaches, including
> wasted time to check in and check out updates to both master and derived
> files, unreasonable bulk of the CVS files for these derived files,
> errors due to timestamp skew (after checking out, it can look like you
> have an up-to-date derived file when you do not), etc etc.

We have not been able to reliably make releases with the proper
timestamps on gram.c, which is critical for end-users, so any change
that will make this gram.c more automatic is welcomed by me.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

  3. Re: [HACKERS] Removing derived files from CVS

    Clark C. Evans <clark.evans@manhattanproject.com> — 1999-03-19T02:50:59Z

    Tom Lane wrote:
> 
> I have been looking into what it would take to remove derived files
> from the CVS repository, and it doesn't look bad at all.  I propose
> we do so before 6.5 beta.

Sure, as long as it is clear what additional tools are need, what are 
their versions, and where do I get them for common platforms.

:) Clark

  4. Re: [HACKERS] Removing derived files from CVS

    Marc G. Fournier <scrappy@hub.org> — 1999-03-19T13:40:59Z

    On Thu, 18 Mar 1999, Bruce Momjian wrote:

> > I have been looking into what it would take to remove derived files
> > from the CVS repository, and it doesn't look bad at all.  I propose
> > we do so before 6.5 beta.
> > 
> > In case anyone's forgotten: the issue is derived files, such as gram.c,
> > which we currently keep in the CVS repository even though they are not
> > master source files.  Doing so causes a number of headaches, including
> > wasted time to check in and check out updates to both master and derived
> > files, unreasonable bulk of the CVS files for these derived files,
> > errors due to timestamp skew (after checking out, it can look like you
> > have an up-to-date derived file when you do not), etc etc.
> 
> We have not been able to reliably make releases with the proper
> timestamps on gram.c, which is critical for end-users, so any change
> that will make this gram.c more automatic is welcomed by me.

Agreed here too...someone at one point mentioned that there might be a
way, inside of CVS, to have it auto-generate these files as its being
checked out (ie. if file is configure.in, run autoconf)...

I just scan'd through the cvs info file, and couldn't find
anything...anyone know about something like this?

Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
Systems Administrator @ hub.org 
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org

  5. Re: [HACKERS] Removing derived files from CVS

    Brian E Gallew <geek+@cmu.edu> — 1999-03-19T13:55:36Z

    Then <scrappy@hub.org> spoke up and said:
> Agreed here too...someone at one point mentioned that there might be a
> way, inside of CVS, to have it auto-generate these files as its being
> checked out (ie. if file is configure.in, run autoconf)...

>From the info file:
Module options
--------------

   Either regular modules or ampersand modules can contain options,
which supply additional information concerning the module.
[snip]
`-i PROG'
     Specify a program PROG to run whenever files in a module are
     committed.  PROG runs with a single argument, the full pathname of
     the affected directory in a source repository.  The `commitinfo',
     `loginfo', and `verifymsg' files provide other ways to call a
     program on commit.

`-o PROG'
     Specify a program PROG to run whenever files in a module are
     checked out.  PROG runs with a single argument, the module name.

>From my reading, it looks like the easiest thing to do is set up
commit rules such that committing gram.y automatically generates
gram.c.  It looks like it might be difficult to have gram.c generated
completely "on the fly" and then passed to the CVS client.

-- 
=====================================================================
| JAVA must have been developed in the wilds of West Virginia.      |
| After all, why else would it support only single inheritance??    |
=====================================================================
| Finger geek@cmu.edu for my public key.                            |
=====================================================================

  6. Re: [HACKERS] Removing derived files from CVS

    Marc G. Fournier <scrappy@hub.org> — 1999-03-19T18:45:07Z

    On 19 Mar 1999 geek+@cmu.edu wrote:

> Then <scrappy@hub.org> spoke up and said:
> > Agreed here too...someone at one point mentioned that there might be a
> > way, inside of CVS, to have it auto-generate these files as its being
> > checked out (ie. if file is configure.in, run autoconf)...
> 
> >From the info file:
> Module options
> --------------
> 
>    Either regular modules or ampersand modules can contain options,
> which supply additional information concerning the module.
> [snip]
> `-i PROG'
>      Specify a program PROG to run whenever files in a module are
>      committed.  PROG runs with a single argument, the full pathname of
>      the affected directory in a source repository.  The `commitinfo',
>      `loginfo', and `verifymsg' files provide other ways to call a
>      program on commit.
> 
> `-o PROG'
>      Specify a program PROG to run whenever files in a module are
>      checked out.  PROG runs with a single argument, the module name.
> 
> >From my reading, it looks like the easiest thing to do is set up
> commit rules such that committing gram.y automatically generates
> gram.c.  It looks like it might be difficult to have gram.c generated
> completely "on the fly" and then passed to the CVS client.

Can you provide an exampmle of using/doing this?  It sounds like the
better solution of them all, if it can be done this way..

Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
Systems Administrator @ hub.org 
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org