Thread

Feature Freeze date for 8.4

Simon Riggs <simon@2ndquadrant.com> — 2007-10-22T18:43:28Z

For planning purposes, I think its always a good idea to lay down some
dates for the next lot of development milestones. These can be
provisional, until declared solid later.

So: When is the next Feature Freeze? 

Is it March 31? If not, when?

I know the answer is "too early to tell for certain", but what will it
be if we release in early Dec/early Jan, or whenever.

If we really are very uncertain, lets at least say it will be "Not
Before DateX". PPPPPPP and all that.

AFAICS, more than 50% of the patches are written by professional
developers, so our various sponsors need to know when the next release
is due. Kinda. Ish.

Thank you.

-- 
  Simon Riggs
  2ndQuadrant  http://www.2ndQuadrant.com

Re: Feature Freeze date for 8.4

Joshua D. Drake <jd@commandprompt.com> — 2007-10-22T18:53:44Z

On Mon, 22 Oct 2007 19:43:28 +0100
Simon Riggs <simon@2ndquadrant.com> wrote:

> For planning purposes, I think its always a good idea to lay down some
> dates for the next lot of development milestones. These can be
> provisional, until declared solid later.

We can't not realistically consider this until we at least come up with
a release date for 8.3.

I seem to recall that we were originally going to release 8.3 in June.

Jsohua D. Drake


> 
> So: When is the next Feature Freeze? 
> 
> Is it March 31? If not, when?
> 
> I know the answer is "too early to tell for certain", but what will it
> be if we release in early Dec/early Jan, or whenever.
> 
> If we really are very uncertain, lets at least say it will be "Not
> Before DateX". PPPPPPP and all that.
> 
> AFAICS, more than 50% of the patches are written by professional
> developers, so our various sponsors need to know when the next release
> is due. Kinda. Ish.
> 
> Thank you.
> 


-- 

      === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564   24x7/Emergency: +1.800.492.2240
PostgreSQL solutions since 1997  http://www.commandprompt.com/
			UNIQUE NOT NULL
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/

Re: Feature Freeze date for 8.4

Simon Riggs <simon@2ndquadrant.com> — 2007-10-22T19:12:26Z

On Mon, 2007-10-22 at 11:53 -0700, Joshua D. Drake wrote:
> On Mon, 22 Oct 2007 19:43:28 +0100
> Simon Riggs <simon@2ndquadrant.com> wrote:
> 
> > For planning purposes, I think its always a good idea to lay down some
> > dates for the next lot of development milestones. These can be
> > provisional, until declared solid later.
> 
> We can't not realistically consider this until we at least come up with
> a release date for 8.3.

There's always a way of planning through unknowns.

We can issue a provisional date. We could also say "at least 6 months
after release date of 8.3". I'm sure there's other options too.

-- 
  Simon Riggs
  2ndQuadrant  http://www.2ndQuadrant.com

Re: Feature Freeze date for 8.4

Josh Berkus <josh@agliodbs.com> — 2007-10-22T19:37:30Z

Simon,

> We can issue a provisional date. We could also say "at least 6 months
> after release date of 8.3". I'm sure there's other options too.

I'm going to suggest 4 months after 8.3.  8.3 was supposed to be a *short* 
release so that we could move our calendar around.  HOT and some of the 
other unexpected massive patches prevented that.  Again, we have enough in 
the "deferred for 8.4" queue that if we finished up only that it would 
qualify as a release.  

So my thought is, shoot for a short release so that we can get away from 
summer consolidations and December releases, and extend the cycle if 
someone dumps another 50,000 lines of attractive patches on us.

In fact, I could see doing a "no-catalog-changes, no major patches we don't 
already know about, 6-month release".  It would reset our cycle and get 
PL/proxy, DSM, clustered indexes, etc. out the door.   It could mean 
turning away patches which look attractive, though, so the whole community 
has to be into this.

-- 
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco

Re: Feature Freeze date for 8.4

Alvaro Herrera <alvherre@commandprompt.com> — 2007-10-22T19:47:41Z

Josh Berkus wrote:

> In fact, I could see doing a "no-catalog-changes, no major patches we don't 
> already know about, 6-month release".  It would reset our cycle and get 
> PL/proxy, DSM, clustered indexes, etc. out the door.   It could mean 
> turning away patches which look attractive, though, so the whole community 
> has to be into this.

Ah, you mean like we planned for 8.0 and failed, then for 8.1 and
failed, then for 8.2 and failed, then for 8.3 and failed?  I can
definitely support that idea.

-- 
Alvaro Herrera                          Developer, http://www.PostgreSQL.org/
"We are who we choose to be", sang the goldfinch
when the sun is high (Sandman)

Re: Feature Freeze date for 8.4

Andrew Dunstan <andrew@dunslane.net> — 2007-10-22T19:47:48Z

Josh Berkus wrote:
> So my thought is, shoot for a short release so that we can get away from 
> summer consolidations and December releases, and extend the cycle if 
> someone dumps another 50,000 lines of attractive patches on us.
>
>   

Before we settle on any dates I think we should have some discussion 
about how we can shorten the period between feature freeze and beta, 
which was far too long this time. Perhaps we need to be more aggressive 
about what what makes the cut and what doesn't.

cheers

andrew

Re: Feature Freeze date for 8.4

Joshua D. Drake <jd@commandprompt.com> — 2007-10-22T19:53:50Z

On Mon, 22 Oct 2007 16:47:41 -0300
Alvaro Herrera <alvherre@commandprompt.com> wrote:

> Josh Berkus wrote:
> 
> > In fact, I could see doing a "no-catalog-changes, no major patches
> > we don't already know about, 6-month release".  It would reset our
> > cycle and get PL/proxy, DSM, clustered indexes, etc. out the
> > door.   It could mean turning away patches which look attractive,
> > though, so the whole community has to be into this.
> 
> Ah, you mean like we planned for 8.0 and failed, then for 8.1 and
> failed, then for 8.2 and failed, then for 8.3 and failed?  I can
> definitely support that idea.
> 
As I recall 8.0 and 8.1 actually went pretty well.

Joshua D. Drake

-- 

      === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564   24x7/Emergency: +1.800.492.2240
PostgreSQL solutions since 1997  http://www.commandprompt.com/
			UNIQUE NOT NULL
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/

Re: Feature Freeze date for 8.4

Tom Lane <tgl@sss.pgh.pa.us> — 2007-10-22T20:34:43Z

"Joshua D. Drake" <jd@commandprompt.com> writes:
> Alvaro Herrera <alvherre@commandprompt.com> wrote:
>> Ah, you mean like we planned for 8.0 and failed, then for 8.1 and
>> failed, then for 8.2 and failed, then for 8.3 and failed?  I can
>> definitely support that idea.
>> 
> As I recall 8.0 and 8.1 actually went pretty well.

I don't recall any such plans for those releases, but certainly Josh's
proposal is *EXACTLY* what the plan was for 8.3, and look how well we
adhered to that one.

In point of fact, the big patches that aren't in 8.3 were rejected
because they weren't ready.  They won't get into 8.4, either, unless
someone does a lot more work on them.  So I don't follow this idea
of how we have a pre-loaded queue of good stuff all ready to go into
8.4.  We thought that was true for the 8.3 cycle, which it wasn't,
but there isn't even any basis to think that about 8.4.

			regards, tom lane

Re: Feature Freeze date for 8.4

Gregory Stark <stark@enterprisedb.com> — 2007-10-22T21:08:55Z

"Tom Lane" <tgl@sss.pgh.pa.us> writes:

> In point of fact, the big patches that aren't in 8.3 were rejected
> because they weren't ready.  They won't get into 8.4, either, unless
> someone does a lot more work on them.  So I don't follow this idea
> of how we have a pre-loaded queue of good stuff all ready to go into
> 8.4.  We thought that was true for the 8.3 cycle, which it wasn't,
> but there isn't even any basis to think that about 8.4.

Incidentally what big features do we have in progress?

I see:

. GII - there's been discussion about some kind of refactoring the index api
        to avoid the layer violations here.

. Bitmap Indexes - needs a design review and probably changes
                   possibly needs the same api refactoring as GII

. DSM - I think Heikki's idea to implement the storage via the buffer manager
        so it doesn't have fixed size storage limitations like the FSM is a
        good one

. Recursive Queries - I haven't really started the meat of it but wouldn't
                      mind feedback on the outline I posted a while back

There are some more in the developer.postgresql.org patch status page but I'm
not too familiar with what's missing for those.

It does seem like most of these are blocked waiting on ideas rather than SMOP
issues, so I'm not sure counting on them to be ready on a particular schedule
is going to be especially safe. Of course the ideas are more likely to come
once we start discussing the issues. I imagine everyone's focused on the beta
right now.

-- 
  Gregory Stark
  EnterpriseDB          http://www.enterprisedb.com

Re: Feature Freeze date for 8.4

Chris Browne <cbbrowne@acm.org> — 2007-10-22T22:02:36Z

josh@agliodbs.com (Josh Berkus) writes:
> Simon,
>
>> We can issue a provisional date. We could also say "at least 6 months
>> after release date of 8.3". I'm sure there's other options too.
>
> I'm going to suggest 4 months after 8.3.  8.3 was supposed to be a *short* 
> release so that we could move our calendar around.  HOT and some of the 
> other unexpected massive patches prevented that.  Again, we have enough in 
> the "deferred for 8.4" queue that if we finished up only that it would 
> qualify as a release.  
>
> So my thought is, shoot for a short release so that we can get away from 
> summer consolidations and December releases, and extend the cycle if 
> someone dumps another 50,000 lines of attractive patches on us.
>
> In fact, I could see doing a "no-catalog-changes, no major patches we don't 
> already know about, 6-month release".  It would reset our cycle and get 
> PL/proxy, DSM, clustered indexes, etc. out the door.   It could mean 
> turning away patches which look attractive, though, so the whole community 
> has to be into this.

There are good things about that idea.

There would also be good things about picking a somewhat *longer*
cycle in that we already just had a cycle where the "feature freeze"
period was supposedly a short one, which precluded implementing
anything requiring more planning.

- It seems at least somewhat unfair to burden the 8.4 cycle with the
  "sins" of the 8.3 cycle.

- There is the risk that even with the restriction, 8.4 might still
  not be a short cycle, which would make the attempt futile.  

- And would we then say "hey, we need for 8.5 to have a shortened
  cycle too"?
-- 
(reverse (concatenate 'string "ofni.secnanifxunil" "@" "enworbbc"))
http://linuxfinances.info/info/multiplexor.html
Space   is   big.  Really    big.   You   won't   believe  how  vastly
mind-bogglingly big it is.  I mean, you may think it's a long way down
the road to the chemist, but that's just peanuts to space.  Listen....

Re: Feature Freeze date for 8.4

Tom Lane <tgl@sss.pgh.pa.us> — 2007-10-23T03:36:00Z

Andrew Dunstan <andrew@dunslane.net> writes:
> Before we settle on any dates I think we should have some discussion 
> about how we can shorten the period between feature freeze and beta, 
> which was far too long this time. Perhaps we need to be more aggressive 
> about what what makes the cut and what doesn't.

I think basically we need to redefine "feature freeze".  The definition
we effectively used for the last couple of cycles was "if you've posted
a patch, even a slushy one, by the stated FF date, you make the cut".
This was compounded in the 8.3 cycle by reviewers (and I'm looking at
myself here) figuring that we could postpone reviewing patches until
after FF because the policy didn't require that they be in good shape
*before* that date.  That would've worked OK if there were only a few
such patches, but we had a lot of big ones.

If we want a short FF-to-beta period then the criterion will have to be
that patches are either committed or darn near ready to commit on the FF
date.  No springing mostly-done patches on the community a few days
before FF.  And the reviewers will need to work harder on reviewing
stuff earlier, and committing before FF whenever possible.  And we need
to be much more ready to bounce stuff that's not quite done, rather than
drag out the cycle to let it get finished.

No, it doesn't sound like any fun :-(.  But this cycle was clearly
mismanaged.  It's not productive to have a freeze this long.

[ thinks for a bit... ]  A truly hard-nosed approach would be to define
FF as "if your patch isn't committed by the FF date, you lose".  The
FF-to-beta delay then is only long enough to make sure we've documented
everything, written release notes, etc.  I'm not sure this would be a
more pleasant way to work, as there'd be a heck of a lot of pressure on
the committers as the days tick down to FF.  But it'd fix the scheduling
problem.

			regards, tom lane

Re: Feature Freeze date for 8.4

Tom Lane <tgl@sss.pgh.pa.us> — 2007-10-23T03:56:02Z

Chris Browne <cbbrowne@acm.org> writes:
> josh@agliodbs.com (Josh Berkus) writes:
>> In fact, I could see doing a "no-catalog-changes, no major patches we don't 
>> already know about, 6-month release".  It would reset our cycle and get 
>> PL/proxy, DSM, clustered indexes, etc. out the door.   It could mean 
>> turning away patches which look attractive, though, so the whole community 
>> has to be into this.

> There are good things about that idea.

> There would also be good things about picking a somewhat *longer*
> cycle in that we already just had a cycle where the "feature freeze"
> period was supposedly a short one, which precluded implementing
> anything requiring more planning.

[ chewing on this a bit... ]  The curious thing about that is that
despite this being designed to be a short release cycle, we ended up
landing a bunch of major patches that weren't on the radar screen at
all at the start of the cycle.  This suggests to me that there's
something wrong with the concept that no one can get anything major done
without a long release cycle to do it in.

Indeed, the thing that seemed to me to be killing us in this freeze
cycle was that the patches coming in were already at the upper limit of
what we can review and digest.  I don't think I want to say "okay guys,
we'll give you a year so that you can write a 200K-line patch and drop
it on us the day before feature freeze".

I'd rather encourage people to work in an incremental, not-so-big-bang
fashion.  Obviously one of the requirements for that will be quicker
review turnaround and commit, so that there's time to build on a
previous patch...

			regards, tom lane

Re: Feature Freeze date for 8.4

Andrew Dunstan <andrew@dunslane.net> — 2007-10-23T05:01:59Z


Tom Lane wrote:
> Andrew Dunstan <andrew@dunslane.net> writes:
>   
>> Before we settle on any dates I think we should have some discussion 
>> about how we can shorten the period between feature freeze and beta, 
>> which was far too long this time. Perhaps we need to be more aggressive 
>> about what what makes the cut and what doesn't.
>>     
>
> I think basically we need to redefine "feature freeze".  The definition
> we effectively used for the last couple of cycles was "if you've posted
> a patch, even a slushy one, by the stated FF date, you make the cut".
> This was compounded in the 8.3 cycle by reviewers (and I'm looking at
> myself here) figuring that we could postpone reviewing patches until
> after FF because the policy didn't require that they be in good shape
> *before* that date.  That would've worked OK if there were only a few
> such patches, but we had a lot of big ones.
>
> If we want a short FF-to-beta period then the criterion will have to be
> that patches are either committed or darn near ready to commit on the FF
> date.  No springing mostly-done patches on the community a few days
> before FF.  And the reviewers will need to work harder on reviewing
> stuff earlier, and committing before FF whenever possible.  And we need
> to be much more ready to bounce stuff that's not quite done, rather than
> drag out the cycle to let it get finished.
>
> No, it doesn't sound like any fun :-(.  But this cycle was clearly
> mismanaged.  It's not productive to have a freeze this long.
>
> [ thinks for a bit... ]  A truly hard-nosed approach would be to define
> FF as "if your patch isn't committed by the FF date, you lose".  The
> FF-to-beta delay then is only long enough to make sure we've documented
> everything, written release notes, etc.  I'm not sure this would be a
> more pleasant way to work, as there'd be a heck of a lot of pressure on
> the committers as the days tick down to FF.  But it'd fix the scheduling
> problem.
>   

I'd settle for your earlier slightly porous formulation, at least for 
one go round. We don't seem very fond of hard and fast rules as a 
community (c.f. txid controversy)  :-) The target should be to get to 
beta within about a month from feature freeze, I think.

Maybe we need a better triage at the start of feature freeze which gives 
candidates a preliminary review, at least enough to say "this is/is not 
very close to being able to be applied". Anything that isn't, even if 
submitted well before freeze, misses out. If reviewers are more active 
then the candidate list would be small.


cheers

andrew

Re: Feature Freeze date for 8.4

Joshua D. Drake <jd@commandprompt.com> — 2007-10-23T05:15:03Z

Tom Lane wrote:
> Andrew Dunstan <andrew@dunslane.net> writes:
>> Before we settle on any dates I think we should have some discussion 
>> about how we can shorten the period between feature freeze and beta, 
>> which was far too long this time. Perhaps we need to be more aggressive 
>> about what what makes the cut and what doesn't.

> [ thinks for a bit... ]  A truly hard-nosed approach would be to define
> FF as "if your patch isn't committed by the FF date, you lose".  The
> FF-to-beta delay then is only long enough to make sure we've documented
> everything, written release notes, etc.  I'm not sure this would be a
> more pleasant way to work, as there'd be a heck of a lot of pressure on
> the committers as the days tick down to FF.  But it'd fix the scheduling
> problem.

I actually don't disagree with anything you have to say but would like 
to offer one other thought. Instead of a hard feature freeze, we offer a 
continual development model. In short, we develop, review, commit etc.. 
until we feel a need to release.

Certainly we don't want to be releasing every 4 years but then again, in 
reality 8.3 could have been released 6 months ago, should we decided, 
hey... Tsearch2 and Hot.. well they aren't done so they get 8.4 and 
perhaps because Tsearch2 and HOT are so cool, once they went in, even 
though it is only 6 months later, it would be worth while to release
8.4. If we followed this model, we would be pushing 8.4 beta1 right now, 
not 8.3.

Further the people wanting specific features of a specific release, 
don't have to wait 12-15 months to get them.

I recognize this would be a *lot* easier if we didn't have the initdb 
requirement but still... release early, release often.

I have really taken to the Ubuntu style of releasing. Every 6 months (or 
so) they release. Every 2 years (or so) the LTS.

Is there potential for that here? I don't know, but it seems worth 
exploring.

Sincerely,

Joshua D. Drake

Re: Feature Freeze date for 8.4

Greg Smith <gsmith@gregsmith.com> — 2007-10-23T05:55:47Z

On Mon, 22 Oct 2007, Tom Lane wrote:

> If we want a short FF-to-beta period then the criterion will have to be 
> that patches are either committed or darn near ready to commit on the FF 
> date.

I think you're stuck with a certain amount of schedule delay regardless of 
how mature code is at submission time when there's a large performance 
component involved, rather than strictly a feature one.  There was a lot 
of that in 8.3, where it seemed to me the benchmarking and similar 
quantifying of the true impact of the patch wasn't correlated so much with 
the code quality at submission time.  Good performance testing of any sort 
takes a long time, there's only so many people who can do it, and having a 
couple of different perspectives is almost mandatory to avoid optimizing 
only for a particular application type.  When you have a couple of such 
things in the pool, you're not going to get a lot of work done on multiple 
patches of that type in parallel, especially when there's any overlap 
between them.

I personally think that shorting the minor release cycle time too far is 
counterproductive anyway.  From the DBA and system administrator 
perspective, new version releases are a giant QA and maintenance mess. 
Better to have less of them that each add larger features rather than a 
more regular stream of small ones from where I'm sitting.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD

Re: Feature Freeze date for 8.4

Pavel Stehule <pavel.stehule@gmail.com> — 2007-10-23T07:03:53Z

> Further the people wanting specific features of a specific release,
> don't have to wait 12-15 months to get them.
>
> I recognize this would be a *lot* easier if we didn't have the initdb
> requirement but still... release early, release often.
>
> I have really taken to the Ubuntu style of releasing. Every 6 months (or
> so) they release. Every 2 years (or so) the LTS.
>
> Is there potential for that here? I don't know, but it seems worth
> exploring.
>

1x per year full version is well. And it's much better then in MySQL
or  in Firebird.  I thing so problem is in long patch queue. There are
lot of sleeping patches (enugh for 8.4 now), and isn't possible
transform to core it any short time (mainly with bombs like tsearch or
hot).

Pavel

Re: Feature Freeze date for 8.4

Martijn van Oosterhout <kleptog@svana.org> — 2007-10-23T07:49:31Z

On Mon, Oct 22, 2007 at 11:56:02PM -0400, Tom Lane wrote:
> I'd rather encourage people to work in an incremental, not-so-big-bang
> fashion.  Obviously one of the requirements for that will be quicker
> review turnaround and commit, so that there's time to build on a
> previous patch...

From my outsider point of view the big difference is that the
probablility of $RANDOM_PATCH being reviewed during features freeze is
very much greater than at any other time. So the best way to get a
patch completed is to post it sometime before FF and after FF use the
subsequent feedback to improve it. If we can fix the disparity I think
most of the problems will go away...

Ofcourse, posting just prior FF doesn't help, but I think that's a
chicken/egg problem. People do what works.

Have a ncie day,
-- 
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

Re: Feature Freeze date for 8.4

Tatsuo Ishii <ishii@postgresql.org> — 2007-10-23T08:28:14Z

> On Mon, 22 Oct 2007, Tom Lane wrote:
> 
> > If we want a short FF-to-beta period then the criterion will have to be 
> > that patches are either committed or darn near ready to commit on the FF 
> > date.
> 
> I think you're stuck with a certain amount of schedule delay regardless of 
> how mature code is at submission time when there's a large performance 
> component involved, rather than strictly a feature one.  There was a lot 
> of that in 8.3, where it seemed to me the benchmarking and similar 
> quantifying of the true impact of the patch wasn't correlated so much with 
> the code quality at submission time.  Good performance testing of any sort 
> takes a long time, there's only so many people who can do it, and having a 
> couple of different perspectives is almost mandatory to avoid optimizing 
> only for a particular application type.  When you have a couple of such 
> things in the pool, you're not going to get a lot of work done on multiple 
> patches of that type in parallel, especially when there's any overlap 
> between them.
> 
> I personally think that shorting the minor release cycle time too far is 
> counterproductive anyway.  From the DBA and system administrator 
> perspective, new version releases are a giant QA and maintenance mess. 
> Better to have less of them that each add larger features rather than a 
> more regular stream of small ones from where I'm sitting.

+1. Shorter release cycles are maybe good for fancy GUI oriented
applications, but not so good for DBMS.
--
Tatsuo Ishii
SRA OSS, Inc. Japan

Re: Feature Freeze date for 8.4

Gregory Stark <stark@enterprisedb.com> — 2007-10-23T08:32:12Z

"Tom Lane" <tgl@sss.pgh.pa.us> writes:

> [ thinks for a bit... ]  A truly hard-nosed approach would be to define
> FF as "if your patch isn't committed by the FF date, you lose".  The
> FF-to-beta delay then is only long enough to make sure we've documented
> everything, written release notes, etc.  

You're assuming nothing committed could possibly be broken. I think "normal"
feature freeze semantics are that you freeze adding new features so you can
get the features that are committed working properly.

While it's nice that we have such a high standard for the patches that are
committed it's also kind of limiting us. For example, I think it worked pretty
well when you committed HOT and then based on experience and discussion added
the per-page xid. Insisting on every patch being 100% perfected before it gets
committed makes it hard to collaborate since it limits the visibility of the
work in progress. 

I do understand the concern that committing lots of broken or partial patches
will make it hard for others to work on unrelated work and make for lots of
work cleaning up the tree later. Tools like git/monotone/svk might help there.
But even with CVS there ought to be a better tradeoff than pushing all the
development into patches and requiring the CVS tree to be 100% perfected
complete patches. Working with patches over email is painful.

-- 
  Gregory Stark
  EnterpriseDB          http://www.enterprisedb.com

Re: Feature Freeze date for 8.4

Gregory Stark <stark@enterprisedb.com> — 2007-10-23T08:39:39Z

"Tom Lane" <tgl@sss.pgh.pa.us> writes:

> I'd rather encourage people to work in an incremental, not-so-big-bang
> fashion.  Obviously one of the requirements for that will be quicker
> review turnaround and commit, so that there's time to build on a
> previous patch...

I'll second that. It's awfully hard to work in isolation building a larger and
larger patch for months without being sure that you're even on the right
track...

Note that quicker turnaround time doesn't have to mean actually doing reviews
faster. It just means not waiting until the patch is in release-ready state.
It's latency that's hurting us, not bandwidth...

-- 
  Gregory Stark
  EnterpriseDB          http://www.enterprisedb.com

Re: Feature Freeze date for 8.4

Rafael Martinez <r.m.guerrero@usit.uio.no> — 2007-10-23T09:00:59Z

Tatsuo Ishii wrote:

> 
> +1. Shorter release cycles are maybe good for fancy GUI oriented
> applications, but not so good for DBMS.
> --

I agree, sure it will be great to have even more and new features as
soon as possible, but not if the quality of the final product decrease.
The most important characteristics of postgresql are stability,
,reliability and performance. It just works. We should not forget this.

And as others have said, from a DBA and system administrator
perspective, maintaining large installations updated in a 6 months
release cycle will be a nightmare.

We are always 1 year back the main release. We are testing and planing
the move to 8.2 now, and it won't happen until desember. In a 6 month
cycle we will have to jump over every second release.

regards
-- 
 Rafael Martinez, <r.m.guerrero@usit.uio.no>
 Center for Information Technology Services
 University of Oslo, Norway

 PGP Public Key: http://folk.uio.no/rafael/

Re: Feature Freeze date for 8.4

Csaba Nagy <nagy@ecircle-ag.com> — 2007-10-23T09:14:41Z

On Tue, 2007-10-23 at 11:00 +0200, Rafael Martinez wrote:
> We are always 1 year back the main release. We are testing and planing
> the move to 8.2 now, and it won't happen until desember. In a 6 month
> cycle we will have to jump over every second release.

We here are also just in the process of upgrading to 8.2, so in
principle I agree... except nobody is forced to go through ALL versions
- skipping over every second release is perfectly OK... but: shorter
releases would allow people to install only releases which actually
bring performance/functionality improvements they really need. All other
releases can be skipped... in fact you could possibly go for installing
a specific release only on some of the DBs based on what you need there.

The main question is if shorter release will impact quality - from a
user point of view (and I can only give my opinion in this respect) I
don't see any other possible disadvantage...

Cheers,
Csaba.

Re: Feature Freeze date for 8.4

Simon Riggs <simon@2ndquadrant.com> — 2007-10-23T09:18:34Z

On Mon, 2007-10-22 at 23:36 -0400, Tom Lane wrote:
> Andrew Dunstan <andrew@dunslane.net> writes:
> > Before we settle on any dates I think we should have some discussion 
> > about how we can shorten the period between feature freeze and beta, 
> > which was far too long this time. Perhaps we need to be more aggressive 
> > about what what makes the cut and what doesn't.
> 
> I think basically we need to redefine "feature freeze". 

Yes, I think so.

>  But this cycle was clearly
> mismanaged.  It's not productive to have a freeze this long.

That's a good starting point. We need to be objective about what worked
and what didn't.

I'm very happy about the way we handled performance in this release. We
took time to measure things, analyse the problems and then fix them with
new code that hadn't even been envisaged before FF. So I'd suggest we
have formal phase in the process *after* the code has been fully
integrated where we can measure performance and tune anything we need
to.

> [ thinks for a bit... ]  A truly hard-nosed approach would be to define
> FF as "if your patch isn't committed by the FF date, you lose".  The
> FF-to-beta delay then is only long enough to make sure we've documented
> everything, written release notes, etc.  I'm not sure this would be a
> more pleasant way to work, as there'd be a heck of a lot of pressure on
> the committers as the days tick down to FF.  But it'd fix the scheduling
> problem.

I'd be happy with that approach. 

My feeling is there is more than one problem that is being discussed
here. Some developers have complained that it sometimes takes a long
time to get a patch reviewed and committed. Committers are clearly
swamped by the volume of patches and we need to avoid another FF peak.

Generally, the project does a good job of committing work as it arrives.
Some patches do get missed and so the transit time through the patch
queue can be very long in some cases.

I'd suggest we have multiple checkpoints during the cycle. Checkpoint is
a "patch queue blitz" where we stop developing and reduce the queue to
nothing. Perhaps a two-week period where everybody helps reduce the
queue, not just Tom and Bruce. Every outstanding patch gets told what
they need to do in order to get it committed. FF is then just the last
in a series of checkpoints. Suggest we do a checkpoint every 2 months.

-- 
  Simon Riggs
  2ndQuadrant  http://www.2ndQuadrant.com

Re: Feature Freeze date for 8.4

Simon Riggs <simon@2ndquadrant.com> — 2007-10-23T09:19:56Z

On Mon, 2007-10-22 at 12:37 -0700, Josh Berkus wrote:
> Simon,
> 
> > We can issue a provisional date. We could also say "at least 6 months
> > after release date of 8.3". I'm sure there's other options too.
> 
> I'm going to suggest 4 months after 8.3.  8.3 was supposed to be a *short* 
> release so that we could move our calendar around.  HOT and some of the 
> other unexpected massive patches prevented that.  Again, we have enough in 
> the "deferred for 8.4" queue that if we finished up only that it would 
> qualify as a release.  
> 
> So my thought is, shoot for a short release so that we can get away from 
> summer consolidations and December releases, and extend the cycle if 
> someone dumps another 50,000 lines of attractive patches on us.
> 
> In fact, I could see doing a "no-catalog-changes, no major patches we don't 
> already know about, 6-month release".  It would reset our cycle and get 
> PL/proxy, DSM, clustered indexes, etc. out the door.   It could mean 
> turning away patches which look attractive, though, so the whole community 
> has to be into this.

Those dates shocked me when I first read them, but I feel we need to
have an aggressive release schedule, so I'll support that.

I hope that the single most important criterion for deciding the release
date(s) is How to Get the Most Good New Features into Postgres. That
should be more important than other considerations, but I guess in the
end, turning patches away may be the only way to ever do a release.

-- 
  Simon Riggs
  2ndQuadrant  http://www.2ndQuadrant.com

Re: Feature Freeze date for 8.4

Tom Lane <tgl@sss.pgh.pa.us> — 2007-10-23T13:29:58Z

Simon Riggs <simon@2ndquadrant.com> writes:
> I'd suggest we have multiple checkpoints during the cycle. Checkpoint is
> a "patch queue blitz" where we stop developing and reduce the queue to
> nothing. Perhaps a two-week period where everybody helps reduce the
> queue, not just Tom and Bruce. Every outstanding patch gets told what
> they need to do in order to get it committed. FF is then just the last
> in a series of checkpoints. Suggest we do a checkpoint every 2 months.

I like this idea ...

			regards, tom lane

Re: Feature Freeze date for 8.4

Josh Berkus <josh@agliodbs.com> — 2007-10-23T15:42:56Z

Tom, all:

> > I'd suggest we have multiple checkpoints during the cycle. Checkpoint is
> > a "patch queue blitz" where we stop developing and reduce the queue to
> > nothing. Perhaps a two-week period where everybody helps reduce the
> > queue, not just Tom and Bruce. Every outstanding patch gets told what
> > they need to do in order to get it committed. FF is then just the last
> > in a series of checkpoints. Suggest we do a checkpoint every 2 months.
>
> I like this idea ...

This would also avoid some bit-rot.  However, I'd suggest every 6 weeks; every 
2 months would only cycle 2-4 times during a development cycle, and every 
month would be too frequently.  So, say that 8.4 dev officially began on 
December 1, it would be:

Dec 1 - Jan 1: development
Jan 1 - Jan 14: queue blitz
Jan 15 - Feb 14: development
Feb 15 - Feb 28: queue blitz

...

June 1: Final patch deadline, 100% stable except for performance/docs, or you 
wait for 8.5.

The other thing we really really need to make this work is *tracking*.  For 
example, I have access to performance and quality testing resources at Sun, 
but it needs to be crystal clear to my team which patches are ready for 
testing, where to get them, how to apply them, and what to test.  And it 
can't be dependant on reading 100 e-mail messages a day and filtering for 
that information the way it currently is.

So we should start doing Stefan's patch grid from day 1 of 8.4, and all patch 
submitters should register with developer.postgresql.org and keep their patch 
information updated.  Do we have any way to upload patch files to the wiki?   
If not, where can we put them?

Or do we want to do a real patch manager?

-- 
Josh Berkus
PostgreSQL @ Sun
San Francisco

Re: Feature Freeze date for 8.4

Joshua D. Drake <jd@commandprompt.com> — 2007-10-23T16:22:31Z

On Tue, 23 Oct 2007 17:28:14 +0900 (JST)
Tatsuo Ishii <ishii@postgresql.org> wrote:

> > On Mon, 22 Oct 2007, Tom Lane wrote:

> > I personally think that shorting the minor release cycle time too
> > far is counterproductive anyway.  From the DBA and system
> > administrator perspective, new version releases are a giant QA and
> > maintenance mess. Better to have less of them that each add larger
> > features rather than a more regular stream of small ones from where
> > I'm sitting.
> 
> +1. Shorter release cycles are maybe good for fancy GUI oriented
> applications, but not so good for DBMS.

Well my original post was only an example. Consider that it could be 1
month, 8 months, 2 years...

The point is there is specific "feature freeze".

We develop and commit like normal *until* the community feels there is
enough for release. Then we announce a feature freeze.

The idea here is that... I could easily argue that 8.3 is enough of a
release with *just* HOT and TSearch2. Or that all the other stuff is
enough for a release without HOT and Tsearch2.

So why did we waste all summer reviewing HOT and TSearch2? We could
have released... (That isn't to say that HOT and Tsearch2 are worth the
effort, they obviously are). After release of 8.3, we could have picked
up immediately on HOT and Tsearch2 and perhaps released 8.4 with other
incrementals in about a month.

There is nothing wrong with that in the least. People "can" choose not
to run a particular version of software, further the community "can"
determine which versions of PostgreSQL will be supported for our
traditional 3+ years.

Joshua D. Drake

> --
> Tatsuo Ishii
> SRA OSS, Inc. Japan
> 
> ---------------------------(end of
> broadcast)--------------------------- TIP 7: You can help support the
> PostgreSQL project by donating at
> 
>                 http://www.postgresql.org/about/donate
> 

-- 

      === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564   24x7/Emergency: +1.800.492.2240
PostgreSQL solutions since 1997  http://www.commandprompt.com/
			UNIQUE NOT NULL
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/

Re: Feature Freeze date for 8.4

Joshua D. Drake <jd@commandprompt.com> — 2007-10-23T16:25:04Z

On Tue, 23 Oct 2007 11:00:59 +0200
Rafael Martinez <r.m.guerrero@usit.uio.no> wrote:

> Tatsuo Ishii wrote:
> 
> > 
> > +1. Shorter release cycles are maybe good for fancy GUI oriented
> > applications, but not so good for DBMS.
> > --
> 
> We are always 1 year back the main release. We are testing and planing
> the move to 8.2 now, and it won't happen until desember. In a 6 month
> cycle we will have to jump over every second release.

Yes you would have to jump over every second release. I fail to see how
this matters? I don't understand the argument.

Sincerely,

Joshua D. Drake

> 
> regards


-- 

      === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564   24x7/Emergency: +1.800.492.2240
PostgreSQL solutions since 1997  http://www.commandprompt.com/
			UNIQUE NOT NULL
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/

Re: Feature Freeze date for 8.4

David Fetter <david@fetter.org> — 2007-10-23T16:26:47Z

On Tue, Oct 23, 2007 at 09:39:39AM +0100, Gregory Stark wrote:
> 
> "Tom Lane" <tgl@sss.pgh.pa.us> writes:
> 
> > I'd rather encourage people to work in an incremental,
> > not-so-big-bang fashion.  Obviously one of the requirements for
> > that will be quicker review turnaround and commit, so that there's
> > time to build on a previous patch...
> 
> I'll second that. It's awfully hard to work in isolation building a
> larger and larger patch for months without being sure that you're
> even on the right track...

It would be much easier with a distributed SCM system because that way
the process of checking patch integration would be branching and
merging, these being cheap and simple operations in DSCMs like git,
where it's pretty nearly impossibly hard with CVS.

You can start using git right away :)

http://repo.or.cz/w/PostgreSQL.git

> Note that quicker turnaround time doesn't have to mean actually
> doing reviews faster. It just means not waiting until the patch is
> in release-ready state.  It's latency that's hurting us, not
> bandwidth...

A DSCM can do a *lot* to get this latency down :)

Cheers,
David (not advocating any particular DSCM at the moment, just the
process of using one)
-- 
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david.fetter@gmail.com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

Re: Feature Freeze date for 8.4

Joshua D. Drake <jd@commandprompt.com> — 2007-10-23T16:28:59Z

On Tue, 23 Oct 2007 09:29:58 -0400
Tom Lane <tgl@sss.pgh.pa.us> wrote:

> Simon Riggs <simon@2ndquadrant.com> writes:
> > I'd suggest we have multiple checkpoints during the cycle.
> > Checkpoint is a "patch queue blitz" where we stop developing and
> > reduce the queue to nothing. Perhaps a two-week period where
> > everybody helps reduce the queue, not just Tom and Bruce. Every
> > outstanding patch gets told what they need to do in order to get it
> > committed. FF is then just the last in a series of checkpoints.
> > Suggest we do a checkpoint every 2 months.
> 
> I like this idea ...

As do I. It will also allow us to decrease the amount of changes that
have to be reviewed for regressions during testing.

I know I just love it when a customer breaks something and I ask what
changed and it is 56 different things ;)

My question is.. with a checkpoint every 2 months, would it make it
very easy to release every 6 (or 4 or 3 or 9) months? I am not saying
we "have" to but it certainly opens up the possibility to the argument
I made. 

With a 2 months checkpoint, we can release when we want. When we feel
we have enough and not have a ginormous back log of patches to go
through.

Sincerely,

Joshua D. Drake




> 
> 			regards, tom lane
> 
> ---------------------------(end of
> broadcast)--------------------------- TIP 7: You can help support the
> PostgreSQL project by donating at
> 
>                 http://www.postgresql.org/about/donate
> 


-- 

      === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564   24x7/Emergency: +1.800.492.2240
PostgreSQL solutions since 1997  http://www.commandprompt.com/
			UNIQUE NOT NULL
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/

Re: Feature Freeze date for 8.4

Brendan Jurd <direvus@gmail.com> — 2007-10-23T16:42:36Z

On 10/24/07, David Fetter <david@fetter.org> wrote:
> On Tue, Oct 23, 2007 at 09:39:39AM +0100, Gregory Stark wrote:
> >
> > "Tom Lane" <tgl@sss.pgh.pa.us> writes:
> >
> > > I'd rather encourage people to work in an incremental,
> > > not-so-big-bang fashion.  Obviously one of the requirements for
> > > that will be quicker review turnaround and commit, so that there's
> > > time to build on a previous patch...
> >
> > I'll second that. It's awfully hard to work in isolation building a
> > larger and larger patch for months without being sure that you're
> > even on the right track...

+1.  I think use of branches would be a big improvement.

>
> It would be much easier with a distributed SCM system because that way
> the process of checking patch integration would be branching and
> merging, these being cheap and simple operations in DSCMs like git,
> where it's pretty nearly impossibly hard with CVS.

It would be much easier with any modern SCM system at all, distributed
or not.  I've been using feature branches in Subversion for a couple
years now and it's been a pleasure.

I'm not trying to derail the discussion into a SCM jihad or anything,
just pointing out that any move *away* from CVS, in any direction, is
a good move.

Re: Feature Freeze date for 8.4

Andrew Dunstan <andrew@dunslane.net> — 2007-10-23T16:45:14Z

Joshua D. Drake wrote:
> We develop and commit like normal *until* the community feels there is
> enough for release. Then we announce a feature freeze.
>
>   

No, I think this is hopeless on several grounds. First, it increases 
uncertainty. People need to be able to work towards a target. Second, 
there will always be a temptation to "just squeeze in killer feature X".

I think there is some virtue in Simon's suggestion, provided that 
adequate reviewer resources are available. That might mean you need to 
put on your fundraising hat and raise some reviewer sponsorship money.

cheers

andrew

Re: Feature Freeze date for 8.4

Joshua D. Drake <jd@commandprompt.com> — 2007-10-23T17:07:35Z

On Tue, 23 Oct 2007 12:45:14 -0400
Andrew Dunstan <andrew@dunslane.net> wrote:

> 
> 
> Joshua D. Drake wrote:
> > We develop and commit like normal *until* the community feels there
> > is enough for release. Then we announce a feature freeze.
> >
> >   
> 
> No, I think this is hopeless on several grounds. First, it increases 
> uncertainty. People need to be able to work towards a target. Second, 
> there will always be a temptation to "just squeeze in killer feature
> X".

Well that flows into that checkpoint idea I think.

> 
> I think there is some virtue in Simon's suggestion, provided that 
> adequate reviewer resources are available. That might mean you need
> to put on your fundraising hat and raise some reviewer sponsorship
> money.

Did someone say money? ;)

Sincerely,

Joshua D. Drake

> 
> cheers
> 
> andrew
> 


-- 

      === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564   24x7/Emergency: +1.800.492.2240
PostgreSQL solutions since 1997  http://www.commandprompt.com/
			UNIQUE NOT NULL
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/

Re: Feature Freeze date for 8.4

Gregory Stark <stark@enterprisedb.com> — 2007-10-23T17:47:39Z

"Joshua D. Drake" <jd@commandprompt.com> writes:

> I know I just love it when a customer breaks something and I ask what
> changed and it is 56 different things ;)
>
> My question is.. with a checkpoint every 2 months, would it make it
> very easy to release every 6 (or 4 or 3 or 9) months? I am not saying
> we "have" to but it certainly opens up the possibility to the argument
> I made. 

We could release "alpha" releases. But that assumes that these reviews
actually result in stuff getting committed even if they're not 100% complete.
I think that would be a good thing but I don't think everyone else agrees.
Also, not all reviewers are committers.

-- 
  Gregory Stark
  EnterpriseDB          http://www.enterprisedb.com

Re: Feature Freeze date for 8.4

Josh Berkus <josh@agliodbs.com> — 2007-10-23T17:55:50Z

All,

> We could release "alpha" releases. But that assumes that these reviews
> actually result in stuff getting committed even if they're not 100%
> complete. I think that would be a good thing but I don't think everyone
> else agrees. Also, not all reviewers are committers.

This is what I'm thinking, too.  It would be a *lot* easier for the Sun 
peformance team (and presumably others) to test performance of stuff which 
was getting committed rather than having to hunt down *this* version of 
*that* patch and apply it against the snapshot from *this specific date* ...

(puts PR hat on) it would also allow me to keep up a stream of info to the 
community and tech press about the new features and optimizations we're 
working on, which would mean better coverage for each feature.

-- 
Josh Berkus
PostgreSQL @ Sun
San Francisco

Re: Feature Freeze date for 8.4

Magnus Hagander <magnus@hagander.net> — 2007-10-23T17:58:43Z

Gregory Stark wrote:
> "Joshua D. Drake" <jd@commandprompt.com> writes:
> 
>> I know I just love it when a customer breaks something and I ask what
>> changed and it is 56 different things ;)
>>
>> My question is.. with a checkpoint every 2 months, would it make it
>> very easy to release every 6 (or 4 or 3 or 9) months? I am not saying
>> we "have" to but it certainly opens up the possibility to the argument
>> I made. 
> 
> We could release "alpha" releases. But that assumes that these reviews
> actually result in stuff getting committed even if they're not 100% complete.

That's sounding a lot like mysql.. Next you'll start recommending people
to run that in production.. :-P

//Magnus

Re: Feature Freeze date for 8.4

Gregory Stark <stark@enterprisedb.com> — 2007-10-23T18:16:53Z

"Josh Berkus" <josh@agliodbs.com> writes:

> All,
>
>> We could release "alpha" releases. But that assumes that these reviews
>> actually result in stuff getting committed even if they're not 100%
>> complete. I think that would be a good thing but I don't think everyone
>> else agrees. Also, not all reviewers are committers.
>
> This is what I'm thinking, too.  It would be a *lot* easier for the Sun 
> peformance team (and presumably others) to test performance of stuff which 
> was getting committed rather than having to hunt down *this* version of 
> *that* patch and apply it against the snapshot from *this specific date* ...

Fwiw I had put together a jumbo patch for three of the patches which you were
interested in, HOT, LDC, and GII. This was specifically for the benefit of
users like you to get an early experience of these patches. I think it was
actually shortly after you had reported a problem running them together which
was blocking you from running benchmarks on them.

The problem I ran into was that by the time I had them all wrapped up major
new commits to the CVS tree made it uninteresting to benchmark the snapshot I
had. Also I think a new version of HOT had been posted.

-- 
  Gregory Stark
  EnterpriseDB          http://www.enterprisedb.com

Re: Feature Freeze date for 8.4

Josh Berkus <josh@agliodbs.com> — 2007-10-23T18:40:23Z

Greg,

> The problem I ran into was that by the time I had them all wrapped up major
> new commits to the CVS tree made it uninteresting to benchmark the snapshot
> I had. Also I think a new version of HOT had been posted.

Yeah, that's exactly what I was thinking of.  The Sun benchmarking folks would 
have done more testing of HOT, much earlier, if not for those issues.  

Anyway, is there anyone who thinks the "cycle the queue every 6 weeks or 2 
months or suitable short period" is a *bad* idea?   It might be hard to pull 
off, but we won't know until we try.

It'll also help keep vacation/summer schedules from being an issue for release 
timelines, since review will go on year-round and not just in the n.h. 
summer.  And, as a bonus for the reviewers, this means that rather than 
having to devote yourselves full-time to review for 4 months out of the year, 
you can space it out in short bursts so that you don't get worn out looking 
at other people's code instead of working on your own for months!

I'm excited about this, and think it's the best suggestion I've heard in years 
for how to improve our review process.  Can we do it?

-- 
Josh Berkus
PostgreSQL @ Sun
San Francisco

Re: Feature Freeze date for 8.4

Andrew Dunstan <andrew@dunslane.net> — 2007-10-23T18:46:22Z

Gregory Stark wrote:
>
> The problem I ran into was that by the time I had them all wrapped up major
> new commits to the CVS tree made it uninteresting to benchmark the snapshot I
> had.
>   

That's going to be a problem with any snapshot approach. For the most 
part, interesting patches are going to hit a number of common hotspots ISTM.

I'm fairly resistant to putting less-than-ready code in the tree, I must 
say.

+1 to the idea of patches in smaller pieces that has been mentioned 
upthread. I tried to do that in one or two places recently, but it could 
go much further.

cheers

andrew

Re: Feature Freeze date for 8.4

Alvaro Herrera <alvherre@commandprompt.com> — 2007-10-23T18:57:17Z

Gregory Stark wrote:
> "Josh Berkus" <josh@agliodbs.com> writes:
> 
> > All,
> >
> >> We could release "alpha" releases. But that assumes that these reviews
> >> actually result in stuff getting committed even if they're not 100%
> >> complete. I think that would be a good thing but I don't think everyone
> >> else agrees. Also, not all reviewers are committers.
> >
> > This is what I'm thinking, too.  It would be a *lot* easier for the Sun 
> > peformance team (and presumably others) to test performance of stuff which 
> > was getting committed rather than having to hunt down *this* version of 
> > *that* patch and apply it against the snapshot from *this specific date* ...
> 
> Fwiw I had put together a jumbo patch for three of the patches which you were
> interested in, HOT, LDC, and GII. This was specifically for the benefit of
> users like you to get an early experience of these patches. I think it was
> actually shortly after you had reported a problem running them together which
> was blocking you from running benchmarks on them.

It would make a lot of sense to keep them in a branch of a distributed
SCMS where you can propagate stuff from mainline into the branches.

Maybe what we, developers, should do is start using a DSCM privately to
keep track of our patches and of mainline.  I know some derived projects
already use one (Postgres-R?)

-- 
Alvaro Herrera       Valdivia, Chile   ICBM: S 39º 49' 18.1", W 73º 13' 56.4"
"Limítate a mirar... y algun día veras"

Re: Feature Freeze date for 8.4

Chris Browne <cbbrowne@acm.org> — 2007-10-23T20:59:51Z

tgl@sss.pgh.pa.us (Tom Lane) writes:
> Simon Riggs <simon@2ndquadrant.com> writes:
>> I'd suggest we have multiple checkpoints during the cycle. Checkpoint is
>> a "patch queue blitz" where we stop developing and reduce the queue to
>> nothing. Perhaps a two-week period where everybody helps reduce the
>> queue, not just Tom and Bruce. Every outstanding patch gets told what
>> they need to do in order to get it committed. FF is then just the last
>> in a series of checkpoints. Suggest we do a checkpoint every 2 months.
>
> I like this idea ...

Don't we need to have a background writer process to gradually flush
this data so that checkpoints aren't so massive?  ;-)
-- 
let name="cbbrowne" and tld="acm.org" in String.concat "@" [name;tld];;
http://linuxfinances.info/info/oses.html
"If there are aliens, they use Lisp."
--Edward Lasker, mutatis mutandi

Re: Feature Freeze date for 8.4

David Fetter <david@fetter.org> — 2007-10-23T21:39:43Z

On Tue, Oct 23, 2007 at 04:59:51PM -0400, Chris Browne wrote:
> tgl@sss.pgh.pa.us (Tom Lane) writes:
> > Simon Riggs <simon@2ndquadrant.com> writes:
> >> I'd suggest we have multiple checkpoints during the cycle.
> >> Checkpoint is a "patch queue blitz" where we stop developing and
> >> reduce the queue to nothing. Perhaps a two-week period where
> >> everybody helps reduce the queue, not just Tom and Bruce. Every
> >> outstanding patch gets told what they need to do in order to get
> >> it committed. FF is then just the last in a series of
> >> checkpoints. Suggest we do a checkpoint every 2 months.
> >
> > I like this idea ...
> 
> Don't we need to have a background writer process to gradually flush
> this data so that checkpoints aren't so massive?  ;-)

I'm proposing a DSCM with easy branching and merging to implement this
background writer process :)

The one below is already available, so we don't have to do a "flag
day" with it.

http://repo.or.cz/w/PostgreSQL.git

There are git-cvsimport and git-cvsexport utilities which make
communication between the legacy SCM and the DSCM straight-forward.

Cheers,
David.
-- 
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david.fetter@gmail.com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

Re: Feature Freeze date for 8.4

Tom Lane <tgl@sss.pgh.pa.us> — 2007-10-23T21:50:39Z

Andrew Dunstan <andrew@dunslane.net> writes:
> I'm fairly resistant to putting less-than-ready code in the tree, I must 
> say.

Me too, at least if "less than ready" means "unstable".  The committed
code has to always be solid enough to let everybody continue working on
their own bits.  However, in the past we've tended to refuse stuff if
it "didn't do anything interesting", and I think that attitude will need
adjustment.  We'll have to be prepared to accept patches that are only
interesting as waystations to some long-term goal.

			regards, tom lane

Re: Feature Freeze date for 8.4

Tom Lane <tgl@sss.pgh.pa.us> — 2007-10-23T22:09:00Z

Josh Berkus <josh@agliodbs.com> writes:
> Anyway, is there anyone who thinks the "cycle the queue every 6 weeks or 2 
> months or suitable short period" is a *bad* idea?   It might be hard to pull 
> off, but we won't know until we try.

It seems worth a try --- we can certainly abandon it easily if it
doesn't work.

Personally I feel every six weeks would be too short: we'd be talking
only a month of work between commit-fests.  I like a two-month cycle
partly because it wouldn't rotate relative to the calendar: we'd always
know that the first half of every odd-numbered month, or something like
that, is commit-fest time.

			regards, tom lane

Re: Feature Freeze date for 8.4

Andrew Dunstan <andrew@dunslane.net> — 2007-10-23T22:15:58Z


Tom Lane wrote:
> Andrew Dunstan <andrew@dunslane.net> writes:
>   
>> I'm fairly resistant to putting less-than-ready code in the tree, I must 
>> say.
>>     
>
> Me too, at least if "less than ready" means "unstable".  The committed
> code has to always be solid enough to let everybody continue working on
> their own bits.  However, in the past we've tended to refuse stuff if
> it "didn't do anything interesting", and I think that attitude will need
> adjustment.  We'll have to be prepared to accept patches that are only
> interesting as waystations to some long-term goal.
>
> 			
>   

Yes, this is a sensible adjustment, even across releases, IMNSHO.

cheers

andrew

Re: Feature Freeze date for 8.4

Andrew Dunstan <andrew@dunslane.net> — 2007-10-23T22:17:51Z


Tom Lane wrote:
> Josh Berkus <josh@agliodbs.com> writes:
>   
>> Anyway, is there anyone who thinks the "cycle the queue every 6 weeks or 2 
>> months or suitable short period" is a *bad* idea?   It might be hard to pull 
>> off, but we won't know until we try.
>>     
>
> It seems worth a try --- we can certainly abandon it easily if it
> doesn't work.
>
> Personally I feel every six weeks would be too short: we'd be talking
> only a month of work between commit-fests.  I like a two-month cycle
> partly because it wouldn't rotate relative to the calendar: we'd always
> know that the first half of every odd-numbered month, or something like
> that, is commit-fest time.
>
>
>   

+1 on both counts.

cheers

andrew

Re: Feature Freeze date for 8.4

Andrew Dunstan <andrew@dunslane.net> — 2007-10-23T22:19:42Z


David Fetter wrote:
> On Tue, Oct 23, 2007 at 04:59:51PM -0400, Chris Browne wrote:
>   
>> tgl@sss.pgh.pa.us (Tom Lane) writes:
>>     
>>> Simon Riggs <simon@2ndquadrant.com> writes:
>>>       
>>>> I'd suggest we have multiple checkpoints during the cycle.
>>>> Checkpoint is a "patch queue blitz" where we stop developing and
>>>> reduce the queue to nothing. Perhaps a two-week period where
>>>> everybody helps reduce the queue, not just Tom and Bruce. Every
>>>> outstanding patch gets told what they need to do in order to get
>>>> it committed. FF is then just the last in a series of
>>>> checkpoints. Suggest we do a checkpoint every 2 months.
>>>>         
>>> I like this idea ...
>>>       
>> Don't we need to have a background writer process to gradually flush
>> this data so that checkpoints aren't so massive?  ;-)
>>     
>
> I'm proposing a DSCM with easy branching and merging to implement this
> background writer process :)
>
> The one below is already available, so we don't have to do a "flag
> day" with it.
>
> http://repo.or.cz/w/PostgreSQL.git
>
> There are git-cvsimport and git-cvsexport utilities which make
> communication between the legacy SCM and the DSCM straight-forward.
>
>
>   

You are way ahead of us here. And my vote *still* goes to Mercurial, if 
we're picking SCMs.

cheers

andrew

Re: Feature Freeze date for 8.4

Josh Berkus <josh@agliodbs.com> — 2007-10-23T22:23:19Z

Tom,

> Personally I feel every six weeks would be too short: we'd be talking
> only a month of work between commit-fests.  I like a two-month cycle
> partly because it wouldn't rotate relative to the calendar: we'd always
> know that the first half of every odd-numbered month, or something like
> that, is commit-fest time.

Well, you'd know better than me, certainly.

-- 
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco

Re: Feature Freeze date for 8.4

David Fetter <david@fetter.org> — 2007-10-23T22:47:50Z

On Tue, Oct 23, 2007 at 06:19:42PM -0400, Andrew Dunstan wrote:
> David Fetter wrote:
> >On Tue, Oct 23, 2007 at 04:59:51PM -0400, Chris Browne wrote:
> >>tgl@sss.pgh.pa.us (Tom Lane) writes:
> >>>Simon Riggs <simon@2ndquadrant.com> writes:
> >>>      
> >>>>I'd suggest we have multiple checkpoints during the cycle.
> >>>>Checkpoint is a "patch queue blitz" where we stop developing and
> >>>>reduce the queue to nothing. Perhaps a two-week period where
> >>>>everybody helps reduce the queue, not just Tom and Bruce. Every
> >>>>outstanding patch gets told what they need to do in order to get
> >>>>it committed. FF is then just the last in a series of
> >>>>checkpoints. Suggest we do a checkpoint every 2 months.
> >>>>        
> >>>I like this idea ...
> >>>      
> >>Don't we need to have a background writer process to gradually
> >>flush this data so that checkpoints aren't so massive?  ;-)
> >>    
> >
> >I'm proposing a DSCM with easy branching and merging to implement
> >this background writer process :)
> >
> >The one below is already available, so we don't have to do a "flag
> >day" with it.
> >
> >http://repo.or.cz/w/PostgreSQL.git
> >
> >There are git-cvsimport and git-cvsexport utilities which make
> >communication between the legacy SCM and the DSCM straight-forward.
> 
> You are way ahead of us here. And my vote *still* goes to Mercurial,
> if we're picking SCMs.

I'm not picking a DSCM.  I'm saying we already have tools in place for
a DSCM *without* having a "flag day."  If Mercurial has a similar
migration/legacy support path, then by all means, let's try that out,
too. :)

Cheers,
David.
-- 
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david.fetter@gmail.com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

Re: Feature Freeze date for 8.4

Josh Berkus <josh@agliodbs.com> — 2007-10-23T22:51:14Z

Folks,

> You are way ahead of us here. And my vote *still* goes to Mercurial, if
> we're picking SCMs.

Will a new SCM actually make this easier, or are people just using it as an 
excuse?

Mind you, I'm in favor of one.  A new SCM would make some other development 
tasks easier. However, I'm reluctant to open the can-of-worms which is the 
"what SCM should we use" discussion again, and complicate something which 
we seem to have consensus on.

-- 
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco

Re: Feature Freeze date for 8.4

Ron Mayer <rm_pg@cheapcomplexdevices.com> — 2007-10-23T22:55:22Z

Joshua D. Drake wrote:
> 
> We develop and commit like normal *until* the community feels there is
> enough for release. Then we announce a feature freeze.

I think you just described what will happen in reality regardless
of whatever is decided to be an official "plan".  :)  I don't
think that's necessarily a bad thing.   IMHO 8.3 is looking like
a better product because it's "late", and DBAs concerned about
stability will likely stay on 8.2 for a while anyway.

Re: Feature Freeze date for 8.4

Simon Riggs <simon@2ndquadrant.com> — 2007-10-23T23:00:36Z

On Tue, 2007-10-23 at 09:28 -0700, Joshua D. Drake wrote:
> On Tue, 23 Oct 2007 09:29:58 -0400
> Tom Lane <tgl@sss.pgh.pa.us> wrote:
> 
> > Simon Riggs <simon@2ndquadrant.com> writes:
> > > I'd suggest we have multiple checkpoints during the cycle.
> > > Checkpoint is a "patch queue blitz" where we stop developing and
> > > reduce the queue to nothing. Perhaps a two-week period where
> > > everybody helps reduce the queue, not just Tom and Bruce. Every
> > > outstanding patch gets told what they need to do in order to get it
> > > committed. FF is then just the last in a series of checkpoints.
> > > Suggest we do a checkpoint every 2 months.
> > 
> > I like this idea ...
> 
> As do I. It will also allow us to decrease the amount of changes that
> have to be reviewed for regressions during testing.

Cool. Nobody seems to have disagreed...

> I know I just love it when a customer breaks something and I ask what
> changed and it is 56 different things ;)
> 
> My question is.. with a checkpoint every 2 months, would it make it
> very easy to release every 6 (or 4 or 3 or 9) months? I am not saying
> we "have" to but it certainly opens up the possibility to the argument
> I made. 
> 
> With a 2 months checkpoint, we can release when we want. When we feel
> we have enough and not have a ginormous back log of patches to go
> through.

Maybe. I'm looking for ways to increase the amount of development time
we have compared with time releasing. If we release twice as often, we
won't get twice the beta test contribution from everybody, so our code
will be less robust, which will hurt us in the long run.

-- 
  Simon Riggs
  2ndQuadrant  http://www.2ndQuadrant.com

Re: Feature Freeze date for 8.4

Josh Berkus <josh@agliodbs.com> — 2007-10-23T23:19:22Z

Simon,

> Maybe. I'm looking for ways to increase the amount of development time
> we have compared with time releasing. If we release twice as often, we
> won't get twice the beta test contribution from everybody, so our code
> will be less robust, which will hurt us in the long run.

I don't think we actually want to release more than once a year, except 
maybe moving the annual release date a couple months one way or the other 
(December is not a great time to be releasing SW).  1 year is a good 
compromise between the developers (who would like a release every 6 
months) and the corporate users/packagers (who would like a release every 
3 years).

Plus, for the developers and other people who really need to be 
bleeding-edge, this new plan would result in less-unstable snapshots every 
2 months with defined feature sets which someone who wanted to run them at 
their own risk could.  Which would result in more bug reports, earlier, 
for us (and lots of forwarding the canned 
"milestone-releases-are-not-stable" canned e-mail).

-- 
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco

Re: Feature Freeze date for 8.4

Tom Lane <tgl@sss.pgh.pa.us> — 2007-10-23T23:24:21Z

Josh Berkus <josh@agliodbs.com> writes:
> Mind you, I'm in favor of one.  A new SCM would make some other development 
> tasks easier. However, I'm reluctant to open the can-of-worms which is the 
> "what SCM should we use" discussion again, and complicate something which 
> we seem to have consensus on.

As near as I can tell, the arguments for a new SCM mostly apply to work
which individual developers are doing outside the main tree.  So, given
the existence of stuff like git-cvsimport, I don't see a strong reason
why anyone who wants to work that way can't already sync the core CVS
with a local SCM-of-their-choice and get on with it.

You're right that this is utterly unrelated to the scheduling question,
anyway.

			regards, tom lane

Re: Feature Freeze date for 8.4

Tom Lane <tgl@sss.pgh.pa.us> — 2007-10-23T23:42:20Z

Josh Berkus <josh@agliodbs.com> writes:
> Plus, for the developers and other people who really need to be 
> bleeding-edge, this new plan would result in less-unstable snapshots every 
> 2 months with defined feature sets which someone who wanted to run them at 
> their own risk could.  Which would result in more bug reports, earlier, 
> for us (and lots of forwarding the canned 
> "milestone-releases-are-not-stable" canned e-mail).

Hmm, I was not envisioning that we'd produce any sort of "release"
corresponding to these checkpoints.  I see it only as a a way to
(a) discipline ourselves to not let patches go unreviewed/uncommitted
for long periods, and (b) encourage developers to submit relatively
small patches rather than enormous six-months-of-work ones.

Since there are always bugs, and we're certainly not going to schedule a
round of formal beta testing right after each commit-fest, I should
think that tarballs made right after a commit-fest would be particularly
unlikely to be good candidates for non-developer use.

(Actually, it might be the case that a CVS snap from just *before*
a commit-fest would be the most stable development-cycle code, since
there'd have been time to shake out bugs committed in the previous
fest... but we're even less likely to do beta testing on that.)

			regards, tom lane

Re: Feature Freeze date for 8.4

Josh Berkus <josh@agliodbs.com> — 2007-10-24T00:07:25Z

Tom,

> Since there are always bugs, and we're certainly not going to schedule a
> round of formal beta testing right after each commit-fest, I should
> think that tarballs made right after a commit-fest would be particularly
> unlikely to be good candidates for non-developer use.
>
> (Actually, it might be the case that a CVS snap from just *before*
> a commit-fest would be the most stable development-cycle code, since
> there'd have been time to shake out bugs committed in the previous
> fest... but we're even less likely to do beta testing on that.)

Hmmm.  Well, what I'm looking for is a milestone build we can grab for 
testing purposes.  Right now we grab the daily snapshots, but there's no 
archive of those, so we're often fighting the fact that a patch developer 
did his patch on a different snapshot than you have and there's no going 
back, so you have to share not just the patch but the entire PG tree it's 
against.

If we had milestone snapshots of each 2 months ... probably just before 
each commit-fest ... and kept them available on ftp.postgresql.org until 
official beta, then it would make it easier for testers to have a common 
point of reference to work with.

-- 
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco

Re: Feature Freeze date for 8.4

Tom Lane <tgl@sss.pgh.pa.us> — 2007-10-24T00:25:43Z

Josh Berkus <josh@agliodbs.com> writes:
> If we had milestone snapshots of each 2 months ... probably just before 
> each commit-fest ... and kept them available on ftp.postgresql.org until 
> official beta, then it would make it easier for testers to have a common 
> point of reference to work with.

This seems pretty entirely orthogonal to the commit-fest proposal.
I see no reason to think that snapshots taken at those times would
be any better than any other nightly snapshot, nor any reason
to memorialize them in an archive.

Now to the extent that regular commit-fests keep patch development
more closely aligned with the mainline CVS, the fest proposal might
indirectly alleviate your pain.  But I'd think that snaps taken
*after* the fests would be the best for that, as they'd be closer
to what any subsequent development would be based on.  The stability
argument I made doesn't seem like a problem for testing purposes,
only if someone were to try to use such a snap for production purposes
(which putting them up for archiving might encourage).

			regards, tom lane

Re: Feature Freeze date for 8.4

Andrew Dunstan <andrew@dunslane.net> — 2007-10-24T01:34:04Z


Tom Lane wrote:
> Now to the extent that regular commit-fests keep patch development
> more closely aligned with the mainline CVS, the fest proposal might
> indirectly alleviate your pain.  But I'd think that snaps taken
> *after* the fests would be the best for that, as they'd be closer
> to what any subsequent development would be based on.  The stability
> argument I made doesn't seem like a problem for testing purposes,
> only if someone were to try to use such a snap for production purposes
> (which putting them up for archiving might encourage).
>
> 			
>   

It might be worth applying a simple tag (but not a branch) at the end 
(and maybe also at the start) of each checkpoint/fest/whatever

cheers

andrew

Re: Feature Freeze date for 8.4

Tom Lane <tgl@sss.pgh.pa.us> — 2007-10-24T01:55:40Z

Andrew Dunstan <andrew@dunslane.net> writes:
> It might be worth applying a simple tag (but not a branch) at the end 
> (and maybe also at the start) of each checkpoint/fest/whatever

Perhaps, though of course one could easily enough pull a CVS snapshot by
date instead (especially if we stick to a pretty predictable schedule
for the fests).

Thinking about that a bit more, it seems like a rigid "two weeks" plan
is pointless, since the amount of work to be done will vary depending
on what's in the queue.  It seems what we ought to do is something like
this:

* Commit-fest starts on the first of each alternate month.  It ends
whenever all the patches that were in the queue on the first are dealt
with; either committed, rejected permanently, or sent back for specific
rework.  It might take a week, or two, or three, but in any case we all
try to focus on patch review rather than new work until it's done.

If we do it that way, then an end-of-fest tag might be worthwhile,
so you'd not have to dig through the mailing list archives to figure out
when a fest ended.

			regards, tom lane

Re: Feature Freeze date for 8.4

Greg Smith <gsmith@gregsmith.com> — 2007-10-24T02:56:08Z

On Tue, 23 Oct 2007, David Fetter wrote:

> If Mercurial has a similar migration/legacy support path, then by all 
> means, let's try that out, too.

There's an import tool at http://hg.beekhof.net/hg/cvs-import but the 
experience of the Mozilla project suggests it's on the buggy and slow side 
for large repositories with lots of history: 
http://weblogs.mozillazine.org/preed/2007/04/version_control_system_shootou_1.html

The main problem with using Git on a much larger scale is that there's 
still limited ability to use it on Win32.  Google is working on that: 
http://code.google.com/p/msysgit/ but it's not quite there yet; there's 
also a partial MinGW port.

The way I suspect any plan for incorporating a new [D]VCS on a larger 
scale in this project will need to progress is like this:

1) Make a converted copy of the existing CVS repository
2) Keep the mirrored repo up to date with new commits
3) Provide working guidelines so that developers can use the new VCS to 
build local patches and improve their productivity
4) Get enough developers using the new system that it becomes a popular 
way to increase visibility on work in progress patches
5) Reach a critical mass of developers showing improved productivity on 
the new system to sway the core toward that particular VCS
6) Convert the main repository to the new VCS

Git has reached (2) already.  I started documenting a process for using 
Subversion, it would only take a little more work on 
http://developer.postgresql.org/index.php/Working_with_CVS and that would 
hit (3).  I personally don't think there's enough potential for gain 
converting to SVN to make it worth the trouble, which is why I haven't 
bothered doing more there.

Mercurial/Monotone/Bazaar are all interesting possibilities as well, but 
it will take an advocate willing to start down this trail for a particular 
tool to kick off a serious investigation of any of them.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD

Re: Feature Freeze date for 8.4

Brendan Jurd <direvus@gmail.com> — 2007-10-24T05:37:28Z

On 10/24/07, Greg Smith <gsmith@gregsmith.com> wrote:
> 1) Make a converted copy of the existing CVS repository
> 2) Keep the mirrored repo up to date with new commits
> 3) Provide working guidelines so that developers can use the new VCS to
> build local patches and improve their productivity
> 4) Get enough developers using the new system that it becomes a popular
> way to increase visibility on work in progress patches
> 5) Reach a critical mass of developers showing improved productivity on
> the new system to sway the core toward that particular VCS
> 6) Convert the main repository to the new VCS
>
> Git has reached (2) already.  I started documenting a process for using
> Subversion, it would only take a little more work on
> http://developer.postgresql.org/index.php/Working_with_CVS and that would
> hit (3).  I personally don't think there's enough potential for gain
> converting to SVN to make it worth the trouble, which is why I haven't
> bothered doing more there.

How up to date is the Git repos?  Does it pull individual commits from
CVS, or does it resync the whole history periodically?  If so, what's
the lag?

An important part of (2) is that the mirrored repos be sufficiently up
to date that using it for your dev work and producing patches doesn't
put you significantly behind HEAD.

I think the Subversion repos resyncs every six hours, but I'm not sure
(strangely there's not a whole lot of information about it on the
wiki).  To me, six hours seems a little slow.

If you hit (3) on Git I for one will gladly start using the Git
repository for my dev work.  I work on minor upgrades, not the major
stuff, so I think the benefits of a distributed system might be slim
for me (because I'm not going to be switching across large branches).
But I'd be happy to give it a shot.

Re: Feature Freeze date for 8.4

Simon Riggs <simon@2ndquadrant.com> — 2007-10-24T06:57:15Z

On Tue, 2007-10-23 at 16:19 -0700, Josh Berkus wrote:

> > Maybe. I'm looking for ways to increase the amount of development time
> > we have compared with time releasing. If we release twice as often, we
> > won't get twice the beta test contribution from everybody, so our code
> > will be less robust, which will hurt us in the long run.
> 
> I don't think we actually want to release more than once a year

I don't want to either!

Not sure how you've taken my words backwards, but apologies for not
being clear enough. I was trying to say that its possible ("maybe"),
*but* that there were arguments against it, so not desirable since it
will hurt us in the long run. I agree with all of your additional
arguments as to why not also.

-- 
  Simon Riggs
  2ndQuadrant  http://www.2ndQuadrant.com

Re: Feature Freeze date for 8.4

David Fetter <david@fetter.org> — 2007-10-24T06:57:28Z

On Tue, Oct 23, 2007 at 07:24:21PM -0400, Tom Lane wrote:
> Josh Berkus <josh@agliodbs.com> writes:
> > Mind you, I'm in favor of one.  A new SCM would make some other
> > development tasks easier. However, I'm reluctant to open the
> > can-of-worms which is the "what SCM should we use" discussion
> > again, and complicate something which we seem to have consensus
> > on.

We don't need to, as the tool is already in place.

> As near as I can tell, the arguments for a new SCM mostly apply to
> work which individual developers are doing outside the main tree.
> So, given the existence of stuff like git-cvsimport, I don't see a
> strong reason why anyone who wants to work that way can't already
> sync the core CVS with a local SCM-of-their-choice and get on with
> it.
> 
> You're right that this is utterly unrelated to the scheduling
> question, anyway.

It's not even slightly unrelated.  It's a way of preventing bit-rot on
large patches and keeping them in sync :)

Cheers,
David.
-- 
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david.fetter@gmail.com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

Re: Feature Freeze date for 8.4

Simon Riggs <simon@2ndquadrant.com> — 2007-10-24T06:59:32Z

On Tue, 2007-10-23 at 19:42 -0400, Tom Lane wrote:
> Josh Berkus <josh@agliodbs.com> writes:
> > Plus, for the developers and other people who really need to be 
> > bleeding-edge, this new plan would result in less-unstable snapshots every 
> > 2 months with defined feature sets which someone who wanted to run them at 
> > their own risk could.  Which would result in more bug reports, earlier, 
> > for us (and lots of forwarding the canned 
> > "milestone-releases-are-not-stable" canned e-mail).
> 
> Hmm, I was not envisioning that we'd produce any sort of "release"
> corresponding to these checkpoints.  I see it only as a a way to
> (a) discipline ourselves to not let patches go unreviewed/uncommitted
> for long periods, and (b) encourage developers to submit relatively
> small patches rather than enormous six-months-of-work ones.
> 
> Since there are always bugs, and we're certainly not going to schedule a
> round of formal beta testing right after each commit-fest, I should
> think that tarballs made right after a commit-fest would be particularly
> unlikely to be good candidates for non-developer use.
> 
> (Actually, it might be the case that a CVS snap from just *before*
> a commit-fest would be the most stable development-cycle code, since
> there'd have been time to shake out bugs committed in the previous
> fest... but we're even less likely to do beta testing on that.)

+1

-- 
  Simon Riggs
  2ndQuadrant  http://www.2ndQuadrant.com

Re: Feature Freeze date for 8.4

Marko Kreen <markokr@gmail.com> — 2007-10-24T07:55:24Z

On 10/24/07, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Josh Berkus <josh@agliodbs.com> writes:
> > Mind you, I'm in favor of one.  A new SCM would make some other development
> > tasks easier. However, I'm reluctant to open the can-of-worms which is the
> > "what SCM should we use" discussion again, and complicate something which
> > we seem to have consensus on.
>
> As near as I can tell, the arguments for a new SCM mostly apply to work
> which individual developers are doing outside the main tree.  So, given
> the existence of stuff like git-cvsimport, I don't see a strong reason
> why anyone who wants to work that way can't already sync the core CVS
> with a local SCM-of-their-choice and get on with it.

Yes, external devs can already work fine with DSCM.

And after using the repo.or.cz git tree for some time, mostly
tracking and also for some small patches I really have no interest
touching CVS anymore.

> You're right that this is utterly unrelated to the scheduling question,
> anyway.

As we seem discussing developement in general, there is one
obstacle in the way of individual use of DSCMs - context diff
format as only one accepted.  Both leading DSCMs - GIT and Mercurial
do not support it.

Currently converting patch from DSCM to context diff is pain,
especially if it contains new files.  It would make lives of DSCM
users much easier if both formats would be accepted in -hackers.

-- 
marko

Re: Feature Freeze date for 8.4

Alvaro Herrera <alvherre@commandprompt.com> — 2007-10-24T11:33:27Z

David Fetter wrote:

> I'm not picking a DSCM.  I'm saying we already have tools in place for
> a DSCM *without* having a "flag day."  If Mercurial has a similar
> migration/legacy support path, then by all means, let's try that out,
> too. :)

There's at least on Mercurial repo, here:

http://www.ubiobio.cl/~gpoo/pgsql/pgsql/

I'm not sure how often it syncs.

For those of you outside Chile my bet is that it will be a bit slow ...

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Re: Feature Freeze date for 8.4

Alvaro Herrera <alvherre@commandprompt.com> — 2007-10-24T11:34:44Z

Marko Kreen escribió:

> As we seem discussing developement in general, there is one
> obstacle in the way of individual use of DSCMs - context diff
> format as only one accepted.  Both leading DSCMs - GIT and Mercurial
> do not support it.

Hmm, in Subversion you can specify a separate diff command and args,
which can be used to generate context diffs.  Is it not possible with
git/Hg?

-- 
Alvaro Herrera                 http://www.amazon.com/gp/registry/DXLWNGRJD34J
"La realidad se compone de muchos sueños, todos ellos diferentes,
pero en cierto aspecto, parecidos..." (Yo, hablando de sueños eróticos)

Re: Feature Freeze date for 8.4

Brendan Jurd <direvus@gmail.com> — 2007-10-24T12:00:20Z

> As we seem discussing developement in general, there is one
> obstacle in the way of individual use of DSCMs - context diff
> format as only one accepted.  Both leading DSCMs - GIT and Mercurial
> do not support it.
>

Really?  I just started playing around with git, and the output from
git diff produced the same kind of diff file I would normally get from
`svn di` or `cvs di -c`.

Maybe I'm misunderstanding what you meant by "context diff", but this
stuff git is producing looks like a context diff to me.

Regards,
BJ

Re: Feature Freeze date for 8.4

Alvaro Herrera <alvherre@commandprompt.com> — 2007-10-24T12:09:44Z

Brendan Jurd escribió:
> > As we seem discussing developement in general, there is one
> > obstacle in the way of individual use of DSCMs - context diff
> > format as only one accepted.  Both leading DSCMs - GIT and Mercurial
> > do not support it.
> >
> 
> Really?  I just started playing around with git, and the output from
> git diff produced the same kind of diff file I would normally get from
> `svn di` 

... which is a unified diff.

> or `cvs di -c`.

Huh, strange.  That's a context diff.

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Re: Feature Freeze date for 8.4

Michael Paesold <mpaesold@gmx.at> — 2007-10-24T12:17:08Z

Alvaro Herrera write:
> Marko Kreen escribió:
> 
>> As we seem discussing developement in general, there is one
>> obstacle in the way of individual use of DSCMs - context diff
>> format as only one accepted.  Both leading DSCMs - GIT and Mercurial
>> do not support it.
> 
> Hmm, in Subversion you can specify a separate diff command and args,
> which can be used to generate context diffs.  Is it not possible with
> git/Hg?

It's possible in Mercurial, too. There is a bundled extension (extdiff, 
http://www.selenic.com/mercurial/wiki/index.cgi/ExtdiffExtension), which 
can do that. You can configure your own command aliases, e.g. "hg cdiff 
..." to do -c diffs.

Best Regards
Michael Paesold

Re: Feature Freeze date for 8.4

Brendan Jurd <direvus@gmail.com> — 2007-10-24T12:20:43Z

On 10/24/07, Alvaro Herrera <alvherre@commandprompt.com> wrote:
> Brendan Jurd escribió:
> > Really?  I just started playing around with git, and the output from
> > git diff produced the same kind of diff file I would normally get from
> > `svn di`
>
> ... which is a unified diff.
>
> > or `cvs di -c`.
>
> Huh, strange.  That's a context diff.

Right.  I was confusing unified and context, because the unified diffs
I'm looking at do actually show "context" lines above and below each
change.  Sorry for the noise.

Re: Feature Freeze date for 8.4

Aidan Van Dyk <aidan@highrise.ca> — 2007-10-24T12:24:39Z

* Brendan Jurd <direvus@gmail.com> [071024 01:41]:
 
> How up to date is the Git repos?  Does it pull individual commits from
> CVS, or does it resync the whole history periodically?  If so, what's
> the lag?

It's updated hourly, which is the same rate the public CVS is updated.

> An important part of (2) is that the mirrored repos be sufficiently up
> to date that using it for your dev work and producing patches doesn't
> put you significantly behind HEAD.
> 
> I think the Subversion repos resyncs every six hours, but I'm not sure
> (strangely there's not a whole lot of information about it on the
> wiki).  To me, six hours seems a little slow.
> 
> If you hit (3) on Git I for one will gladly start using the Git
> repository for my dev work.  I work on minor upgrades, not the major
> stuff, so I think the benefits of a distributed system might be slim
> for me (because I'm not going to be switching across large branches).
> But I'd be happy to give it a shot.

Please - feel free to use the Git repository for your work.  It's there,
and staying there, based on the PostgreSQL CVS repository.

In other projects I'm involved in, some developers are using GIT pretty
heavily *as developers*.  The projects "main repository" is still in
CVS, but that isn't much of a problem, because the tools git offers are
excellent at producing patches, patch series, rebasing them, sharing
them, and even directly committing them to CVS.  The advantages of GIT
aren't necessarily that it's a *distributed* SCM/VCS, but that since it
is a DSCM/DVCS, it's understanding of history and the tools that enable
are extremely powerful.

But CVS is (unfortunately) the defacto standard.  Everybody understands
it (well, the basics of it), and all the tools know how to interact with
it.  So leave the main repository in CVS, and let each developer use the
tools they are most comfortable with.  The fact is, no matter *what* the
"official" repository is, I would *still* use git to develop.  And I'm
guessing that others who prefer the tools in darcs/monotone/hg will
continue to use their favourite tools too.

But the more arcane, unique, or just plain "different" the main
repository is, the higher *perceived* barrier to entry will be.

a.

-- 
Aidan Van Dyk                                             Create like a god,
aidan@highrise.ca                                       command like a king,
http://www.highrise.ca/                                   work like a slave.

Re: Feature Freeze date for 8.4

Germán Poó-Caamaño <gpoo@ubiobio.cl> — 2007-10-24T12:26:18Z

On Wed, 2007-10-24 at 08:33 -0300, Alvaro Herrera wrote:
> David Fetter wrote:
> 
> > I'm not picking a DSCM.  I'm saying we already have tools in place for
> > a DSCM *without* having a "flag day."  If Mercurial has a similar
> > migration/legacy support path, then by all means, let's try that out,
> > too. :)
> 
> There's at least on Mercurial repo, here:
> 
> http://www.ubiobio.cl/~gpoo/pgsql/pgsql/
> 
> I'm not sure how often it syncs.

Once an hour.

In short, the way it works is:
* Having a CVS copy through rsync.
* Any CVS change is added incrementally.  You can stop the process and
  the next time will continue from that point.

> For those of you outside Chile my bet is that it will be a bit slow ...

Through Internet2 should less slower than Internet, but I guess still
slow.

But it will be slow the first time you check out the repository.
Next time you only will get differences, and Mercurial is network
friendly, AFAIU.

-- 
Germán Poó Caamaño
Concepción - Chile

Re: Feature Freeze date for 8.4

Martijn van Oosterhout <kleptog@svana.org> — 2007-10-24T12:32:13Z

On Tue, Oct 23, 2007 at 02:39:43PM -0700, David Fetter wrote:
> The one below is already available, so we don't have to do a "flag
> day" with it.
> 
> http://repo.or.cz/w/PostgreSQL.git

As someone who hasn't used GIT: if I have a modified CVS tree from some time
back (>1 year) can I use this to manage the bitrot? Just doing a CVS
update will probably just mark everything conflicted, so I'm wondering
is GIT can do it better.

Have a nice day,
-- 
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

Re: Feature Freeze date for 8.4

Tom Lane <tgl@sss.pgh.pa.us> — 2007-10-24T12:45:14Z

Aidan Van Dyk <aidan@highrise.ca> writes:
> * Brendan Jurd <direvus@gmail.com> [071024 01:41]:
>> How up to date is the Git repos?  Does it pull individual commits from
>> CVS, or does it resync the whole history periodically?  If so, what's
>> the lag?

> It's updated hourly, which is the same rate the public CVS is updated.

FWIW, that's no longer the case: there's some sort of trigger mechanism
now that causes the anoncvs mirror to update within a few minutes of a
commit to the master.  So it might be worth the trouble to sync more
often.

			regards, tom lane

Re: Feature Freeze date for 8.4

Brian Hurt <bhurt@janestcapital.com> — 2007-10-24T12:57:42Z

Josh Berkus wrote:

>Folks,
>
>  
>
>>You are way ahead of us here. And my vote *still* goes to Mercurial, if
>>we're picking SCMs.
>>    
>>
>
>Will a new SCM actually make this easier, or are people just using it as an 
>excuse?
>
>  
>
We use mercurial here at work, having switched to it recently, and while 
I don't claim to be an expert, it does seem nice.  For example, you can 
have a local repository you're checking code into, and can pull from and 
merge up with some shared repository.  Also, you can pull from one 
repository and check into another- so, for example, we have a staging 
repository and a compiles repository (unless you welcome the pain)- you 
pull from the compiles repository, but push changes back to the staging 
repository.  Then we have a script that pulls recent changes from the 
staging repository, make sure they compile and the unit tests run, 
before moving them over to the compiles repository.  This way, the 
version you're pulling at least compiles and passes some minimal unit tests.

A similiar process could work for postgres- except instead of "staging" 
and "compiles" you'd have a "sumbitted" and "accepted" repositories.  
And instead of a compile daemon, it'd be reviewers who would move code 
from one to the other. 

Note that everything I'm talking about here is not unique to Mercurial- 
you can do this just about as easily in darcs or git (I'd advise against 
Bazaar/bzr)- so don't take this as being pro-Mercurial, just pro-SCM.

Brian

Re: Feature Freeze date for 8.4

Gregory Stark <stark@enterprisedb.com> — 2007-10-24T13:12:16Z

"Martijn van Oosterhout" <kleptog@svana.org> writes:

> On Tue, Oct 23, 2007 at 02:39:43PM -0700, David Fetter wrote:
>> The one below is already available, so we don't have to do a "flag
>> day" with it.
>> 
>> http://repo.or.cz/w/PostgreSQL.git
>
> As someone who hasn't used GIT: if I have a modified CVS tree from some time
> back (>1 year) can I use this to manage the bitrot? Just doing a CVS
> update will probably just mark everything conflicted, so I'm wondering
> is GIT can do it better.

Well it could except not here where the git repository is being fed from CVS.
CVS is just not gathering a lot of the information that helps git and the
other distributed systems do their job. Namely it has no idea which changes
were committed together as part of the same patch, and it has no idea where
those changes came from and whether they're also in your tree.

-- 
  Gregory Stark
  EnterpriseDB          http://www.enterprisedb.com

Re: Feature Freeze date for 8.4

Germán Poó-Caamaño <gpoo@ubiobio.cl> — 2007-10-24T13:14:45Z

On Tue, 2007-10-23 at 19:24 -0400, Tom Lane wrote:
> Josh Berkus <josh@agliodbs.com> writes:
> > Mind you, I'm in favor of one.  A new SCM would make some other development 
> > tasks easier. However, I'm reluctant to open the can-of-worms which is the 
> > "what SCM should we use" discussion again, and complicate something which 
> > we seem to have consensus on.
> 
> As near as I can tell, the arguments for a new SCM mostly apply to work
> which individual developers are doing outside the main tree.  So, given
> the existence of stuff like git-cvsimport, I don't see a strong reason
> why anyone who wants to work that way can't already sync the core CVS
> with a local SCM-of-their-choice and get on with it.

According the proposal of reduce/avoid working with big patches, it
would help.

While it's even possible to handle separately a SCM and then send a big
patch, it's a pain merging it using CVS. And finally the history of that
patch is lost.

The workflow shouldn't change at all, but the tools can help to
improve the process of reviewing/merging patches.  If that process
may improve, then a 'fixed' release schedule is even more possible, 
IMVVHO.

-- 
Germán Poó Caamaño
Concepción - Chile

Re: Feature Freeze date for 8.4

Tom Lane <tgl@sss.pgh.pa.us> — 2007-10-24T13:24:47Z

"Marko Kreen" <markokr@gmail.com> writes:
> As we seem discussing developement in general, there is one
> obstacle in the way of individual use of DSCMs - context diff
> format as only one accepted.

Well, that's not a hard-and-fast rule, just a preference.  At least for
me, unidiff is vastly harder to read than cdiff for anything much beyond
one-line changes.  (For one-liners it's great ;-), but beyond that it
intermixes old and new lines too freely.)  That's not merely an
impediment to quick review of the patch; if there's any manual
patch-merging to be done, it significantly increases the risk of error.

I don't recall that we've rejected any patches lately just because they
were unidiffs.  But I'd be sad if a large fraction of incoming patches
started to be unidiffs.

			regards, tom lane

Re: Feature Freeze date for 8.4

Magnus Hagander <magnus@hagander.net> — 2007-10-24T13:34:25Z

On Wed, Oct 24, 2007 at 09:24:47AM -0400, Tom Lane wrote:
> I don't recall that we've rejected any patches lately just because they
> were unidiffs.  But I'd be sad if a large fraction of incoming patches
> started to be unidiffs.

We bounce them back to the author pretty m uch every time with "context
diff please". But I don't recall any case where the patch was actually
rejected because of a submitter refusing to provide -c diff.

//Magnus

Re: Feature Freeze date for 8.4

Gregory Stark <stark@enterprisedb.com> — 2007-10-24T13:35:41Z

"Tom Lane" <tgl@sss.pgh.pa.us> writes:

> "Marko Kreen" <markokr@gmail.com> writes:
>> As we seem discussing developement in general, there is one
>> obstacle in the way of individual use of DSCMs - context diff
>> format as only one accepted.
>
> Well, that's not a hard-and-fast rule, just a preference.  At least for
> me, unidiff is vastly harder to read than cdiff for anything much beyond
> one-line changes.  (For one-liners it's great ;-), but beyond that it
> intermixes old and new lines too freely.)  That's not merely an
> impediment to quick review of the patch; if there's any manual
> patch-merging to be done, it significantly increases the risk of error.
>
> I don't recall that we've rejected any patches lately just because they
> were unidiffs.  But I'd be sad if a large fraction of incoming patches
> started to be unidiffs.

It seems hard to believe this would be a hard problem to overcome. It's not
like either format contains more or less information than the other. In fact
Emacs's diff-mode can convert between them on the fly.

-- 
  Gregory Stark
  EnterpriseDB          http://www.enterprisedb.com

Re: Feature Freeze date for 8.4

Germán Poó-Caamaño <gpoo@ubiobio.cl> — 2007-10-24T13:36:15Z

On Wed, 2007-10-24 at 14:32 +0200, Martijn van Oosterhout wrote:
> On Tue, Oct 23, 2007 at 02:39:43PM -0700, David Fetter wrote:
> > The one below is already available, so we don't have to do a "flag
> > day" with it.
> > 
> > http://repo.or.cz/w/PostgreSQL.git
> 
> As someone who hasn't used GIT: if I have a modified CVS tree from some time
> back (>1 year) can I use this to manage the bitrot? Just doing a CVS
> update will probably just mark everything conflicted, so I'm wondering
> is GIT can do it better.

The way you can work could be (it's the same if you are using
git/hg/bzr):

* Have a copy (clone) of the main repo, which is a pristine copy of
  CVS. call it 'master'.
* Clone or create a branch from the 'master'.  This is the working
  copy.

Master can be updated regularly.  And also, you can merge the changes
in your branch or even, only look the changes without touching 
anything.

What can be merge automatically, it will.  Otherwise, a external
tool is called (such as meld or whatever you use).

Evenmore, you can try merging with previous cvs' commits to avoid
the pain.

-- 
Germán Poó Caamaño
Concepción - Chile

Re: Feature Freeze date for 8.4

Marko Kreen <markokr@gmail.com> — 2007-10-24T13:38:14Z

On 10/24/07, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> "Marko Kreen" <markokr@gmail.com> writes:
> > As we seem discussing developement in general, there is one
> > obstacle in the way of individual use of DSCMs - context diff
> > format as only one accepted.
>
> Well, that's not a hard-and-fast rule, just a preference.  At least for
> me, unidiff is vastly harder to read than cdiff for anything much beyond
> one-line changes.  (For one-liners it's great ;-), but beyond that it
> intermixes old and new lines too freely.)  That's not merely an
> impediment to quick review of the patch; if there's any manual
> patch-merging to be done, it significantly increases the risk of error.
>
> I don't recall that we've rejected any patches lately just because they
> were unidiffs.  But I'd be sad if a large fraction of incoming patches
> started to be unidiffs.

Thanks, maybe the DEVFAQ can be changed that both -u and -c are
accepted but -c is preferred.

The matter of -c vs. -u is mostly a matter of taste and habit but
there is also a technical argument - you can always clean up
hard-to-read unidiff with simple /^-/d.  But there is no simple
way to make hard-to-read context diff readable.

-- 
marko

Re: Feature Freeze date for 8.4

Andrew Dunstan <andrew@dunslane.net> — 2007-10-24T13:58:40Z


Marko Kreen wrote:
> On 10/24/07, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>   
>> "Marko Kreen" <markokr@gmail.com> writes:
>>     
>>> As we seem discussing developement in general, there is one
>>> obstacle in the way of individual use of DSCMs - context diff
>>> format as only one accepted.
>>>       
>> Well, that's not a hard-and-fast rule, just a preference.  At least for
>> me, unidiff is vastly harder to read than cdiff for anything much beyond
>> one-line changes.  (For one-liners it's great ;-), but beyond that it
>> intermixes old and new lines too freely.)  That's not merely an
>> impediment to quick review of the patch; if there's any manual
>> patch-merging to be done, it significantly increases the risk of error.
>>
>> I don't recall that we've rejected any patches lately just because they
>> were unidiffs.  But I'd be sad if a large fraction of incoming patches
>> started to be unidiffs.
>>     
>
> Thanks, maybe the DEVFAQ can be changed that both -u and -c are
> accepted but -c is preferred.
>
> The matter of -c vs. -u is mostly a matter of taste and habit but
> there is also a technical argument - you can always clean up
> hard-to-read unidiff with simple /^-/d.  But there is no simple
> way to make hard-to-read context diff readable.
>
>   

I would rather stick generally to one style. It's a question of whose 
convenience prevails, the author's or the reviewer's. I think it should 
be the reviewer's, and since Tom reviews far more than anyone else his 
voice accordingly matters most.

cheers

andrew

Re: Feature Freeze date for 8.4

Tom Lane <tgl@sss.pgh.pa.us> — 2007-10-24T13:59:31Z

Magnus Hagander <magnus@hagander.net> writes:
> On Wed, Oct 24, 2007 at 09:24:47AM -0400, Tom Lane wrote:
>> I don't recall that we've rejected any patches lately just because they
>> were unidiffs.  But I'd be sad if a large fraction of incoming patches
>> started to be unidiffs.

> We bounce them back to the author pretty m uch every time with "context
> diff please".

We have, and will continue to, bounce patches that arrive as whole files
or no-context-lines patches.  But I know we've taken unidiffs without
complaint.  Personally, if I have to read one that's more than isolated
one-line changes, I apply it locally and then use "cvs diff -c" to get a
version I can read ... which makes unidiff only a minor annoyance *as
long as it applies cleanly*.  If it doesn't then it's a PITA ...

			regards, tom lane

Re: Feature Freeze date for 8.4

Heikki Linnakangas <heikki@enterprisedb.com> — 2007-10-24T14:34:40Z

Tom Lane wrote:
> Magnus Hagander <magnus@hagander.net> writes:
>> On Wed, Oct 24, 2007 at 09:24:47AM -0400, Tom Lane wrote:
>>> I don't recall that we've rejected any patches lately just because they
>>> were unidiffs.  But I'd be sad if a large fraction of incoming patches
>>> started to be unidiffs.
> 
>> We bounce them back to the author pretty m uch every time with "context
>> diff please".
> 
> We have, and will continue to, bounce patches that arrive as whole files
> or no-context-lines patches.  But I know we've taken unidiffs without
> complaint.  Personally, if I have to read one that's more than isolated
> one-line changes, I apply it locally and then use "cvs diff -c" to get a
> version I can read ... which makes unidiff only a minor annoyance *as
> long as it applies cleanly*.  If it doesn't then it's a PITA ...

You can use "filterdiff -v --format=context". Or emacs's built in
command to do the conversion.

Because it's easy to convert from one to another, I think the unified
vs. context diff issue is a non-issue.

-- 
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

Re: Feature Freeze date for 8.4

tomas@tuxteam.de — 2007-10-24T15:50:01Z

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, Oct 24, 2007 at 02:32:13PM +0200, Martijn van Oosterhout wrote:
> On Tue, Oct 23, 2007 at 02:39:43PM -0700, David Fetter wrote:
> > The one below is already available, so we don't have to do a "flag
> > day" with it.
> > 
> > http://repo.or.cz/w/PostgreSQL.git
> 
> As someone who hasn't used GIT: if I have a modified CVS tree from some time
> back (>1 year) can I use this to manage the bitrot? Just doing a CVS
> update will probably just mark everything conflicted, so I'm wondering
> is GIT can do it better.

It's no magic, but three-way merges[1] and automatic tracking of file
renames do help quite a bit.

- ------
[1] most distributed SCMs keep enough info around to dig out the last
common ancestor of the two files to be merged and do then a three-way
merge, which is better at resolving conflicts than a naïve direct merge.

Regards
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFHH2mpBcgs9XrR2kYRAg/uAJoCSjQ3GDIEZidWYG9IX5J7RF4N9QCfXp+i
614RlRkN38tPTBhZ9ctir0I=
=AsX7
-----END PGP SIGNATURE-----

Re: Feature Freeze date for 8.4

Tom Lane <tgl@sss.pgh.pa.us> — 2007-10-24T15:57:40Z

Heikki Linnakangas <heikki@enterprisedb.com> writes:
> You can use "filterdiff -v --format=context".

Cool, I'll have to get a copy of that.

> Because it's easy to convert from one to another, I think the unified
> vs. context diff issue is a non-issue.

Fair enough then; we should just change the policy.

			regards, tom lane

Re: Feature Freeze date for 8.4

Bruce Momjian <bruce@momjian.us> — 2007-10-24T16:26:31Z

Marko Kreen wrote:
> On 10/24/07, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > "Marko Kreen" <markokr@gmail.com> writes:
> > > As we seem discussing developement in general, there is one
> > > obstacle in the way of individual use of DSCMs - context diff
> > > format as only one accepted.
> >
> > Well, that's not a hard-and-fast rule, just a preference.  At least for
> > me, unidiff is vastly harder to read than cdiff for anything much beyond
> > one-line changes.  (For one-liners it's great ;-), but beyond that it
> > intermixes old and new lines too freely.)  That's not merely an
> > impediment to quick review of the patch; if there's any manual
> > patch-merging to be done, it significantly increases the risk of error.
> >
> > I don't recall that we've rejected any patches lately just because they
> > were unidiffs.  But I'd be sad if a large fraction of incoming patches
> > started to be unidiffs.
> 
> Thanks, maybe the DEVFAQ can be changed that both -u and -c are
> accepted but -c is preferred.
> 
> The matter of -c vs. -u is mostly a matter of taste and habit but
> there is also a technical argument - you can always clean up
> hard-to-read unidiff with simple /^-/d.  But there is no simple
> way to make hard-to-read context diff readable.

Context diff shows you the old block and new block in its entirety. 
Unidiff does not, hence the context diff preference.

-- 
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

Re: Feature Freeze date for 8.4

Bruce Momjian <bruce@momjian.us> — 2007-10-24T16:29:53Z

Tom Lane wrote:
> Andrew Dunstan <andrew@dunslane.net> writes:
> > I'm fairly resistant to putting less-than-ready code in the tree, I must 
> > say.
> 
> Me too, at least if "less than ready" means "unstable".  The committed
> code has to always be solid enough to let everybody continue working on
> their own bits.  However, in the past we've tended to refuse stuff if
> it "didn't do anything interesting", and I think that attitude will need
> adjustment.  We'll have to be prepared to accept patches that are only
> interesting as waystations to some long-term goal.

I know for the Win32 port I committed things in parts, e.g. fix absolute
path detection.  It would have been impossible to submit one large patch
to do the entire Win32 port, and in fact the feature spread over two
major releases.  I think if someone took on a similar-sized project we
would just have to give them CVS commit access.

-- 
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

Re: Feature Freeze date for 8.4

Bruce Momjian <bruce@momjian.us> — 2007-10-24T16:35:00Z

Tom Lane wrote:
> Josh Berkus <josh@agliodbs.com> writes:
> > Anyway, is there anyone who thinks the "cycle the queue every 6 weeks or 2 
> > months or suitable short period" is a *bad* idea?   It might be hard to pull 
> > off, but we won't know until we try.
> 
> It seems worth a try --- we can certainly abandon it easily if it
> doesn't work.
> 
> Personally I feel every six weeks would be too short: we'd be talking
> only a month of work between commit-fests.  I like a two-month cycle
> partly because it wouldn't rotate relative to the calendar: we'd always
> know that the first half of every odd-numbered month, or something like
> that, is commit-fest time.

Sounds fine to me.  Basically it is a "reviewers, get your heads out of
your monitors and help other patch submitters".  Not a lot of additional
fun for reviewers, of course, but probably necessary.

-- 
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

Re: Feature Freeze date for 8.4

Bruce Momjian <bruce@momjian.us> — 2007-10-24T16:36:54Z

Tom Lane wrote:
> [ chewing on this a bit... ]  The curious thing about that is that
> despite this being designed to be a short release cycle, we ended up
> landing a bunch of major patches that weren't on the radar screen at
> all at the start of the cycle.  This suggests to me that there's
> something wrong with the concept that no one can get anything major done
> without a long release cycle to do it in.
> 
> Indeed, the thing that seemed to me to be killing us in this freeze
> cycle was that the patches coming in were already at the upper limit of
> what we can review and digest.

Yep.  Hopefully with the patches applied in 8.3 we can have more
confidence in several patch submitters so there is less reviewing
required, and these new people can help review too.

> I'd rather encourage people to work in an incremental, not-so-big-bang
> fashion.  Obviously one of the requirements for that will be quicker
> review turnaround and commit, so that there's time to build on a
> previous patch...

Sounds good.

-- 
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

Re: Feature Freeze date for 8.4

Gregory Stark <stark@enterprisedb.com> — 2007-10-24T16:53:37Z

"Tom Lane" <tgl@sss.pgh.pa.us> writes:

> Heikki Linnakangas <heikki@enterprisedb.com> writes:
>> You can use "filterdiff -v --format=context".
>
> Cool, I'll have to get a copy of that.

Huh, I hadn't heard of that either. The Debian package patchutils says it was
downloaded from:

http://cyberelk.net/tim/data/patchutils

What's really cool is that patchutils also appears to have the utility I've
been looking for for a while: a tool to view the differences between two
diffs!

>> Because it's easy to convert from one to another, I think the unified
>> vs. context diff issue is a non-issue.
>
> Fair enough then; we should just change the policy.

It would be cool if the mail list software (optionally) automatically stripped
out diffs and replaced them with links to download the diff in either unified
or context format from the mail archives.

-- 
  Gregory Stark
  EnterpriseDB          http://www.enterprisedb.com

Re: Feature Freeze date for 8.4

Aidan Van Dyk <aidan@highrise.ca> — 2007-10-24T17:01:36Z

* Tom Lane <tgl@sss.pgh.pa.us> [071024 08:45]:
> Aidan Van Dyk <aidan@highrise.ca> writes:
> > * Brendan Jurd <direvus@gmail.com> [071024 01:41]:
> >> How up to date is the Git repos?  Does it pull individual commits from
> >> CVS, or does it resync the whole history periodically?  If so, what's
> >> the lag?
> 
> > It's updated hourly, which is the same rate the public CVS is updated.
> 
> FWIW, that's no longer the case: there's some sort of trigger mechanism
> now that causes the anoncvs mirror to update within a few minutes of a
> commit to the master.  So it might be worth the trouble to sync more
> often.

Note that it's working off the anoncvs rsync.  But I've set it to sync
every 15 minutes for now, we'll see how it goes...

-- 
Aidan Van Dyk                                             Create like a god,
aidan@highrise.ca                                       command like a king,
http://www.highrise.ca/                                   work like a slave.

Re: Feature Freeze date for 8.4

Alvaro Herrera <alvherre@commandprompt.com> — 2007-10-24T17:18:42Z

tomas@tuxteam.de wrote:
> On Wed, Oct 24, 2007 at 02:32:13PM +0200, Martijn van Oosterhout wrote:
> > On Tue, Oct 23, 2007 at 02:39:43PM -0700, David Fetter wrote:
> > > The one below is already available, so we don't have to do a "flag
> > > day" with it.
> > > 
> > > http://repo.or.cz/w/PostgreSQL.git
> > 
> > As someone who hasn't used GIT: if I have a modified CVS tree from some time
> > back (>1 year) can I use this to manage the bitrot? Just doing a CVS
> > update will probably just mark everything conflicted, so I'm wondering
> > is GIT can do it better.
> 
> It's no magic, but three-way merges[1] and automatic tracking of file
> renames do help quite a bit.

The other thing that helps is being able to apply (merge, propagate) the
changesets from mainline into your branch, one at a time, automatically
if they merge cleanly, and require manual intervention when they don't.
That way the changes required to merge each time are likely trivial.

-- 
Alvaro Herrera                 http://www.amazon.com/gp/registry/DXLWNGRJD34J
"Investigación es lo que hago cuando no sé lo que estoy haciendo"
(Wernher von Braun)

Re: Feature Freeze date for 8.4

David Fetter <david@fetter.org> — 2007-10-24T17:24:31Z

On Wed, Oct 24, 2007 at 02:18:42PM -0300, Alvaro Herrera wrote:
> tomas@tuxteam.de wrote:
> > On Wed, Oct 24, 2007 at 02:32:13PM +0200, Martijn van Oosterhout wrote:
> > > On Tue, Oct 23, 2007 at 02:39:43PM -0700, David Fetter wrote:
> > > > The one below is already available, so we don't have to do a "flag
> > > > day" with it.
> > > > 
> > > > http://repo.or.cz/w/PostgreSQL.git
> > > 
> > > As someone who hasn't used GIT: if I have a modified CVS tree
> > > from some time back (>1 year) can I use this to manage the
> > > bitrot? Just doing a CVS update will probably just mark
> > > everything conflicted, so I'm wondering is GIT can do it better.
> > 
> > It's no magic, but three-way merges[1] and automatic tracking of
> > file renames do help quite a bit.
> 
> The other thing that helps is being able to apply (merge, propagate)
> the changesets from mainline into your branch, one at a time,
> automatically if they merge cleanly, and require manual intervention
> when they don't.  That way the changes required to merge each time
> are likely trivial.

Yes :)

Cheers,
David.
-- 
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david.fetter@gmail.com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

Re: Feature Freeze date for 8.4

Florian Pflug <fgp.phlo.org@gmail.com> — 2007-10-24T22:27:33Z

Brendan Jurd wrote:
> On 10/24/07, Alvaro Herrera <alvherre@commandprompt.com> wrote:
>> Brendan Jurd escribió:
>>> Really?  I just started playing around with git, and the output from git 
>>> diff produced the same kind of diff file I would normally get from `svn 
>>> di`
>> ... which is a unified diff.
>> 
>>> or `cvs di -c`.
>> Huh, strange.  That's a context diff.
> 
> Right.  I was confusing unified and context, because the unified diffs I'm 
> looking at do actually show "context" lines above and below each change. 
> Sorry for the noise.

You can get git to produce a context diff by using GIT_EXTERNAL_DIFF to specify
an external diff program. I use a little shell script called
git-context-diff.sh, containing:
#!/bin/bash
diff --context=5 -L a/$1 -L b/$1 -p $2 $5
exit 0
and then create a diff with:
GIT_EXTERNAL_DIFF=git-context-diff.sh git diff <head to diff against>

Just in case you wondered - writing that script as just "exec diff ..." fails,
because diff seems to exit with a nonzero exitcode if the files differ, and "git
diff" aborts in that case...

regards, Florian Pflug

Re: Feature Freeze date for 8.4

kris.shannon@gmail.com — 2007-10-25T06:28:51Z

On 10/25/07, Gregory Stark <stark@enterprisedb.com> wrote:
> Huh, I hadn't heard of that either. The Debian package patchutils says it was
> downloaded from:
>
> http://cyberelk.net/tim/data/patchutils
>
> What's really cool is that patchutils also appears to have the utility I've
> been looking for for a while: a tool to view the differences between two
> diffs!

I wouldn't trust that too much.  I've played with it quite a bit and there are
quite a few corner cases where it gives the wrong answer (or at least a
different diff than I get when I manually apply them and then take the diff)

--
Kris Shannon <kris.shannon@gmail.com>

Re: Feature Freeze date for 8.4

Josh Berkus <josh@agliodbs.com> — 2007-10-27T00:24:37Z

Tom,

> This seems pretty entirely orthogonal to the commit-fest proposal.
> I see no reason to think that snapshots taken at those times would
> be any better than any other nightly snapshot, nor any reason
> to memorialize them in an archive.

I can see that.  And it would be pretty hard to keep members of the public 
from treating the persistent snapshots as releases, and then flooding the 
lists when they break.

So, scratch an idea with unintended consequences.  I'll just have to keep 
the snapshots on our CVS slave.

I do hope that the commit-fests will lead to more testing, though, just 
because people waiting for a specific feature will want to try out the 
initial commit of that feature.

-- 
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco