Thread

Win32, PITR, nested transactions, tablespaces

Bruce Momjian <pgman@candle.pha.pa.us> — 2004-05-27T20:31:57Z

Nearing our June 1 feature freeze, we now have patches for all our major
outstanding features.  Their status is:

	Win32 - almost done, fsync needed, bug fixes, etc
	PITR - Simon is away until Sunday and we are working on it
	Nested transactions - patch needs review, trigger issues
	Tablespaces - patch needs review

Congratulations to all the developers who finished up some very complex
patches.  I realize the patches will need adjustments, but it is amazing
they all got done so quickly.

Tom and I will be focused on Win32 and PITR for the next few days it
would help if others could review the other two patches for adjustments.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: Win32, PITR, nested transactions, tablespaces

Tatsuo Ishii <t-ishii@sra.co.jp> — 2004-05-28T01:05:52Z

Hi Bruce,

Is that core's decision that:

o below all four major features will be incorporated into 7.5
o beta freeze will be June 1

In the upcoming big event "Linux World" (in Tokyo, from June 2 to 4) I
need to present about current status of PostgreSQL development. Even
if cores have not decided yet, kind of information "it's likely
happen..." would be very valuable for the audience. Suggestion?
--
Tatsuo Ishii

From: Bruce Momjian <pgman@candle.pha.pa.us>
Subject: [HACKERS] Win32, PITR, nested transactions, tablespaces
Date: Thu, 27 May 2004 16:31:57 -0400 (EDT)
Message-ID: <200405272031.i4RKVvC13764@candle.pha.pa.us>

> Nearing our June 1 feature freeze, we now have patches for all our major
> outstanding features.  Their status is:
> 
> 	Win32 - almost done, fsync needed, bug fixes, etc
> 	PITR - Simon is away until Sunday and we are working on it
> 	Nested transactions - patch needs review, trigger issues
> 	Tablespaces - patch needs review
> 
> Congratulations to all the developers who finished up some very complex
> patches.  I realize the patches will need adjustments, but it is amazing
> they all got done so quickly.
> 
> Tom and I will be focused on Win32 and PITR for the next few days it
> would help if others could review the other two patches for adjustments.
> 
> -- 
>   Bruce Momjian                        |  http://candle.pha.pa.us
>   pgman@candle.pha.pa.us               |  (610) 359-1001
>   +  If your life is a hard drive,     |  13 Roberts Road
>   +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 2: you can get off all lists at once with the unregister command
>     (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
>

Re: Win32, PITR, nested transactions, tablespaces

Christopher Kings-Lynne <chriskl@familyhealth.com.au> — 2004-05-28T02:04:56Z

> Nearing our June 1 feature freeze, we now have patches for all our major
> outstanding features.  Their status is:
> 
> 	Win32 - almost done, fsync needed, bug fixes, etc
> 	PITR - Simon is away until Sunday and we are working on it
> 	Nested transactions - patch needs review, trigger issues
> 	Tablespaces - patch needs review
> 
> Congratulations to all the developers who finished up some very complex
> patches.  I realize the patches will need adjustments, but it is amazing
> they all got done so quickly.

I should point out to interested testers that the tablespace patch is 
the most mature out of the above list.  It's basically finished.

Chris

Re: Win32, PITR, nested transactions, tablespaces

Bruce Momjian <pgman@candle.pha.pa.us> — 2004-05-28T02:41:02Z

Tatsuo Ishii wrote:
> Hi Bruce,
> 
> Is that core's decision that:
> 
> o below all four major features will be incorporated into 7.5

We don't know.  If they can be added to CVS without major changes, they
will be in 7.5.  As far as I can tell:

	Win32 has 98% of its code in CVS, so it will make it
	Tablespaces - Christopher says it is ready, and has run tests
	PITR - some functionality might be in 7.5, but we aren't sure
	Nested transactions - Alvaro thinks it is close, but we don't know
 
> o beta freeze will be June 1

Yes, I think so.  That was Marc's decision actually.

> In the upcoming big event "Linux World" (in Tokyo, from June 2 to 4) I
> need to present about current status of PostgreSQL development. Even
> if cores have not decided yet, kind of information "it's likely
> happen..." would be very valuable for the audience. Suggestion?

I think some of them will make it, not sure about all.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: Win32, PITR, nested transactions, tablespaces

Chris Browne <cbbrowne@acm.org> — 2004-05-28T03:48:26Z

pgman@candle.pha.pa.us (Bruce Momjian) wrote:
> 	Win32 has 98% of its code in CVS, so it will make it
> 	Tablespaces - Christopher says it is ready, and has run tests
> 	PITR - some functionality might be in 7.5, but we aren't sure
> 	Nested transactions - Alvaro thinks it is close, but we don't know

Does this mean that 2PC is likely to be deferred?  I believe that
integration work on that was awaiting having nested transactions in
the codebase...
-- 
(reverse (concatenate 'string "gro.mca" "@" "enworbbc"))
http://www3.sympatico.ca/cbbrowne/emacs.html
Rules  of the  Evil  Overlord #30.   "All  bumbling conjurers,  clumsy
squires, no-talent  bards, and  cowardly thieves in  the land  will be
preemptively put  to death.  My foes  will surely give  up and abandon
their quest if they have no source of comic relief."
<http://www.eviloverlord.com/>

Re: Win32, PITR, nested transactions, tablespaces

Tatsuo Ishii <t-ishii@sra.co.jp> — 2004-05-28T04:00:33Z

Ok, BTW have we decided that next verison is marked as 7.5 or 8.0?
--
Tatsuo Ishii

> Tatsuo Ishii wrote:
> > Hi Bruce,
> > 
> > Is that core's decision that:
> > 
> > o below all four major features will be incorporated into 7.5
> 
> We don't know.  If they can be added to CVS without major changes, they
> will be in 7.5.  As far as I can tell:
> 
> 	Win32 has 98% of its code in CVS, so it will make it
> 	Tablespaces - Christopher says it is ready, and has run tests
> 	PITR - some functionality might be in 7.5, but we aren't sure
> 	Nested transactions - Alvaro thinks it is close, but we don't know
>  
> > o beta freeze will be June 1
> 
> Yes, I think so.  That was Marc's decision actually.
> 
> > In the upcoming big event "Linux World" (in Tokyo, from June 2 to 4) I
> > need to present about current status of PostgreSQL development. Even
> > if cores have not decided yet, kind of information "it's likely
> > happen..." would be very valuable for the audience. Suggestion?
> 
> I think some of them will make it, not sure about all.
> 
> -- 
>   Bruce Momjian                        |  http://candle.pha.pa.us
>   pgman@candle.pha.pa.us               |  (610) 359-1001
>   +  If your life is a hard drive,     |  13 Roberts Road
>   +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster
>

Re: Win32, PITR, nested transactions, tablespaces

Bruce Momjian <pgman@candle.pha.pa.us> — 2004-05-28T04:40:14Z

Tatsuo Ishii wrote:
> Ok, BTW have we decided that next verison is marked as 7.5 or 8.0?

No. Some argue that an 8.0 should only be done if we break backward
compatibility, while others feel major features should cause an 8.0.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: Win32, PITR, nested transactions, tablespaces

Tom Lane <tgl@sss.pgh.pa.us> — 2004-05-28T05:32:30Z

Bruce Momjian <pgman@candle.pha.pa.us> writes:
> Tatsuo Ishii wrote:
>> Ok, BTW have we decided that next verison is marked as 7.5 or 8.0?

> No. Some argue that an 8.0 should only be done if we break backward
> compatibility, while others feel major features should cause an 8.0.

In any case, it's premature to have that discussion until we know for
sure which of these features will make it into the release.  (If they
all do, I think there'd be a case for calling it 8.0.)

			regards, tom lane

Re: Win32, PITR, nested transactions, tablespaces

Tatsuo Ishii <t-ishii@sra.co.jp> — 2004-05-28T06:19:43Z

> > > Hi Bruce,
> > > 
> > > Is that core's decision that:
> > > 
> > > o below all four major features will be incorporated into 7.5
> > 
> > We don't know.  If they can be added to CVS without major changes, they
> > will be in 7.5.  As far as I can tell:
> > 
> > 	Win32 has 98% of its code in CVS, so it will make it
> > 	Tablespaces - Christopher says it is ready, and has run tests
> > 	PITR - some functionality might be in 7.5, but we aren't sure
> > 	Nested transactions - Alvaro thinks it is close, but we don't know

Today is May 28. So if nobody will not commit above within 3 days, non
of them will be in 7.5(8.0). Am I correct?
--
Tatsuo Ishii

Re: Win32, PITR, nested transactions, tablespaces

Marc G. Fournier <scrappy@postgresql.org> — 2004-05-28T13:52:52Z

On Fri, 28 May 2004, Tatsuo Ishii wrote:

>>>> Hi Bruce,
>>>>
>>>> Is that core's decision that:
>>>>
>>>> o below all four major features will be incorporated into 7.5
>>>
>>> We don't know.  If they can be added to CVS without major changes, they
>>> will be in 7.5.  As far as I can tell:
>>>
>>> 	Win32 has 98% of its code in CVS, so it will make it
>>> 	Tablespaces - Christopher says it is ready, and has run tests
>>> 	PITR - some functionality might be in 7.5, but we aren't sure
>>> 	Nested transactions - Alvaro thinks it is close, but we don't know
>
> Today is May 28. So if nobody will not commit above within 3 days, non
> of them will be in 7.5(8.0). Am I correct?

Not necessarily ... there are several Win32 related features that still 
haven't been committed (that I've seen) that will push back the release, 
and hopefully give some time for the other ones to get in ...

Off the top of my head ... the Installer and Service Manager come to mind 
... and Tom's fsync changes ...

Right now, I'd say feature freeze is looking more like next Friday (June 
4th), and we're evaluate it then ... that should hopefully give the above 
time to flesh out and get into CVS ...

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664

Re: Win32, PITR, nested transactions, tablespaces

Mike Mascari <mascarm@mascari.com> — 2004-05-28T14:23:47Z

Marc G. Fournier wrote:

> Right now, I'd say feature freeze is looking more like next Friday (June 
> 4th), and we're evaluate it then ... that should hopefully give the 
> above time to flesh out and get into CVS ...

Speaking of CVS, a CERT advisory was issued yesterday documenting a 
vulnerability:

http://www.us-cert.gov/cas/techalerts/TA04-147A.html

For what it's worth,

Mike Mascari

Re: Win32, PITR, nested transactions, tablespaces

Andrew Dunstan <andrew@dunslane.net> — 2004-05-28T14:57:48Z

Marc G. Fournier wrote:

> On Fri, 28 May 2004, Tatsuo Ishii wrote:
>
>>>>> Hi Bruce,
>>>>>
>>>>> Is that core's decision that:
>>>>>
>>>>> o below all four major features will be incorporated into 7.5
>>>>
>>>>
>>>> We don't know.  If they can be added to CVS without major changes, 
>>>> they
>>>> will be in 7.5.  As far as I can tell:
>>>>
>>>>     Win32 has 98% of its code in CVS, so it will make it
>>>>     Tablespaces - Christopher says it is ready, and has run tests
>>>>     PITR - some functionality might be in 7.5, but we aren't sure
>>>>     Nested transactions - Alvaro thinks it is close, but we don't know
>>>
>>
>> Today is May 28. So if nobody will not commit above within 3 days, non
>> of them will be in 7.5(8.0). Am I correct?
>
>
> Not necessarily ... there are several Win32 related features that 
> still haven't been committed (that I've seen) that will push back the 
> release, and hopefully give some time for the other ones to get in ...
>
> Off the top of my head ... the Installer and Service Manager come to 
> mind ... and Tom's fsync changes ...
>
> Right now, I'd say feature freeze is looking more like next Friday 
> (June 4th), and we're evaluate it then ... that should hopefully give 
> the above time to flesh out and get into CVS ...
>

The Installer is a pgFoundry project, and unaffected by feature freeze. 
See http://pginstaller.projects.postgresql.org (Note: Magnus hasn't put 
anything there yet, though.)

I certainly get the feeling that things are being rushed just a bit too 
much, and think having a extra few days of breathing space makes sense.

cheers

andrew

Re: Win32, PITR, nested transactions, tablespaces

Marc G. Fournier <scrappy@postgresql.org> — 2004-05-28T15:27:35Z

On Fri, 28 May 2004, Andrew Dunstan wrote:

> The Installer is a pgFoundry project, and unaffected by feature freeze. 
> See http://pginstaller.projects.postgresql.org (Note: Magnus hasn't put 
> anything there yet, though.)

'k, wasn't sure about that one ... but the SVC and fsync stuff is still 
outstanding, and they are both backend commits ...

Bruce, do you have a full list of what is outstanding on the Win32 side, 
or is it just those two?

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664

Re: Win32, PITR, nested transactions, tablespaces

Matthew T. O'Connor <matthew@zeut.net> — 2004-05-28T16:18:10Z

> I certainly get the feeling that things are being rushed just a bit too
> much, and think having a extra few days of breathing space makes sense.
>
> cheers
>
> andrew


I have that feeling too, and I'm working still working on pg_autovacuum
integration which I was hoping to get in, so I would welcome a few more
days to a week.

Matthew

Re: Win32, PITR, nested transactions, tablespaces

Tom Lane <tgl@sss.pgh.pa.us> — 2004-05-28T16:48:40Z

"Marc G. Fournier" <scrappy@postgresql.org> writes:
> Off the top of my head ... the Installer and Service Manager come to mind 
> ... and Tom's fsync changes ...

> Right now, I'd say feature freeze is looking more like next Friday (June 
> 4th), and we're evaluate it then ... that should hopefully give the above 
> time to flesh out and get into CVS ...

I intend to be done with fsync by Monday, but I won't have any time to
review these other patches until after that.  So pushing feature freeze
back a week sounds like a good idea to me too.  We need enough time to
make unhasty decisions about whether each of these features is ready
enough to go into 7.5 (or whatever we call it).

			regards, tom lane

Re: Win32, PITR, nested transactions, tablespaces

Bruce Momjian <pgman@candle.pha.pa.us> — 2004-05-28T18:08:09Z

Tom Lane wrote:
> "Marc G. Fournier" <scrappy@postgresql.org> writes:
> > Off the top of my head ... the Installer and Service Manager come to mind 
> > ... and Tom's fsync changes ...
> 
> > Right now, I'd say feature freeze is looking more like next Friday (June 
> > 4th), and we're evaluate it then ... that should hopefully give the above 
> > time to flesh out and get into CVS ...
> 
> I intend to be done with fsync by Monday, but I won't have any time to
> review these other patches until after that.  So pushing feature freeze
> back a week sounds like a good idea to me too.  We need enough time to
> make unhasty decisions about whether each of these features is ready
> enough to go into 7.5 (or whatever we call it).

Agreed, but I am concerned we are entering a period like we did with 7.3
where we delay things a week at a time rather than in one-month chunks.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: Win32, PITR, nested transactions, tablespaces

Marc G. Fournier <scrappy@postgresql.org> — 2004-05-28T20:54:45Z

On Fri, 28 May 2004, Bruce Momjian wrote:

> Tom Lane wrote:
>> "Marc G. Fournier" <scrappy@postgresql.org> writes:
>>> Off the top of my head ... the Installer and Service Manager come to mind
>>> ... and Tom's fsync changes ...
>>
>>> Right now, I'd say feature freeze is looking more like next Friday (June
>>> 4th), and we're evaluate it then ... that should hopefully give the above
>>> time to flesh out and get into CVS ...
>>
>> I intend to be done with fsync by Monday, but I won't have any time to
>> review these other patches until after that.  So pushing feature freeze
>> back a week sounds like a good idea to me too.  We need enough time to
>> make unhasty decisions about whether each of these features is ready
>> enough to go into 7.5 (or whatever we call it).
>
> Agreed, but I am concerned we are entering a period like we did with 7.3
> where we delay things a week at a time rather than in one-month chunks.

The *only* thing we are pushing back the beta for is so that the Win32 
stuff can be completed ...

There is no reason why those working on the other features should be 
slowing down as a result that they think they might not make it, since 
even if they don't make it, it would be *really* nice to see the features 
added at the start of the next dev cycle ...


----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664

Re: Win32, PITR, nested transactions, tablespaces

Bruce Momjian <pgman@candle.pha.pa.us> — 2004-05-28T21:17:21Z

Marc G. Fournier wrote:
> On Fri, 28 May 2004, Bruce Momjian wrote:
> 
> > Tom Lane wrote:
> >> "Marc G. Fournier" <scrappy@postgresql.org> writes:
> >>> Off the top of my head ... the Installer and Service Manager come to mind
> >>> ... and Tom's fsync changes ...
> >>
> >>> Right now, I'd say feature freeze is looking more like next Friday (June
> >>> 4th), and we're evaluate it then ... that should hopefully give the above
> >>> time to flesh out and get into CVS ...
> >>
> >> I intend to be done with fsync by Monday, but I won't have any time to
> >> review these other patches until after that.  So pushing feature freeze
> >> back a week sounds like a good idea to me too.  We need enough time to
> >> make unhasty decisions about whether each of these features is ready
> >> enough to go into 7.5 (or whatever we call it).
> >
> > Agreed, but I am concerned we are entering a period like we did with 7.3
> > where we delay things a week at a time rather than in one-month chunks.
> 
> The *only* thing we are pushing back the beta for is so that the Win32 
> stuff can be completed ...
> 
> There is no reason why those working on the other features should be 
> slowing down as a result that they think they might not make it, since 
> even if they don't make it, it would be *really* nice to see the features 
> added at the start of the next dev cycle ...

Reality check  --- why should they keep going if the stuff if appication
is going to be months away, and these big features are going to change
CVS significanly anyway?

Let's be honest --- if they aren't going to make it, they will stop
working hard and do something else with their time.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: Win32, PITR, nested transactions, tablespaces

Marc G. Fournier <scrappy@postgresql.org> — 2004-05-28T21:39:20Z

On Fri, 28 May 2004, Bruce Momjian wrote:

> Marc G. Fournier wrote:
>> On Fri, 28 May 2004, Bruce Momjian wrote:
>>
>>> Tom Lane wrote:
>>>> "Marc G. Fournier" <scrappy@postgresql.org> writes:
>>>>> Off the top of my head ... the Installer and Service Manager come to mind
>>>>> ... and Tom's fsync changes ...
>>>>
>>>>> Right now, I'd say feature freeze is looking more like next Friday (June
>>>>> 4th), and we're evaluate it then ... that should hopefully give the above
>>>>> time to flesh out and get into CVS ...
>>>>
>>>> I intend to be done with fsync by Monday, but I won't have any time to
>>>> review these other patches until after that.  So pushing feature freeze
>>>> back a week sounds like a good idea to me too.  We need enough time to
>>>> make unhasty decisions about whether each of these features is ready
>>>> enough to go into 7.5 (or whatever we call it).
>>>
>>> Agreed, but I am concerned we are entering a period like we did with 7.3
>>> where we delay things a week at a time rather than in one-month chunks.
>>
>> The *only* thing we are pushing back the beta for is so that the Win32
>> stuff can be completed ...
>>
>> There is no reason why those working on the other features should be
>> slowing down as a result that they think they might not make it, since
>> even if they don't make it, it would be *really* nice to see the features
>> added at the start of the next dev cycle ...
>
> Reality check  --- why should they keep going if the stuff if appication
> is going to be months away, and these big features are going to change
> CVS significanly anyway?
>
> Let's be honest --- if they aren't going to make it, they will stop
> working hard and do something else with their time.

'k, sounds reasonable ... so, since the feature freeze will most likely 
happen next Friday (not guaranteed, of course), everyone working on 
features that will not be done in time should just stop working on it ... 
that should free up a bunch of ppl for testing those features that are 
already in ...

Of course, that *is* based on the Reality Check you've so kindly provided 
me ...

And, of course, that puts us into the next dev cycle with a bunch of half 
finished, not ready to commit, features that will most likely be committed 
days ahead of the feature freeze ... instead of a bunch of features that 
were finished off *during* the beta cycle and ready to be committed the 
day after we create the branch ...

Your Reality sounds like such a waste of time and talent, doesn't it?

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664

Re: Win32, PITR, nested transactions, tablespaces

Greg Stark <gsstark@mit.edu> — 2004-05-29T05:23:42Z

Bruce Momjian <pgman@candle.pha.pa.us> writes:

> Reality check  --- why should they keep going if the stuff if appication
> is going to be months away, and these big features are going to change
> CVS significantly anyway?
> 
> Let's be honest --- if they aren't going to make it, they will stop
> working hard and do something else with their time.

I subscribe to a lot of development mailing lists. This is the only place I've
seen people say things like this. You're not the only one, someone else made a
similar comment recently too.

Usually in other projects when a new development cycle starts and the tree
frees up after a release there's a rush of new features. That's when people
finally feel free to start applying all the radical changes they dreamed up
during the previous development cycle. Normally they hold off on them if
they're towards the end of the development cycle even if they're not in a
formal freeze. It just just takes time for these radical changes to settle out
and they feel more comfortable applying them early in the cycle.

This is the only place where I see hardly any movement on major items the
whole development cycle, then a rush of radical changes just before the
freeze.

It's a little weird.

-- 
greg

Re: Win32, PITR, nested transactions, tablespaces

Tom Lane <tgl@sss.pgh.pa.us> — 2004-05-29T06:09:58Z

Greg Stark <gsstark@mit.edu> writes:
> This is the only place where I see hardly any movement on major items the
> whole development cycle, then a rush of radical changes just before the
> freeze.

[blink]  There's been plenty of stuff done all through this development
cycle (and previous ones too).  Read the CVS logs if you've forgotten.

Bruce has got some kind of idee fixe that we've accomplished little
during the 7.5 development cycle, but I respectfully disagree.  I could
point out the indexability of "WHERE int8col = 42" as just one thing
that is a major step forward in usability.  It is true that there are
a few more big-ticket items that seem to be just over the horizon,
but isn't that the usual state of affairs?  If we were always willing
to wait "just one more month" for the next almost-done project, we'd
never get a release out at all.

Now having said that, I did just vote in the core discussion to push the
freeze date out a month.  But I don't have any patience for the idea
that 7.5 won't be a pretty respectable release if we freeze it Monday
per original plan.

			regards, tom lane

Re: Win32, PITR, nested transactions, tablespaces

Greg Stark <gsstark@mit.edu> — 2004-05-29T14:04:04Z

Tom Lane <tgl@sss.pgh.pa.us> writes:

> Greg Stark <gsstark@mit.edu> writes:
> > This is the only place where I see hardly any movement on major items the
> > whole development cycle, then a rush of radical changes just before the
> > freeze.
> 
> [blink]  There's been plenty of stuff done all through this development
> cycle (and previous ones too).  Read the CVS logs if you've forgotten.

Sure, but that's parallel to what I'm saying. This is the only place I see
"Please delay the freeze so I can squeeze this major change in just before the
release". In other projects I see "Please hurry up and release so I can start
committing major changes again".

Perhaps it's an artifact of people doing most of their work offline and
submitting patches, rather than using the CVS tree as their development
environment. Or perhaps it's an artifact of ~nobody using the CVS version of
postgres except for testing patches. Or perhaps it's a consequence of the
freeze period being so long.

Or perhaps the serious postgres developers are just so good that they're
justified in being confident applying major changes just before a freeze.
Experience does seem to justify that somewhat; I've been repeatedly impressed
at how such drastic changes seem to just work with hardly any changes.

Fwiw, I do feel that 7.4 is pretty fresh. At least in my case I don't plan on
upgrading to 7.5 immediately because 7.4 meets all our needs. When we upgrade
it'll probably be for PITR, but we don't really need it yet.

-- 
greg

Re: Win32, PITR, nested transactions, tablespaces

Chris Browne <cbbrowne@acm.org> — 2004-05-29T18:59:22Z

The world rejoiced as gsstark@mit.edu (Greg Stark) wrote:
> Tom Lane <tgl@sss.pgh.pa.us> writes:
>
>> Greg Stark <gsstark@mit.edu> writes:
>> > This is the only place where I see hardly any movement on major
>> > items the whole development cycle, then a rush of radical changes
>> > just before the freeze.
>> 
>> [blink] There's been plenty of stuff done all through this
>> development cycle (and previous ones too).  Read the CVS logs if
>> you've forgotten.
>
> Sure, but that's parallel to what I'm saying. This is the only place
> I see "Please delay the freeze so I can squeeze this major change in
> just before the release". In other projects I see "Please hurry up
> and release so I can start committing major changes again".
>
> Perhaps it's an artifact of people doing most of their work offline
> and submitting patches, rather than using the CVS tree as their
> development environment. Or perhaps it's an artifact of ~nobody
> using the CVS version of postgres except for testing patches. Or
> perhaps it's a consequence of the freeze period being so long.

I did a compile the other day of the "latest CVS" version on AIX; I
wasn't seriously testing it in general, but was at least pleased to
see it passing regression cleanly.

One of the things that looks interesting is 2 Phase Commit.  THAT has
been holding off for nested transactions to get in place, as both
these features twiddle with how transactions work.  Regrettably,
integrating it all together is sure to take a while.  From what I see
now, I hope nested transactions make it into 7.5 so that 2PC can make
it into 7.6.

> Or perhaps the serious postgres developers are just so good that
> they're justified in being confident applying major changes just
> before a freeze.  Experience does seem to justify that somewhat;
> I've been repeatedly impressed at how such drastic changes seem to
> just work with hardly any changes.

I suspect we may need additional regression tests to check the
behaviour of lazy vacuum and ARC, since they are _usually_ supposed to
be pretty invisible to applications; the regular application and
reapplication of the set of regression tests has been pretty effective
at preventing the code from getting too far away from working validly
along the way.

> Fwiw, I do feel that 7.4 is pretty fresh. At least in my case I
> don't plan on upgrading to 7.5 immediately because 7.4 meets all our
> needs. When we upgrade it'll probably be for PITR, but we don't
> really need it yet.

ARC and lazy vacuum look like valuable things for the "transactional"
systems I'm working with.  It looks like we sometimes see delays
coming from checkpoints that 7.5 looks like it may moderate.  Mind
you, some systems only recently "leapt" to 7.4, so it would be quite
surprising to quickly leap again to 7.5.

PITR may turn out to be a "don't care" item if Slony1 winds up
providing its own approach to PITR.  (e.g. - if you  write out to
disk the sets of SQL statements that are to be applied to a replica,
then the spooled sets of these statements represent a history of
updates that can be used to do PITR.)
-- 
"cbbrowne","@","ntlug.org"
http://www.ntlug.org/~cbbrowne/emacs.html
I'm sorry Dave, I can't let you do that.
Why don't you lie down and take a stress pill?

Re: Win32, PITR, nested transactions, tablespaces

Greg Stark <gsstark@mit.edu> — 2004-06-01T16:44:37Z

Christopher Browne <cbbrowne@acm.org> writes:

> PITR may turn out to be a "don't care" item if Slony1 winds up
> providing its own approach to PITR.  (e.g. - if you  write out to
> disk the sets of SQL statements that are to be applied to a replica,
> then the spooled sets of these statements represent a history of
> updates that can be used to do PITR.)

In the long run nothing can substitute for PITR. It's the only way to get a
backup that is guaranteed to restore you to exactly what you had before.

Logical dumps a la pg_dump suffer from having to unparse and parse all the
data. Any data types that don't accurately store themselves as text (such as
arrays currently, due to the index lower bound) get corrupted.

SQL level replication, a la Slony wouldn't serve if you have any
non-deterministic behaviour. And given SQL's set theoretic roots
non-deterministic behaviour can creep in in places where it's not expected,
such as any query that doesn't have an ORDER BY clause.

Both of these tools have their uses, but they don't provide a rock solid
guarantee that you can restore a machine to exactly what you had previously.
To do that you really want something that works at the storage level and
doesn't try to re-interpret data or reapply any logic.

-- 
greg

Re: Win32, PITR, nested transactions, tablespaces

Heikki Linnakangas <hlinnaka@iki.fi> — 2004-06-01T19:53:50Z

On Thu, 27 May 2004, Christopher Browne wrote:

> pgman@candle.pha.pa.us (Bruce Momjian) wrote:
> > 	Win32 has 98% of its code in CVS, so it will make it
> > 	Tablespaces - Christopher says it is ready, and has run tests
> > 	PITR - some functionality might be in 7.5, but we aren't sure
> > 	Nested transactions - Alvaro thinks it is close, but we don't know
>
> Does this mean that 2PC is likely to be deferred?  I believe that
> integration work on that was awaiting having nested transactions in
> the codebase...

I'm not waiting for the nested transactions to land anymore. I have simply
been too busy to finish up 2PC. :(

When I said I'd wait for nested transactions, some weeks ago, I thought
that Alvaros patch was going to be checked in within a couple of days or a
week. That impression was wrong. Anyway, I don't believe the patches
conflict very badly, so there's not really a need for either one to wait.

Pushing the freeze date to 1. July just might be enough for me to finish
the 2PC patch, but don't get your hopes up. I don't know how much time I
can spend on it in June.

The issue that I'm currently pondering is what to do with SET SESSION
variables on 2PC. This far my thinking has been that the PREPARE
TRANSACTION command (1st phase) detaches the transaction from the backend
You can do whatever you want with the backend, run other transaction,
drop the connection etc. The prepared transaction acts just like a
running transaction in another backend. You can also use a different
backend to commit the transaction. The question is, what happens to
session variables when the first transaction commits? The original
backend could be in the middle of another transaction, surely you can't
just modify the variables. But you can't really ignore them either.

There is also a lot of other functionality that throws "not implemented"
errors if you try to use 2PC with them. For example, DDL and
notifications. I believe they are quite straightforward, I just
haven't had the time to tackle them yet.

You can always check the latest version of the patch at
http://www.hut.fi/~hlinnaka/pgsql/, I try to update it whenever there is
progress.

- Heikki

Re: Win32, PITR, nested transactions, tablespaces

Tom Lane <tgl@sss.pgh.pa.us> — 2004-06-01T21:27:19Z

Heikki Linnakangas <hlinnaka@iki.fi> writes:
> There is also a lot of other functionality that throws "not implemented"
> errors if you try to use 2PC with them. For example, DDL and
> notifications.

Why would DDL be different from any other query?

			regards, tom lane