Thread

RE: (one more time) Patches with vacuum fixes available .

Vadim Mikheev <vmikheev@sectorbase.com> — 2000-12-12T00:33:24Z

> > > If Vadim isn't sufficiently confident of it to commit it 
> > > on his own authority, I'm inclined to leave it out of 7.1.
> > > My concern is mostly schedule. We are well into beta cycle
> > > now and this seems like way too critical (not to say high-risk)
> > > a feature to be adding after start of beta.
> >
> > I was wondering if Vadim was hesitant because he had done this
> > under contract.  Vadim, are you concerned about reliability or
> > are there other issues?
>
> Irrelevant .. we are post-beta release, and this doesn't fix 
> a bug, so it doesn't go in ...

1. I'm confident about code quality.
2. Copyright issue is resolved.
3. But we're in beta now.

If there are no objections then I'm ready to add changes to 7.1.
Else, I'll produce patches for 7.1 just after release and incorporate
changes into 7.2.

Comments?

Vadim

RE: (one more time) Patches with vacuum fixes available .

Marc G. Fournier <scrappy@hub.org> — 2000-12-12T01:12:55Z

On Mon, 11 Dec 2000, Mikheev, Vadim wrote:

> > > > If Vadim isn't sufficiently confident of it to commit it 
> > > > on his own authority, I'm inclined to leave it out of 7.1.
> > > > My concern is mostly schedule. We are well into beta cycle
> > > > now and this seems like way too critical (not to say high-risk)
> > > > a feature to be adding after start of beta.
> > >
> > > I was wondering if Vadim was hesitant because he had done this
> > > under contract.  Vadim, are you concerned about reliability or
> > > are there other issues?
> >
> > Irrelevant .. we are post-beta release, and this doesn't fix 
> > a bug, so it doesn't go in ...
> 
> 1. I'm confident about code quality.
> 2. Copyright issue is resolved.
> 3. But we're in beta now.
> 
> If there are no objections then I'm ready to add changes to 7.1.
> Else, I'll produce patches for 7.1 just after release and incorporate
> changes into 7.2.
> 
> Comments?

IMHO, we are in beta now and this doesn't fix a bug ... if we could make
this available as a patch in /contrib until 7.1 is released, i would much
prefer that ...

Re: (one more time) Patches with vacuum fixes available .

Bruce Momjian <pgman@candle.pha.pa.us> — 2000-12-12T01:34:06Z

> > If there are no objections then I'm ready to add changes to 7.1.
> > Else, I'll produce patches for 7.1 just after release and incorporate
> > changes into 7.2.
> > 
> > Comments?
> 
> IMHO, we are in beta now and this doesn't fix a bug ... if we could make
> this available as a patch in /contrib until 7.1 is released, i would much
> prefer that ...

But we just entered beta.  Can't we let this slide?  Seems like a nice
feature.  I can't image it delaying anything.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Re: (one more time) Patches with vacuum fixes available .

Marc G. Fournier <scrappy@hub.org> — 2000-12-12T01:37:17Z

Go for it Vadim ... it is only a couple of days in, and I know there are
several places I could personally use it ... 

On Mon, 11 Dec 2000, Bruce Momjian wrote:

> > > If there are no objections then I'm ready to add changes to 7.1.
> > > Else, I'll produce patches for 7.1 just after release and incorporate
> > > changes into 7.2.
> > > 
> > > Comments?
> > 
> > IMHO, we are in beta now and this doesn't fix a bug ... if we could make
> > this available as a patch in /contrib until 7.1 is released, i would much
> > prefer that ...
> 
> But we just entered beta.  Can't we let this slide?  Seems like a nice
> feature.  I can't image it delaying anything.
> 
> -- 
>   Bruce Momjian                        |  http://candle.pha.pa.us
>   pgman@candle.pha.pa.us               |  (610) 853-3000
>   +  If your life is a hard drive,     |  830 Blythe Avenue
>   +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026
> 

Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
Systems Administrator @ hub.org 
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org

Re: (one more time) Patches with vacuum fixes available .

Bruce Momjian <pgman@candle.pha.pa.us> — 2000-12-12T02:01:47Z

Thanks.  The other good item is that is already being used in production
use, so it seems it is pretty well tested.

> 
> Go for it Vadim ... it is only a couple of days in, and I know there are
> several places I could personally use it ... 
> 
> On Mon, 11 Dec 2000, Bruce Momjian wrote:
> 
> > > > If there are no objections then I'm ready to add changes to 7.1.
> > > > Else, I'll produce patches for 7.1 just after release and incorporate
> > > > changes into 7.2.
> > > > 
> > > > Comments?
> > > 
> > > IMHO, we are in beta now and this doesn't fix a bug ... if we could make
> > > this available as a patch in /contrib until 7.1 is released, i would much
> > > prefer that ...
> > 
> > But we just entered beta.  Can't we let this slide?  Seems like a nice
> > feature.  I can't image it delaying anything.
> > 
> > -- 
> >   Bruce Momjian                        |  http://candle.pha.pa.us
> >   pgman@candle.pha.pa.us               |  (610) 853-3000
> >   +  If your life is a hard drive,     |  830 Blythe Avenue
> >   +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026
> > 
> 
> Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
> Systems Administrator @ hub.org 
> primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 
> 
> 


-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Re: (one more time) Patches with vacuum fixes available .

Tom Lane <tgl@sss.pgh.pa.us> — 2000-12-12T02:05:17Z

"Mikheev, Vadim" <vmikheev@SECTORBASE.COM> writes:
> If there are no objections then I'm ready to add changes to 7.1.
> Else, I'll produce patches for 7.1 just after release and incorporate
> changes into 7.2.

I'd vote for the second choice.  I do not think we should be adding new
features now.  Also, I don't know about you, but I have enough bug fix,
testing, and documentation work to keep me busy till January even
without any new features...

			regards, tom lane

Re: (one more time) Patches with vacuum fixes available .

Marc G. Fournier <scrappy@hub.org> — 2000-12-12T02:42:10Z

On Mon, 11 Dec 2000, Tom Lane wrote:

> "Mikheev, Vadim" <vmikheev@SECTORBASE.COM> writes:
> > If there are no objections then I'm ready to add changes to 7.1.
> > Else, I'll produce patches for 7.1 just after release and incorporate
> > changes into 7.2.
> 
> I'd vote for the second choice.  I do not think we should be adding new
> features now.  Also, I don't know about you, but I have enough bug fix,
> testing, and documentation work to keep me busy till January even
> without any new features...

A few points in favor of including this ... first, when Vadim does do the
storage manager rewrite for v7.2, the patch is essentially useless ... and
second, its currently being used in production on a server that is/will
tax it heavily, so it isn't untested ...

I'd almost extend that to the point that it is probably more tested right
now then any other feature that has been added to v7.1 pre-beta,
considering my knowledge of Alfred's environment ...

Re: (one more time) Patches with vacuum fixes available .

Bruce Momjian <pgman@candle.pha.pa.us> — 2000-12-12T03:23:03Z

> A few points in favor of including this ... first, when Vadim does do the
> storage manager rewrite for v7.2, the patch is essentially useless ... and
> second, its currently being used in production on a server that is/will
> tax it heavily, so it isn't untested ...
> 
> I'd almost extend that to the point that it is probably more tested right
> now then any other feature that has been added to v7.1 pre-beta,
> considering my knowledge of Alfred's environment ...

Ewe, that is a strong argument.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Re: (one more time) Patches with vacuum fixes available .

Tom Lane <tgl@sss.pgh.pa.us> — 2000-12-12T03:56:32Z

Bruce Momjian <pgman@candle.pha.pa.us> writes:
> But we just entered beta.  Can't we let this slide?  Seems like a nice
> feature.  I can't image it delaying anything.

I can imagine it *breaking* lots of things.

We just today discovered that we didn't understand the interaction
between VACUUM and TOAST well enough.  How well do you think we
understand this new VACUUM behavior that no one but Vadim has even
seen?

With all due respect to Vadim, this scares me a lot.  I vote NO.

			regards, tom lane

Re: (one more time) Patches with vacuum fixes available .

Tom Lane <tgl@sss.pgh.pa.us> — 2000-12-12T04:06:48Z

The Hermit Hacker <scrappy@hub.org> writes:
> I'd almost extend that to the point that it is probably more tested right
> now then any other feature that has been added to v7.1 pre-beta,
> considering my knowledge of Alfred's environment ...

And wasn't Alfred hollering yesterday about sudden instability?
See the Re: [COMMITTERS] pgsql/src/include (config.h.in) thread
over in committers.

My initial guess is that he's seeing the VACUUM patch tickling a
syscache-entry-drop problem --- if so, it shouldn't be a problem in 7.1
--- but I have no evidence for that guess yet.  I'm sure not confident
enough of the guess to not be afraid of dropping the patch into 7.1.

			regards, tom lane

Re: (one more time) Patches with vacuum fixes available .

Andrew Snow <als@fl.net.au> — 2000-12-12T04:12:12Z

On Mon, 11 Dec 2000, Tom Lane wrote:

> "Mikheev, Vadim" <vmikheev@SECTORBASE.COM> writes:
> > If there are no objections then I'm ready to add changes to 7.1.
> > Else, I'll produce patches for 7.1 just after release and incorporate
> > changes into 7.2.
> 
> I'd vote for the second choice.  I do not think we should be adding new
> features now.  Also, I don't know about you, but I have enough bug fix,
> testing, and documentation work to keep me busy till January even
> without any new features...

It'd be really naughty to add it to the beta at this stage.  Would it be
possible to add it to the 7.1 package with some kind of compile-time option?
So that those of us who do want to use it, can.

- Andrew

Re: (one more time) Patches with vacuum fixes available .

Marc G. Fournier <scrappy@hub.org> — 2000-12-12T04:18:41Z

On Mon, 11 Dec 2000, Tom Lane wrote:

> The Hermit Hacker <scrappy@hub.org> writes:
> > I'd almost extend that to the point that it is probably more tested right
> > now then any other feature that has been added to v7.1 pre-beta,
> > considering my knowledge of Alfred's environment ...
> 
> And wasn't Alfred hollering yesterday about sudden instability?
> See the Re: [COMMITTERS] pgsql/src/include (config.h.in) thread
> over in committers.
> 
> My initial guess is that he's seeing the VACUUM patch tickling a
> syscache-entry-drop problem --- if so, it shouldn't be a problem in 7.1
> --- but I have no evidence for that guess yet.  I'm sure not confident
> enough of the guess to not be afraid of dropping the patch into 7.1.

should be easily testable though, no?  worst case, we pull it out
afterwards ...

Re: (one more time) Patches with vacuum fixes available .

Bruce Momjian <pgman@candle.pha.pa.us> — 2000-12-12T04:19:09Z

> > I'd vote for the second choice.  I do not think we should be adding new
> > features now.  Also, I don't know about you, but I have enough bug fix,
> > testing, and documentation work to keep me busy till January even
> > without any new features...
> 
> It'd be really naughty to add it to the beta at this stage.  Would it be
> possible to add it to the 7.1 package with some kind of compile-time option?
> So that those of us who do want to use it, can.

That usually adds confusion.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Re: (one more time) Patches with vacuum fixes available .

Tom Lane <tgl@sss.pgh.pa.us> — 2000-12-12T04:32:17Z

The Hermit Hacker <scrappy@hub.org> writes:
> should be easily testable though, no?

What makes you think that?  

> worst case, we pull it out afterwards ...

No, worst case is that we release a seriously broken 7.1, and don't
find out till afterwards.

There are plenty of new features on my to-do list, if beta no longer
means anything...

			regards, tom lane

Re: (one more time) Patches with vacuum fixes available .

Marc G. Fournier <scrappy@hub.org> — 2000-12-12T04:41:20Z

On Mon, 11 Dec 2000, Tom Lane wrote:

> The Hermit Hacker <scrappy@hub.org> writes:
> > should be easily testable though, no?
> 
> What makes you think that?  

Alfred could volunteer to move to v7.1? *grin*

Re: (one more time) Patches with vacuum fixes available .

Alfred Perlstein <bright@wintelcom.net> — 2000-12-12T06:20:24Z

* Andrew Snow <als@fl.net.au> [001211 20:21] wrote:
> 
> On Mon, 11 Dec 2000, Tom Lane wrote:
> 
> > "Mikheev, Vadim" <vmikheev@SECTORBASE.COM> writes:
> > > If there are no objections then I'm ready to add changes to 7.1.
> > > Else, I'll produce patches for 7.1 just after release and incorporate
> > > changes into 7.2.
> > 
> > I'd vote for the second choice.  I do not think we should be adding new
> > features now.  Also, I don't know about you, but I have enough bug fix,
> > testing, and documentation work to keep me busy till January even
> > without any new features...
> 
> It'd be really naughty to add it to the beta at this stage.  Would it be
> possible to add it to the 7.1 package with some kind of compile-time option?
> So that those of us who do want to use it, can.

One is a compile time option (CFLAGS+=-DMMNB), the other doesn't
happen unless you ask for it:

vacuum lazy <table>;

I don't understand what the deal here is, as I said, it's optional
code that you won't see unless you ask for it.

            [children: 0 12/11/2000 21:57:20 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx]
Vacuuming link.
            [children: 0 12/11/2000 21:57:54 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx]

-rw-------  1 pgsql  pgsql  134627328 Dec 11 21:57 link
-rw-------  1 pgsql  pgsql  261201920 Dec 11 21:57 link_triple_idx

Yup, 30 seconds, the table is 134 megabytes and the index is 261 megs.

I think normally this takes about 10 or so _minutes_.

On our faster server:

            [children: 0 12/11/2000 22:17:50 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx]
Vacuuming referer_link.
            [children: 0 12/11/2000 22:18:09 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx]

-rw-------  1 pgsql  wheel  273670144 Dec 11 22:15 link
-rw-------  1 pgsql  wheel  641048576 Dec 11 22:15 link_triple_idx

time is ~19seconds, table is 273 megs, and index 641 megs.

dual 800mhz, raid 5 disks.

I think the users deserve this patch. :)

-- 
-Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org]
"I have the heart of a child; I keep it in a jar on my desk."

Re: (one more time) Patches with vacuum fixes available .

Nathan Myers <ncm@zembu.com> — 2000-12-12T23:51:43Z

On Mon, Dec 11, 2000 at 11:32:17PM -0500, Tom Lane wrote:
> The Hermit Hacker <scrappy@hub.org> writes:
> > worst case, we pull it out afterwards ...
> 
> No, worst case is that we release a seriously broken 7.1, and don't
> find out till afterwards.
> 
> There are plenty of new features on my to-do list, if beta no longer
> means anything...

Does this mean the code gets put in contrib/, with a prominent
pointer in the release notes?

Nathan Myers
ncm@zembu.com

Re: (one more time) Patches with vacuum fixes available .

Bruce Momjian <pgman@candle.pha.pa.us> — 2001-01-24T14:44:08Z

Did we decide against LAZY?  Seems we have a number of people concerned
about vacuum downtime, and I can see this as a win for them.  If they
don't specify LAZY, the code is not run.

> The Hermit Hacker <scrappy@hub.org> writes:
> > should be easily testable though, no?
> 
> What makes you think that?  
> 
> > worst case, we pull it out afterwards ...
> 
> No, worst case is that we release a seriously broken 7.1, and don't
> find out till afterwards.
> 
> There are plenty of new features on my to-do list, if beta no longer
> means anything...
> 
> 			regards, tom lane
> 


-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Re: (one more time) Patches with vacuum fixes available .

Tom Lane <tgl@sss.pgh.pa.us> — 2001-01-24T15:16:32Z

Bruce Momjian <pgman@candle.pha.pa.us> writes:
> Did we decide against LAZY?

I thought the core consensus was that it was too risky to install
post-beta.  On the other hand, we're installing some other pretty
major fixes.  Do we want to re-open that discussion?

			regards, tom lane

Re: (one more time) Patches with vacuum fixes available .

Marc G. Fournier <scrappy@hub.org> — 2001-01-24T15:52:38Z

On Wed, 24 Jan 2001, Tom Lane wrote:

> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > Did we decide against LAZY?
>
> I thought the core consensus was that it was too risky to install
> post-beta.  On the other hand, we're installing some other pretty
> major fixes.  Do we want to re-open that discussion?

I would like to see it in myself ... it will be useless post v7.1, with
all the work planned for v7.2 ... if there is some way we could get it in
before beta4 goes out, I'd feel even better about it ... Vadim? :)

Re: (one more time) Patches with vacuum fixes available .

Lamar Owen <lamar.owen@wgcr.org> — 2001-01-24T18:50:10Z

Bruce Momjian wrote:
> 
> Did we decide against LAZY?  Seems we have a number of people concerned
> about vacuum downtime, and I can see this as a win for them.  If they
> don't specify LAZY, the code is not run.

I see a number of possibilities:
1.)	A tested 'feature patch' available for separate download;
2.)	A configure switch '--enable-lazy-vacuum' perhaps;
3.)	The (marginally if at all documented) LAZY parameter, with the code
sitting there dormant until the parameter is passed. Those who need it
probably read this list anyway.

Are we anywhere near comfortable that the code to support LAZY doesn't
impact standard VACUUM in any way?  That for me is the acid test -- if
VACUUM proper gets a bug due to the addition, that would be _bad_.  But,
if someone either applied the feature patch or enabled the configure
switch, well, then that's their choice, and their problem.

Then those who don't need it can still have reasonable confidence in the
solidity of the VACUUM code.  The fact that LAZY will get really lazy
and go away at 7.2 is a factor, as well.  But the fact that 7.2, which
will obviate all this (hopefully), is several months at the very least
down the road makes it desireable NOW to have the LAZY behavior.

I for one don't _need_ the LAZY behavior -- my VACUUMs take seconds, not
hours.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11