Horton
Menu

Thread

  1. beta5 packages ...

    Marc Fournier <scrappy@hub.org> — 2001-02-23T18:25:55Z

    ... if anyone wants to take a quick gander at it while I wait to announce
its availability ... let me know if therea re any obvious problems iwht it
...


Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org

  2. Re: beta5 packages ...

    Bruce Momjian <pgman@candle.pha.pa.us> — 2001-02-23T20:19:28Z

    > 
> ... if anyone wants to take a quick gander at it while I wait to announce
> its availability ... let me know if therea re any obvious problems iwht it
> ...

I was wondering what open items are left?  Are we ready to start the
release process with a docs freeze?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

  3. Re: beta5 packages ...

    Lamar Owen <lamar.owen@wgcr.org> — 2001-02-23T20:26:40Z

    The Hermit Hacker wrote:
> 
> ... if anyone wants to take a quick gander at it while I wait to announce
> its availability ... let me know if therea re any obvious problems iwht it
> ...

Quick note: it will be Sunday at the earliest before I can build RPM's
of beta5.  If the package release is after Sunday, it will be the
following Sunday, as my day job has its busiest time this coming week
(IOW, I'm going to be swamped -- actually, I am already swamped right
now preparing for next week, but I will be virutally off-line next
week).

It has already been requested that I get the contrib stuff in beta5's
RPMset -- I will attempt to do that, but I'm making no guarantees at
this point.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

  4. Re: beta5 packages ...

    Martin A. Marques <martin@math.unl.edu.ar> — 2001-02-23T20:51:51Z

    Mensaje citado por: Bruce Momjian <pgman@candle.pha.pa.us>:

> > 
> > ... if anyone wants to take a quick gander at it while I wait to
> announce
> > its availability ... let me know if therea re any obvious problems
> iwht it
> > ...
> 
> I was wondering what open items are left?  Are we ready to start the
> release process with a docs freeze?

How did the PHP-4.0.4pl1+Postgres-7.1Beta5 end?
I followed it, but don't remember where it ended. Is it OK for compiling? Is
some hacking needed?

Saludos... :-)


System Administration: It's a dirty job,
but someone told I had to do it.
-----------------------------------------------------------------
Martín Marqués                  email:  martin@math.unl.edu.ar
Santa Fe - Argentina            http://math.unl.edu.ar/~martin/
Administrador de sistemas en math.unl.edu.ar
-----------------------------------------------------------------

  5. Re: beta5 packages ...

    Tom Lane <tgl@sss.pgh.pa.us> — 2001-02-23T20:58:43Z

    Bruce Momjian <pgman@candle.pha.pa.us> writes:
> I was wondering what open items are left?  Are we ready to start the
> release process with a docs freeze?

I need some feedback on my commitdelay proposal first.  If we add a
runtime parameter to control that, it had better be documented.

I have a couple of other bugs outstanding, but nothing in docs.

			regards, tom lane

  6. Re: beta5 packages ...

    Bruce Momjian <pgman@candle.pha.pa.us> — 2001-02-23T21:16:41Z

    > Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > I was wondering what open items are left?  Are we ready to start the
> > release process with a docs freeze?
> 
> I need some feedback on my commitdelay proposal first.  If we add a
> runtime parameter to control that, it had better be documented.

I think we need to give up on the delay for 7.1.X.  I don't see any
good/easy solutions.  Looking at the existing proc bit seems like it
doesn't give us enough information to know if we should wait, and
because there really isn't much time between start commit and fsync(),
my idea is dead.  I think we have to keep it at zero and try again in
7.2.  We may have to get ugly and hack a bit change in the executor when
we are winding up the query.  (yikes!)

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

  7. Re: beta5 packages ...

    Bruce Momjian <pgman@candle.pha.pa.us> — 2001-02-23T21:17:08Z

    > Bruce Momjian writes:
> 
> > I was wondering what open items are left?  Are we ready to start the
> > release process with a docs freeze?
> 
> I still have the JDBC docs to finish and someone was going to send some
> PL/pgSQL stuff, but I guess I'll have to remind him again.  What exactly
> is the goal of a docs freeze?

It is so Thomas can package the docs into various formats.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

  8. Re: beta5 packages ...

    Peter Eisentraut <peter_e@gmx.net> — 2001-02-23T21:24:56Z

    Bruce Momjian writes:

> I was wondering what open items are left?  Are we ready to start the
> release process with a docs freeze?

I still have the JDBC docs to finish and someone was going to send some
PL/pgSQL stuff, but I guess I'll have to remind him again.  What exactly
is the goal of a docs freeze?

-- 
Peter Eisentraut      peter_e@gmx.net       http://yi.org/peter-e/

  9. Re: beta5 packages ...

    Tom Lane <tgl@sss.pgh.pa.us> — 2001-02-23T21:35:10Z

    Bruce Momjian <pgman@candle.pha.pa.us> writes:
> I think we need to give up on the delay for 7.1.X.  I don't see any
> good/easy solutions.

I take it you think my idea is not even worth trying.  Why not?

			regards, tom lane

  10. Re: beta5 packages ...

    Bruce Momjian <pgman@candle.pha.pa.us> — 2001-02-23T21:46:02Z

    > Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > I think we need to give up on the delay for 7.1.X.  I don't see any
> > good/easy solutions.
> 
> I take it you think my idea is not even worth trying.  Why not?

You are suggesting looking at the "I have modified something" bit in
Proc, and using that to trigger the delay, right?

Well, clearly it would help because a single backend would not do any
delay, however, that is the same as doing a zero delay all the time,
which is what we are doing now.

So, the change would have to show that doing the delay when some other
backend has dirtied a buffer is _better_ than doing no delay.

I guess the question is "What is the average time from that bit being
set to the actual commit, and what is its relation to the duration of an
fsync()?"  If the bit set/commit time is small by comparison, it would
be worth using the bit.  However, we have also seen that the delay
itself is usually 10ms, which is pretty long by itself.

Your bit does allow us to _not_ wait if there aren't other backends in
process, which is a good thing.

OK, let's look at the average duration from bit set to commit.  If the
user is in a multi-statement transaction, the delay could be quite long.
If they are doing an UPDATE/DELETE that is doing a sequential scan, that will
be long too.   If they are doing an INSERT, that should be quick, though
INSERT/SELECT could be long.

I guess the 10ms minimum delay time is a problem for me.  The good thing
is that this delay happens _only_ if other backends are actually
running, though if someone is sitting in psql and the are inside a
transaction, that is going to cause a wait too.

Let's keep talking.  I see us so near release, I am not sure if we can
get something that is a clear win, and we saw how the 5us fix almost got
out in the final before we realized the performance problems with it.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

  11. Commit delay (was Re: beta5 packages)

    Tom Lane <tgl@sss.pgh.pa.us> — 2001-02-23T22:05:07Z

    Bruce Momjian <pgman@candle.pha.pa.us> writes:
> So, the change would have to show that doing the delay when some other
> backend has dirtied a buffer is _better_ than doing no delay.

Agreed.  However, we have as yet no data that proves nonzero commit
delay is bad in the presence of multiple active backends.  As Hiroshi
pointed out, all the pgbench results we did last weekend are garbage
(unless they were done with scale factor > 1) because write conflicts on
the single "branch" row would prevent more than one backend from ever
being ready to commit at the same time.  Hiroshi's results suggest that
positive commit delay can be worthwhile when there are nonconflicting
transactions.

Note that with the extension to ignore blocked backends, my proposal
would not count backends waiting on a write conflict, and would
therefore not execute the delay in the scalefactor=1 pgbench case.
So those benchmarks do not prove it would hurt anything to have
commit delay > 0 with my proposal.

> I guess the question is "What is the average time from that bit being
> set to the actual commit,

This is obviously very application-dependent, but we know that pgbench
speeds of 40-200 tr/sec are easily achieved by 7.1 for single backends
with fsync off.  So it's evident that the total transaction time before
commit starts is a small number of milliseconds for transactions of that
complexity.

> and what is its relation to the duration of an fsync()?

fsync is slow, slow, slow, at least on my platform ... I did kernel
traces on pgbench last weekend and saw multiple clock-tick interrupts
during the fsync call.

> I guess the 10ms minimum delay time is a problem for me.

Yeah, the whole thing would be a lot better if we could get a shorter
delay.  But that doesn't mean it's no good at all.

> The good thing
> is that this delay happens _only_ if other backends are actually
> running, though if someone is sitting in psql and the are inside a
> transaction, that is going to cause a wait too.

Hmm.  A further refinement would be to add a waiting-for-client-input
bit to PROC, although if you have a fast-responding client, ignoring
such backends wouldn't necessarily be a good thing.  Notice that the
pgbench transaction involves multiple client requests ...

> Let's keep talking.  I see us so near release, I am not sure if we can
> get something that is a clear win, and we saw how the 5us fix almost got
> out in the final before we realized the performance problems with it.

Yeah, because our attention hadn't been drawn to it.  It won't escape
so easily now ;-).  The real concern here is that I'm not currently
convinced that commit_delay = 0 is a good answer under heavy load.

			regards, tom lane

  12. Re: Commit delay (was Re: beta5 packages)

    Bruce Momjian <pgman@candle.pha.pa.us> — 2001-02-23T22:10:07Z

    > Hmm.  A further refinement would be to add a waiting-for-client-input
> bit to PROC, although if you have a fast-responding client, ignoring
> such backends wouldn't necessarily be a good thing.  Notice that the
> pgbench transaction involves multiple client requests ...
> 
> > Let's keep talking.  I see us so near release, I am not sure if we can
> > get something that is a clear win, and we saw how the 5us fix almost got
> > out in the final before we realized the performance problems with it.
> 
> Yeah, because our attention hadn't been drawn to it.  It won't escape
> so easily now ;-).  The real concern here is that I'm not currently
> convinced that commit_delay = 0 is a good answer under heavy load.

OK, clearly your looking at the bit is better than what we have now, so
how about committing something that looks at the bit, but leave the
default at zero.  Then, let people test zero and non-zero delays and
let's see what they find.  That seems safe because we aren't enabling
the problematic delay by default, at least until we find it is a help in
most cases.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

  13. Re: Commit delay (was Re: beta5 packages)

    Tom Lane <tgl@sss.pgh.pa.us> — 2001-02-23T22:23:16Z

    Bruce Momjian <pgman@candle.pha.pa.us> writes:
> OK, clearly your looking at the bit is better than what we have now, so
> how about committing something that looks at the bit, but leave the
> default at zero.  Then, let people test zero and non-zero delays and
> let's see what they find.  That seems safe because we aren't enabling
> the problematic delay by default, at least until we find it is a help in
> most cases.

What I think I will do is write the code and try some pgbench tests with
scalefactor > 1.  If that looks promising, I'll post or commit the code
and ask people to do more tests.  We can hold off changing the default
delay back to nonzero until we have more data...

			regards, tom lane

  14. Re: beta5 packages ...

    Justin Clift <aa2@bigpond.net.au> — 2001-02-26T00:18:37Z

    Hi,

Is it desirable for me to build Solaris 8 SPARC packages (Solaris .pkg
format) of beta5?

I have experience in doing this.

Regards and best wishes,

Justin Clift
Database Administrator