Thread

RE: postgres 7.2 features.

Vadim Mikheev <vmikheev@sectorbase.com> — 2000-07-11T01:06:40Z

> Is WAL planned for 7.1? What is the story with WAL?

Yes.

> I'm a bit concerned that the current storage manager is going to be
> thrown in the bit bucket without any thought for its benefits. There's
> some stuff I want to do with it like resurrecting time travel,

Why don't use triggers for time-travel?
Disadvantages of transaction-commit-time based time travel was pointed out
a days ago.

> some database replication stuff which can make use of the non-destructive

It was mentioned here that triggers could be used for async replication,
as well as WAL.

> storage method etc. There's a whole lot of interesting stuff that can be
> done with the current storage manager.

Vadim

Re: postgres 7.2 features.

Chris <chrisb@nimrod.itg.telstra.com.au> — 2000-07-11T01:27:39Z

"Mikheev, Vadim" wrote:
> Yes.
> 
> > I'm a bit concerned that the current storage manager is going to be
> > thrown in the bit bucket without any thought for its benefits. There's
> > some stuff I want to do with it like resurrecting time travel,
> 
> Why don't use triggers for time-travel?
> Disadvantages of transaction-commit-time based time travel was pointed out
> a days ago.

Triggers for time travel is MUCH less efficient. There is no copying
involved
either in memory or on disk with the original postgres time travel, nor
is
there any logic to be executed. Then you've got to figure out strategies
for efficiently deleting old data if you want to. The old postgres was
the 
Right Thing, if you want access to time travel.

> It was mentioned here that triggers could be used for async replication,
> as well as WAL.

Same story. Major inefficency. Replication is tough enough without
mucking
around with triggers. Once the trigger executes you've got to go and
store
the data in the database again anyway. Then figure out when to delete
it.

> > storage method etc. There's a whole lot of interesting stuff that can be
> > done with the current storage manager.
> 
> Vadim

Re: postgres 7.2 features.

Philip Warner <pjw@rhyme.com.au> — 2000-07-11T06:37:13Z

At 11:27 11/07/00 +1000, Chris Bitmead wrote:
>
>> It was mentioned here that triggers could be used for async replication,
>> as well as WAL.
>
>Same story. Major inefficency. Replication is tough enough without
>mucking
>around with triggers. Once the trigger executes you've got to go and
>store
>the data in the database again anyway. Then figure out when to delete
>it.
>

The WAL *should* be the most efficient technique for replication (this said
without actually having seen it ;-}). 



----------------------------------------------------------------
Philip Warner                    |     __---_____
Albatross Consulting Pty. Ltd.   |----/       -  \
(A.C.N. 008 659 498)             |          /(@)   ______---_
Tel: (+61) 0500 83 82 81         |                 _________  \
Fax: (+61) 0500 83 82 82         |                 ___________ |
Http://www.rhyme.com.au          |                /           \|
                                 |    --________--
PGP key available upon request,  |  /
and from pgp5.ai.mit.edu:11371   |/

Re: Storage Manager (was postgres 7.2 features.)

Chris <chrisb@nimrod.itg.telstra.com.au> — 2000-07-11T07:05:42Z

Has sufficient research been done to warrant destruction of what is
currently there?

According to the postgres research papers, the no-overwrite storage
manager has the following attributes...

* It's always faster than WAL in the presence of stable main memory.
(Whether the stable caches in modern disk drives is an approximation I
don't know).

* It's more scalable and has less logging contention. This allows
greater scalablility in the presence of multiple processors.

* Instantaneous crash recovery.

* Time travel is available at no cost.

* Easier to code and prove correctness. (I used to work for a database
company that implemented WAL, and it took them a large number of years
before they supposedly corrected every bug and crash condition on
recovery).

* Ability to keep archival records on an archival medium.

Is there any research on the level of what was done previously to
warrant abandoning these benefits? Obviously WAL has its own benefits, I
just don't want to see the current benefits lost.

Re: Storage Manager (was postgres 7.2 features.)

Jan Wieck <janwieck@t-online.de> — 2000-07-11T10:09:14Z

Chris Bitmead wrote:
>
> Has sufficient research been done to warrant destruction of what is
> currently there?

    What's  currently there doesn't have TT any more. So there is
    nothing we would destroy with an overwriting SMGR.

> According to the postgres research papers, the no-overwrite storage
> manager has the following attributes...

    I started using (and hacking) Postgres in version 4.2,  which
    was  the  last official release from Stonebrakers team at UCB
    (and the last one with the PostQUEL query language).

    The no-overwriting SMGR concept was one of  the  things,  the
    entire  project  should  aprove.  The  idea  was  to  combine
    rollback and logging information with  the  data  itself,  by
    only  storing  new  values  and  remembering  when  something
    appeared or disappeared. Stable memory just means "if I  know
    my write made it to some point, I can read it back later even
    in the case of a crash".

    This has never been implemented to a degree that  is  capable
    to  catch hardware failures like unexpected loss of power. So
    the project finally told "it might be possible".  Many  other
    questions have been answered by the project, but exactly this
    one is still open.

> * It's always faster than WAL in the presence of stable main memory.
> (Whether the stable caches in modern disk drives is an approximation I
> don't know).

    For writing, yes. But for high updated tables, the scans will
    soon slow down due to the junk contention.

> * It's more scalable and has less logging contention. This allows
> greater scalablility in the presence of multiple processors.
>
> * Instantaneous crash recovery.

    Because  this never worked reliable, Vadim is working on WAL.

> * Time travel is available at no cost.
>
> * Easier to code and prove correctness. (I used to work for a database
> company that implemented WAL, and it took them a large number of years
> before they supposedly corrected every bug and crash condition on
> recovery).
>
> * Ability to keep archival records on an archival medium.

    Has this ever been implemented?

> Is there any research on the level of what was done previously to
> warrant abandoning these benefits? Obviously WAL has its own benefits, I
> just don't want to see the current benefits lost.

    I see your points. Maybe we can leave the no-overwriting SMGR
    in the code, and just make the new one the default.


Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #

Re: Storage Manager (was postgres 7.2 features.)

Chris Bitmead <chris@bitmead.com> — 2000-07-11T13:20:28Z

Jan Wieck wrote:

>     What's  currently there doesn't have TT any more. So there is
>     nothing we would destroy with an overwriting SMGR.

I know, but I wanted to resurrect it at some stage, and I think a lot of
important bits are still there.

> > * It's always faster than WAL in the presence of stable main memory.
> > (Whether the stable caches in modern disk drives is an approximation I
> > don't know).
> 
>     For writing, yes. But for high updated tables, the scans will
>     soon slow down due to the junk contention.

I imagine highly updated applications won't be interested in time
travel. If they are then the alternative of a user-maintained time-stamp
and triggers will still leave you with "junk".

> > * Instantaneous crash recovery.
> 
>     Because  this never worked reliable, Vadim is working on WAL.

Postgres recovery is not reliable?