Horton

Thread

  1. [v9.2] Start new timeline for PITR

    David Fetter <david@fetter.org> — 2011-06-09T23:51:45Z

    Folks,

The nice people at VMware, where I work, have come up with a small
patch to allow PITR to create a new timeline.  This is useful in cases
where you're using filesystem snapshots of $PGDATA which may be old.

PFA a patch implementing and documenting same :)

Cheers,
David.
-- 
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david.fetter@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

  2. Re: [v9.2] Start new timeline for PITR

    Josh Berkus <josh@agliodbs.com> — 2011-06-10T00:02:48Z

    On 6/9/11 4:51 PM, David Fetter wrote:
> Folks,
> 
> The nice people at VMware, where I work, have come up with a small
> patch to allow PITR to create a new timeline.  This is useful in cases
> where you're using filesystem snapshots of $PGDATA which may be old.
> 
> PFA a patch implementing and documenting same :)

Can you explain here in email what the specific goals and expected
behavior of the option are?

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

  3. Re: [v9.2] Start new timeline for PITR

    Tom Lane <tgl@sss.pgh.pa.us> — 2011-06-10T00:59:04Z

    David Fetter <david@fetter.org> writes:
> The nice people at VMware, where I work, have come up with a small
> patch to allow PITR to create a new timeline.  This is useful in cases
> where you're using filesystem snapshots of $PGDATA which may be old.

Huh?  We already start a new timeline when doing a non-crash-recovery
replay scenario.

The code looks pretty confused too, which makes it difficult to
reverse-engineer what your point is.

			regards, tom lane

  4. Re: [v9.2] Start new timeline for PITR

    Robert Haas <robertmhaas@gmail.com> — 2011-06-10T05:20:25Z

    On Thu, Jun 9, 2011 at 8:59 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> David Fetter <david@fetter.org> writes:
>> The nice people at VMware, where I work, have come up with a small
>> patch to allow PITR to create a new timeline.  This is useful in cases
>> where you're using filesystem snapshots of $PGDATA which may be old.
>
> Huh?  We already start a new timeline when doing a non-crash-recovery
> replay scenario.
>
> The code looks pretty confused too, which makes it difficult to
> reverse-engineer what your point is.

I am guessing that they are taking a filesystem snapshot, and then
using that to fire up PG.  So to PG it looks like a crash recovery,
but they want a new timeline anyway.

<waves hands>

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

  5. Re: [v9.2] Start new timeline for PITR

    David Fetter <david@fetter.org> — 2011-06-10T16:30:29Z

    On Fri, Jun 10, 2011 at 01:20:25AM -0400, Robert Haas wrote:
> On Thu, Jun 9, 2011 at 8:59 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > David Fetter <david@fetter.org> writes:
> >> The nice people at VMware, where I work, have come up with a small
> >> patch to allow PITR to create a new timeline.  This is useful in cases
> >> where you're using filesystem snapshots of $PGDATA which may be old.
> >
> > Huh?  We already start a new timeline when doing a non-crash-recovery
> > replay scenario.
> >
> > The code looks pretty confused too, which makes it difficult to
> > reverse-engineer what your point is.
> 
> I am guessing that they are taking a filesystem snapshot, and then
> using that to fire up PG.  So to PG it looks like a crash recovery,
> but they want a new timeline anyway.
> 
> <waves hands>

That's pretty much it.  More detail:

Let's imagine we're taking filesystem snapshots each day by whatever
means.  We're also archiving xlogs, but only have space for 48 hours'
worth.  Now we want to recover to 3 days ago, but there are no WALs
from that time, so we do a crash recovery from the filesystem
snapshot.  Doing continuous archiving from this conflicts with the
existing WALs, which we solve by creating a new timeline.

This also allows subsequent PITR to other times on the original
timeline.

Josh B pointed out that since this option to true conflicts with
another option, having both should prevent recovery from even
starting, and I'll work up a patch for this tonight or at latest
tomorrow.

Cheers,
David.
-- 
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david.fetter@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

  6. Re: [v9.2] Start new timeline for PITR

    Josh Berkus <josh@agliodbs.com> — 2011-06-10T18:53:49Z

    David,

> Let's imagine we're taking filesystem snapshots each day by whatever
> means.  We're also archiving xlogs, but only have space for 48 hours'
> worth.  Now we want to recover to 3 days ago, but there are no WALs
> from that time, so we do a crash recovery from the filesystem
> snapshot.  Doing continuous archiving from this conflicts with the
> existing WALs, which we solve by creating a new timeline.

How is this different from just changing the recovery_command?

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

  7. Re: [v9.2] Start new timeline for PITR

    Josh Berkus <josh@agliodbs.com> — 2011-06-10T18:57:38Z

    David,

> Let's imagine we're taking filesystem snapshots each day by whatever
> means.  We're also archiving xlogs, but only have space for 48 hours'
> worth.  Now we want to recover to 3 days ago, but there are no WALs
> from that time, so we do a crash recovery from the filesystem
> snapshot.  Doing continuous archiving from this conflicts with the
> existing WALs, which we solve by creating a new timeline.

How is this different from just changing the recovery_command?

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

  8. Re: [v9.2] Start new timeline for PITR

    Robert Haas <robertmhaas@gmail.com> — 2011-06-10T19:34:23Z

    On Fri, Jun 10, 2011 at 2:53 PM, Josh Berkus <josh@agliodbs.com> wrote:
>> Let's imagine we're taking filesystem snapshots each day by whatever
>> means.  We're also archiving xlogs, but only have space for 48 hours'
>> worth.  Now we want to recover to 3 days ago, but there are no WALs
>> from that time, so we do a crash recovery from the filesystem
>> snapshot.  Doing continuous archiving from this conflicts with the
>> existing WALs, which we solve by creating a new timeline.
>
> How is this different from just changing the recovery_command?

*scratches head*

How is it the same?

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

  9. Re: [v9.2] Start new timeline for PITR

    Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> — 2011-06-10T19:38:37Z

    On 10.06.2011 22:34, Robert Haas wrote:
> On Fri, Jun 10, 2011 at 2:53 PM, Josh Berkus<josh@agliodbs.com>  wrote:
>>> Let's imagine we're taking filesystem snapshots each day by whatever
>>> means.  We're also archiving xlogs, but only have space for 48 hours'
>>> worth.  Now we want to recover to 3 days ago, but there are no WALs
>>> from that time, so we do a crash recovery from the filesystem
>>> snapshot.  Doing continuous archiving from this conflicts with the
>>> existing WALs, which we solve by creating a new timeline.
>>
>> How is this different from just changing the recovery_command?
>
> *scratches head*
>
> How is it the same?

Creating a dummy recovery.conf with bogus recovery_command would do the 
trick.

-- 
   Heikki Linnakangas
   EnterpriseDB   http://www.enterprisedb.com

  10. Re: [v9.2] Start new timeline for PITR

    Josh Berkus <josh@agliodbs.com> — 2011-06-10T19:39:03Z

    On 6/10/11 12:34 PM, Robert Haas wrote:
> On Fri, Jun 10, 2011 at 2:53 PM, Josh Berkus <josh@agliodbs.com> wrote:
>>> Let's imagine we're taking filesystem snapshots each day by whatever
>>> means.  We're also archiving xlogs, but only have space for 48 hours'
>>> worth.  Now we want to recover to 3 days ago, but there are no WALs
>>> from that time, so we do a crash recovery from the filesystem
>>> snapshot.  Doing continuous archiving from this conflicts with the
>>> existing WALs, which we solve by creating a new timeline.
>>
>> How is this different from just changing the recovery_command?
> 
> *scratches head*
> 
> How is it the same?

Well, presumably I can just change recovery_command to recover from an
empty directory.  Then the PITR copy will just come up as soon as it
finishes processing local snapshot WAL, and it'll start its own
timeline.  How is this different from what the patch does?

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

  11. Re: [v9.2] Start new timeline for PITR

    Jaime Casanova <jaime@2ndquadrant.com> — 2011-06-17T14:57:13Z

    On Fri, Jun 10, 2011 at 11:30 AM, David Fetter <david@fetter.org> wrote:
>
> This also allows subsequent PITR to other times on the original
> timeline.
>
> Josh B pointed out that since this option to true conflicts with
> another option, having both should prevent recovery from even
> starting, and I'll work up a patch for this tonight or at latest
> tomorrow.
>

Hi,

Are you still working on this? should we expect a new patch?

-- 
Jaime Casanova         www.2ndQuadrant.com
Professional PostgreSQL: Soporte 24x7 y capacitación

  12. Re: [v9.2] Start new timeline for PITR

    David Fetter <david@fetter.org> — 2011-06-17T18:54:35Z

    On Fri, Jun 17, 2011 at 09:57:13AM -0500, Jaime Casanova wrote:
> On Fri, Jun 10, 2011 at 11:30 AM, David Fetter <david@fetter.org> wrote:
> >
> > This also allows subsequent PITR to other times on the original
> > timeline.
> >
> > Josh B pointed out that since this option to true conflicts with
> > another option, having both should prevent recovery from even
> > starting, and I'll work up a patch for this tonight or at latest
> > tomorrow.
> 
> Hi,
> 
> Are you still working on this? should we expect a new patch?

Yes, sorry about that.  I let work get on top of me.  Will try for a
new patch this evening.

Cheers,
David.
-- 
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david.fetter@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

  13. Re: [v9.2] Start new timeline for PITR

    Jaime Casanova <jaime@2ndquadrant.com> — 2011-06-18T00:21:54Z

    On Fri, Jun 17, 2011 at 1:54 PM, David Fetter <david@fetter.org> wrote:
> On Fri, Jun 17, 2011 at 09:57:13AM -0500, Jaime Casanova wrote:
>>
>> Are you still working on this? should we expect a new patch?
>
> Yes, sorry about that.  I let work get on top of me.  Will try for a
> new patch this evening.
>

ok... i will wait it to review... just in advance, i really don't like
this name "create_new_timeline"... it will drive confusion

-- 
Jaime Casanova         www.2ndQuadrant.com
Professional PostgreSQL: Soporte 24x7 y capacitación