Re: Streaming a base backup from master

Bruce Momjian <bruce@momjian.us>

From: Bruce Momjian <bruce@momjian.us>
To: Greg Stark <gsstark@mit.edu>
Cc: Martijn van Oosterhout <kleptog@svana.org>, Stephen Frost <sfrost@snowman.net>, Robert Haas <robertmhaas@gmail.com>, Dave Page <dpage@pgadmin.org>, Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>, Magnus Hagander <magnus@hagander.net>
Date: 2010-09-08T01:28:39Z
Lists: pgsql-hackers
Greg Stark wrote:
> The industry standard solution that we're missing that we *should* be
> figuring out how to implement is incremental backups.
> 
> I've actually been thinking about this recently and I think we could
> do it fairly easily with our existing infrastructure. I was planning
> on doing it as an external utility but it would be tempting to be able
> to request an external backup via the streaming protocol so maybe it
> would be better a bit more integrated.
> 
> The way I see it there are two alternatives. You need to start by
> figuring out which blocks have been modified since the last backup (or
> selected reference point). You can do this either by scanning every
> data file and picking every block with an LSN > the reference LSN. Or
> you can do it by scanning the WAL since that point and accumulating a
> list of block numbers.

That's what pgrman does already:

	http://code.google.com/p/pg-rman/

Are you saying you want to do that over the libpq connection?

-- 
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + It's impossible for everything to be true. +