Horton
Menu

Re: cleanup patches for incremental backup

Robert Haas <robertmhaas@gmail.com>

View thread
From: Robert Haas <robertmhaas@gmail.com>
To: Matthias van de Meent <boekewurm+postgres@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2024-01-16T20:49:20Z
Lists: pgsql-hackers 
On Tue, Jan 16, 2024 at 3:22 PM Matthias van de Meent
<boekewurm+postgres@gmail.com> wrote:
> > One other thought is that the incremental backup only replaces
> > relation files with incremental files, and it never does anything
> > about FSM files. So the statement that it only contains data that was
> > potentially changed isn't quite correct. It might be better to phrase
> > it the other way around i.e. it is like a full backup, except that
> > some files can be replaced by incremental files which omit blocks to
> > which no WAL-logged changes have been made.
>
> How about the attached?

I like the direction.

+     A special <glossterm linkend="glossary-basebackup">base backup</glossterm>
+     that for some WAL-logged relations only contains the pages that were
+     modified since a previous backup, as opposed to the full relation data of
+     normal base backups. Like base backups, it is generated by the tool
+     <xref linkend="app-pgbasebackup"/>.

Could we say "that for some files may contain only those pages that
were modified since a previous backup, as opposed to the full contents
of every file"? My thoughts are (1) there's no hard guarantee that an
incremental backup will replace even one file with an incremental
file, although in practice it is probably almost always going to
happen and (2) pg_combinebackup would actually be totally fine with
any file at all being sent incrementally; it's only that the server
isn't smart enough to figure out how to do this with e.g. SLRU data
right now.

+     To restore incremental backups the tool <xref
linkend="app-pgcombinebackup"/>
+     is used, which combines the incremental backups with a base backup and
+     <glossterm linkend="glossary-wal">WAL</glossterm> to restore a
+     <glossterm linkend="glossary-db-cluster">database cluster</glossterm> to
+     a consistent state.

I wondered if this needed to be clearer that the chain of backups
could have length > 2. But on further reflection, I think it's fine,
unless you feel otherwise.

The rest LGTM.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Avoid overflow in MaybeRemoveOldWalSummaries().

  2. Revert "Temporary patch to help debug pg_walsummary test failures."

  3. Revise pg_walsummary's 002_blocks test to avoid spurious failures.

  4. Temporary patch to help debug pg_walsummary test failures.

  5. More documentation updates for incremental backup.

  6. Fix typos.

  7. Fix typo.

  8. Try to fix pg_walsummary buildfarm failures.

  9. Repair various defects in dc212340058b4e7ecfc5a7a81ec50e7a207bf288.

  10. Add new pg_walsummary tool.

  11. Add new function pg_get_wal_summarizer_state().

  12. Document WAL summarization information functions.