Re: base backup client as auxiliary backend process

Peter Eisentraut <peter.eisentraut@2ndquadrant.com>

From: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2019-11-09T21:13:13Z
Lists: pgsql-hackers

Attachments

On 2019-11-07 05:16, Michael Paquier wrote:
>>> The current defaults of pg_basebackup have been thought so as the
>>> backups taken have a good stability and so as monitoring is eased
>>> thanks to --wal-method=stream and the use of replication slots.
>>> Shouldn't the use of a least a temporary replication slot be mandatory
>>> for the stability of the copy?  It seems to me that there is a good
>>> argument for having a second process which streams WAL on top of the
>>> main backup process, and just use a WAL receiver for that.
>> Is this something that the walreceiver should be doing independent of this
>> patch?
> There could be an argument for switching a WAL receiver to use a
> temporary replication slot by default.  Still, it seems to me that
> this backup solution suffers from the same set of problems we have
> spent years to fix with pg_basebackup with missing WAL files caused by
> concurrent checkpoints removing things needed while the copy of the
> main data folder and other tablespaces happens.

I looked into this.  It seems trivial to make walsender create and use a 
temporary replication slot by default if no permanent replication slot 
is specified.  This is basically the logic that pg_basebackup has but 
done server-side.  See attached patch for a demonstration.  Any reason 
not to do that?

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Commits

  1. Factor out InitControlFile() from BootStrapXLOG()

  2. Reformat code comment

  3. pg_resetwal: Rename function to avoid potential conflict

  4. Expose PQbackendPID() through walreceiver API

  5. walreceiver uses a temporary replication slot by default

  6. Make lsn argument of walrcv_create_slot() optional

  7. More precise errors from initial pg_control check