Horton

[Patch] pg_rewind: options to use restore_command from recovery.conf or command line

Alexey Kondratov <a.kondratov@postgrespro.ru>

View thread
From: Alexey Kondratov <a.kondratov@postgrespro.ru>
To: pgsql-hackers@postgresql.org
Date: 2018-10-19T17:49:34Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Move frontend-side archive APIs from src/common/ to src/fe_utils/

  2. Add -c/--restore-target-wal to pg_rewind

  3. Move routine definitions of xlogarchive.c to a new header file

  4. Move routine building restore_command to src/common/

  5. Integrate recovery.conf into postgresql.conf

Attachments

Hi hackers,

Currently Postgres has options for continuous WAL files archiving, which 
is quite often used along with master-replica setup. OK, then the worst 
is happened and it's time to get your old master back and synchronize it 
with new master (ex-replica) with pg_rewind. However, required WAL files 
may be already archived and pg_rewind will fail. You can copy these 
files manually, but it is difficult to calculate, which ones you need. 
Anyway, it complicates building failover system with automatic failure 
recovery.

I expect, that it will be a good idea to allow pg_rewind to look for a 
restore_command in the target data directory recovery.conf or pass it is 
as a command line argument. Then pg_rewind can use it to get missing WAL 
files from the archive. I had a few talks with DBAs and came to 
conclusion, that this is a highly requested feature.

I prepared a proof of concept patch (please, find attached), which does 
exactly what I described above. I played with it a little and it seems 
to be working, tests were accordingly updated to verify this archive 
retrieval functionality too.

Patch is relatively simple excepting the one part: if we want to parse 
recovery.conf (with all possible includes, etc.) and get 
restore_command, then we should use guc-file.l parser, which is heavily 
linked to backend, e.g. in error reporting part. So I copied it and made 
frontend-safe version guc-file-fe.l. Personally, I don't think it's a 
good idea, but nothing else came to mind. It is also possible to leave 
the only one option -- passing restore_command as command line argument.

What do you think?


--

Alexey Kondratov

Postgres Professional: https://www.postgrespro.com

Russian Postgres Company