Horton

Re: Refactor pg_rewind code and make it work against a standby

Heikki Linnakangas <hlinnaka@iki.fi>

View thread View on postgr.es
From: Heikki Linnakangas <hlinnaka@iki.fi>
To: Soumyadeep Chakraborty <soumyadeep2007@gmail.com>
Cc: Kyotaro Horiguchi <horikyota.ntt@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2020-11-12T12:58:02Z
Lists: pgsql-hackers

Attachments

On 04/11/2020 11:23, Heikki Linnakangas wrote:
> I read through the patches one more time, fixed a bunch of typos and
> such, and pushed patches 1-4. I'm going to spend some more time on
> testing the last patch. It allows using a standby server as the source,
> and we don't have any tests for that yet. Thanks for the review!

Did some more testing, fixed one bug, and pushed.

To test this, I set up a cluster with one primary, a standby, and a 
cascaded standby. I launched a test workload against the primary that 
creates tables, inserts rows, and drops tables continuously. In another 
shell, I promoted the cascaded standby, run some updates on the promoted 
server, and finally, run pg_rewind pointed at the standby, and start it 
again as a cascaded standby. Repeat.

Attached are the scripts I used. I edited them between test runs to test 
slightly different scenarios. I don't expect them to be very useful to 
anyone else, but the Internet is my backup.

I did find one bug in the patch with that, so the time was well spent: 
the code in process_queued_fetch_requests() got confused and errored 
out, if a file was removed in the source system while pg_rewind was 
running. There was code to deal with that, but it was broken. Fixed that.

- Heikki

Commits

  1. Make pg_rewind test case more stable.

  2. Fix timing issue in pg_rewind test.

  3. Allow pg_rewind to use a standby server as the source system.

  4. pg_rewind: Refactor the abstraction to fetch from local/libpq source.

  5. pg_rewind: Replace the hybrid list+array data structure with simplehash.

  6. Refactor pg_rewind for more clear decision making.

  7. pg_rewind: Move syncTargetDirectory() to file_ops.c