Horton

RE: Synchronizing slots from primary to standby

Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>

View thread View on postgr.es
From: "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>
To: 'shveta malik' <shveta.malik@gmail.com>
Cc: "Drouvot, Bertrand" <bertranddrouvot.pg@gmail.com>, Amit Kapila <amit.kapila16@gmail.com>, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, Bruce Momjian <bruce@momjian.us>, Ashutosh Sharma <ashu.coek88@gmail.com>, Andres Freund <andres@anarazel.de>, pgsql-hackers <pgsql-hackers@postgresql.org>, Ajin Cherian <itsajin@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>
Date: 2023-09-08T11:10:21Z
Lists: pgsql-hackers 
Dear Shveta,

I resumed to check the thread. Here are my high-level comments.
Sorry if you have been already discussed.

01. General

I think the documentation can be added, not only GUCs. How about adding examples
for combinations of physical and logical replications?  You can say that both of
physical primary can be publisher and slots on primary/standby are synchronized.

02. General

standby_slot_names ensures that physical standby is always ahead subscriber, but I
think it may be not sufficient. There is a possibility that primary server does
not have any physical slots. In this case the physical standby may be behind the
subscriber and the system may be confused when the failover is occured. Can't
we specify the name of standby via application_name or something?


03. General

In this architecture, the syncslot worker is launched per db and they
independently connects to primary, right? I'm not sure it is efficient, but I
come up with another architecture - only a worker (syncslot receiver)connects
to the primary and other workers (syncslot worker) receives infos from it and
updates. This can reduce the number of connections so that it may slightly
improve the latency of network. How do you think?

04. General

test file recovery/t/051_slot_sync.pl is missing.

04. ReplSlotSyncMain

Does the worker have to connect to the specific database?


```
	/* Connect to our database. */
	BackgroundWorkerInitializeConnectionByOid(MySlotSyncWorker->dbid,
											  MySlotSyncWorker->userid,
											  0);
```

05. SlotSyncInitSlotNamesLst()

"Lst" should be "List".

Best Regards,
Hayato Kuroda
FUJITSU LIMITED

Commits

  1. Doc: Add the new section "Logical Replication Failover".

  2. Fix the review comments and a bug in the slot sync code.

  3. Fix a test in failover slots regression test.

  4. Fix the intermittent buildfarm failures in 040_standby_failover_slots_sync.

  5. Split XLogCtl->LogwrtResult into separate struct members

  6. Ensure that the sync slots reach a consistent state after promotion without losing data.

  7. Introduce a new GUC 'standby_slot_names'.

  8. Fix BF failure introduced by commit b3f6b14cf4.

  9. Fixups for commit 93db6cbda0.

  10. Fix BF failure in commit 93db6cbda0.

  11. Add a new slot sync worker to synchronize logical slots.

  12. Improve ERROR/LOG messages added by commits ddd5f4f54a and 7a424ece48.

  13. Disable autovacuum on primary in 040_standby_failover_slots_sync test.

  14. Fix the incorrect format specifier used in commit 7a424ece48.

  15. Change the LOG level in 040_standby_failover_slots_sync.pl to DEBUG2.

  16. Add more LOG and DEBUG messages for slot synchronization.

  17. Another try to fix BF failure introduced in commit ddd5f4f54a.

  18. Fix BF introduced in commit ddd5f4f54a.

  19. Add a slot synchronization function.

  20. Clean-ups for 776621a5e4 and 7329240437.

  21. Enhance libpqrcv APIs to support slot synchronization.

  22. Add a failover option to subscriptions.

  23. Allow setting failover property in the replication command.

  24. Allow to enable failover property for replication slots via SQL API.

  25. Add macros for looping through a List without a ListCell.

  26. Track conflict_reason in pg_replication_slots.

  27. Allow upgrades to preserve the full subscription's state.

  28. Add support for incremental backup.

  29. Simplify some logic in CreateReplicationSlot()

  30. Fix uninitialized access to InitialRunningXacts during decoding after ERROR.

  31. Flexible options for CREATE_REPLICATION_SLOT.

  32. Empty search_path in logical replication apply worker and walsender.

  33. Fix oldest xmin and LSN computation across repslots after advancing

  34. Require C99 (and thus MSCV 2013 upwards).