Horton

Re: Synchronizing slots from primary to standby

Amit Kapila <amit.kapila16@gmail.com>

View thread View on postgr.es
From: Amit Kapila <amit.kapila16@gmail.com>
To: "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>
Cc: shveta malik <shveta.malik@gmail.com>, "Drouvot, Bertrand" <bertranddrouvot.pg@gmail.com>, Peter Smith <smithpb2250@gmail.com>, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, Bruce Momjian <bruce@momjian.us>, Ashutosh Sharma <ashu.coek88@gmail.com>, Andres Freund <andres@anarazel.de>, pgsql-hackers <pgsql-hackers@postgresql.org>, Ajin Cherian <itsajin@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>
Date: 2023-11-23T10:45:34Z
Lists: pgsql-hackers 
On Wed, Nov 22, 2023 at 10:02 AM Zhijie Hou (Fujitsu)
<houzj.fnst@fujitsu.com> wrote:
>
> On Tuesday, November 21, 2023 5:33 PM shveta malik <shveta.malik@gmail.com> wrote:
> >
> >
> > v37 fails to apply to HEAD due to a recent commit e83aa9f92fdd, rebased the
> > patches.  PFA v37_2 patches.
>
> Thanks for updating the patches.
>
> I'd like to discuss one issue related to the correct handling of failover flag
> when executing ALTER SUBSCRIPTION SET (slot_name = 'new_slot')".
>
> Since the command intends to use a new slot on the primary, the new slot needs
> to reflect the "failover" state that the subscription currently has. If the
> failoverstate of the Subscription is LOGICALREP_FAILOVER_STATE_ENABLED, then I
> can reset it to LOGICALREP_FAILOVER_STATE_PENDING and allow the apply worker to
> handle it the way it is handled today (just like two_phase handling).
>
> But if the failoverstate is LOGICALREP_FAILOVER_STATE_DISABLED, the original
> idea is to call walrcv_alter_slot and alter the slot from the "ALTER
> SUBSCRIPTION" handling backend itself. This works if the slot is currently
> disabled. But the " ALTER SUBSCRIPTION SET (slot_name = 'new_slot')" command is
> supported even if the subscription is enabled. If the subscription is enabled,
> then calling walrcv_alter_slot() fails because the slot is still acquired by
> apply worker.
>
> So, I am thinking do we need a new mechanism to change the failover flag to
> false on an enabled subscription ? For example, we could call walrcv_alter_slot
> on startup of apply worker if AllTablesyncsReady(), for both true and false
> values of failover flag. This way, every time apply worker is started, it calls
> walrcv_alter_slot to set the failover flag on the primary.
>

I think for the false case, we need to execute walrcv_alter_slot()
every time at the start of apply worker and it doesn't sound like an
ideal way to achieve it.

> Or we could just document that it is user's responsibility to match the failover
> property in case it changes the slot_name.
>

Personally, I think we should document this behavior instead of
complicating the patch and the user anyway has a way to achieve it.

-- 
With Regards,
Amit Kapila.

Commits

  1. Doc: Add the new section "Logical Replication Failover".

  2. Fix the review comments and a bug in the slot sync code.

  3. Fix a test in failover slots regression test.

  4. Fix the intermittent buildfarm failures in 040_standby_failover_slots_sync.

  5. Split XLogCtl->LogwrtResult into separate struct members

  6. Ensure that the sync slots reach a consistent state after promotion without losing data.

  7. Introduce a new GUC 'standby_slot_names'.

  8. Fix BF failure introduced by commit b3f6b14cf4.

  9. Fixups for commit 93db6cbda0.

  10. Fix BF failure in commit 93db6cbda0.

  11. Add a new slot sync worker to synchronize logical slots.

  12. Improve ERROR/LOG messages added by commits ddd5f4f54a and 7a424ece48.

  13. Disable autovacuum on primary in 040_standby_failover_slots_sync test.

  14. Fix the incorrect format specifier used in commit 7a424ece48.

  15. Change the LOG level in 040_standby_failover_slots_sync.pl to DEBUG2.

  16. Add more LOG and DEBUG messages for slot synchronization.

  17. Another try to fix BF failure introduced in commit ddd5f4f54a.

  18. Fix BF introduced in commit ddd5f4f54a.

  19. Add a slot synchronization function.

  20. Clean-ups for 776621a5e4 and 7329240437.

  21. Enhance libpqrcv APIs to support slot synchronization.

  22. Add a failover option to subscriptions.

  23. Allow setting failover property in the replication command.

  24. Allow to enable failover property for replication slots via SQL API.

  25. Add macros for looping through a List without a ListCell.

  26. Track conflict_reason in pg_replication_slots.

  27. Allow upgrades to preserve the full subscription's state.

  28. Add support for incremental backup.

  29. Simplify some logic in CreateReplicationSlot()

  30. Fix uninitialized access to InitialRunningXacts during decoding after ERROR.

  31. Flexible options for CREATE_REPLICATION_SLOT.

  32. Empty search_path in logical replication apply worker and walsender.

  33. Fix oldest xmin and LSN computation across repslots after advancing

  34. Require C99 (and thus MSCV 2013 upwards).