Re: Logical Replication of sequences
Peter Smith <smithpb2250@gmail.com>
Commits
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Doc: Add documentation for sequence synchronization.
- 55cefadde874 19 (unreleased) landed
-
Remove unused assignment in CREATE PUBLICATION grammar.
- bfb7419b0bbe 19 (unreleased) landed
-
Add seq_sync_error_count to subscription statistics.
- f6a4c498dcf6 19 (unreleased) landed
-
Fix few issues in commit 5509055d69.
- 5a4eba558aa7 19 (unreleased) landed
-
Add sequence synchronization for logical replication.
- 5509055d6956 19 (unreleased) landed
-
Add worker type argument to logical replication worker functions.
- 3e8e05596a02 19 (unreleased) landed
-
Introduce "REFRESH SEQUENCES" for subscriptions.
- f0b3573c3aac 19 (unreleased) landed
-
Refactor logical worker synchronization code into a separate file.
- 41c674d2e31e 19 (unreleased) landed
-
Standardize use of REFRESH PUBLICATION in code and messages.
- 2436b8c047ff 19 (unreleased) landed
-
Add "ALL SEQUENCES" support to publications.
- 96b378497346 19 (unreleased) landed
-
Expose sequence page LSN via pg_get_sequence_data.
- b93172ca59f4 19 (unreleased) landed
-
Resume conflict-relevant data retention automatically.
- 0d48d393d465 19 (unreleased) cited
-
Post-commit review fixes for 228c370868.
- 1f7e9ba3ac4e 19 (unreleased) cited
-
Generate GUC tables from .dat file
- 63599896545c 19 (unreleased) cited
Hi Vignesh, This is mostly a repeat of my previous mail from a while ago [1] but includes some corrections, answers, and more examples. I'm going to try to persuade one last time because the current patch is becoming stable, so I wanted to revisit this syntax proposal before it gets too late to change anything. If there is some problem with the proposed idea please let me know because I can see only the advantages and no disadvantages of doing it this way. ~~~ The current patchset offers two forms of subscription refresh: 1. ALTER SUBSCRIPTION name REFRESH PUBLICATION [ WITH ( refresh_option [= value] [, ... ] ) ] 2. ALTER SUBSCRIPTION name REFRESH PUBLICATION SEQUENCES Since 'copy_data' is the only supported refresh_option, really it is more like: 1. ALTER SUBSCRIPTION name REFRESH PUBLICATION [ WITH ( copy_data [= true|false] ) ] 2. ALTER SUBSCRIPTION name REFRESH PUBLICATION SEQUENCES ~~~ I proposed previously that instead of having 2 commands for refreshing subscriptions we should have a single refresh command: ALTER SUBSCRIPTION name REFRESH PUBLICATION [TABLES|SEQUENCES] [ WITH ( copy_data [= true|false] ) ] Why? - IMO it is less confusing than having 2 commands that both refresh sequences in slightly different ways - It is more flexible because apart from refreshing everything, a user can choose to refresh only tables or only sequences if desired; IMO more flexibility is always good. - There is no loss of functionality from the current implementation AFAICT. You can still say "ALTER SUBSCRIPTION sub REFRESH PUBLICATION SEQUENCES" exactly the same as the patchset allows. - The implementation code will become simpler. For example, the current implementation of AlterSubscription_refresh(...) includes the (hacky?) 'resync_all_sequences' parameter and has an overcomplicated relationship with other parameters as demonstrated by the assertions below. IMO using the prosed syntax means this coding will become not only simpler, but shorter too. + /* resync_all_sequences cannot be specified with refresh_tables */ + Assert(!(resync_all_sequences && refresh_tables)); + + /* resync_all_sequences cannot be specified with copy_data as false */ + Assert(!(resync_all_sequences && !copy_data)); ~~~ So, to continue this proposal, let the meaning of 'copy_data' for SEQUENCES be as follows: - when copy_data == false: it means don't copy data (i.e. don't synchronize anything). Add/remove sequences from pg_subscriber_rel as needed. - when copy_data == true: it means to copy data (i.e. synchronize) for all sequences. Add/remove sequences from pg_subscriber_rel as needed) ~~~ EXAMPLES using the proposed syntax: Refreshing TABLES only... ex1. ALTER SUBSCRIPTION sub REFRESH PUBLICATION TABLES WITH (copy_data = false) - same as PG17 functionality for "ALTER SUBSCRIPTION sub REFRESH PUBLICATION WITH (copy_data = false)" ex2. ALTER SUBSCRIPTION sub REFRESH PUBLICATION TABLES WITH (copy_data = true) - same as PG17 functionality for "ALTER SUBSCRIPTION sub REFRESH PUBLICATION WITH (copy_data = true)" ex3. (using default copy_data) ALTER SUBSCRIPTION sub REFRESH PUBLICATION TABLES - same as ex2. ~ Refreshing SEQUENCES only... ex4. ALTER SUBSCRIPTION sub REFRESH PUBLICATION SEQUENCES WITH (copy data = false) - this adds/removes only sequences to pg_subscription_rel but doesn't update the sequence values ex5. ALTER SUBSCRIPTION sub REFRESH PUBLICATION SEQUENCES WITH (copy data = true) - this adds/removes only sequences to pg_subscription_rel and also updates (synchronizes) all sequence values. - same functionality as "ALTER SUBSCRIPTION sub REFRESH PUBLICATION SEQUENCES" in your current patchset ex6. (using default copy_data) ALTER SUBSCRIPTION sub REFRESH PUBLICATION SEQUENCES - same as ex5. - note, that this command has the same syntax and functionality as the current patchset ~~~ When no object_type is specified it has intuitive meaning to refresh both TABLES and SEQUENCES... ex7. ALTER SUBSCRIPTION sub REFRESH PUBLICATION WITH (copy_data = false) - For tables, it is the same as the PG17 functionality - For sequences it includes the same behaviour of ex4. ex8. ALTER SUBSCRIPTION sub REFRESH PUBLICATION WITH (copy_data = true) - For tables, it is the same as the PG17 functionality - For sequences it includes the same behaviour of ex5. - There is one subtle difference from the current patchset because this proposal will synchronize *all* sequences instead of only new ones. But, this is a good thing. The current documentation is complicated by having to explain the differences between REFRESH PUBLICATION and REFRESH PUBLICATION SEQUENCES. The current patchset also raises questions like how the user chooses whether to use "REFRESH PUBLICATION SEQUENCES" versus "REFRESH PUBLICATION WITH (copy_data=true)". OTHO, the proposed syntax eliminates ambiguity. ex9. (using default copy_data) ALTER SUBSCRIPTION sub REFRESH PUBLICATION - same as ex8 ====== [1] https://www.postgresql.org/message-id/CAHut%2BPuFH1OCj-P1UKoRQE2X4-0zMG%2BN1V7jdn%3DtOQV4RNbAbw%40mail.gmail.com Kind Regards, Peter Smith. Fujitsu Australia