Re: Single transaction in the tablesync worker?

Peter Smith <smithpb2250@gmail.com>

From: Peter Smith <smithpb2250@gmail.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2021-01-14T05:23:23Z
Lists: pgsql-hackers

Attachments

Hi Amit.

PSA the v15 patch for the Tablesync Solution1.

Main differences from v14:
+ Addresses review comment, posted 13/Jan [ak13]

[ak13] = https://www.postgresql.org/message-id/CAA4eK1KzNbudfwmJD-ureYigX6sNyCU6YgHscg29xWoZG6osvA%40mail.gmail.com

====

Features:

* The tablesync slot is now permanent instead of temporary.

* The tablesync slot name is no longer tied to the Subscription slot name.

* The tablesync slot cleanup (drop) code is added for
DropSubscription, AlterSubscription_refresh and for
process_syncing_tables_for_sync functions. Drop/AlterSubscription will
issue WARNING instead of ERROR in case the slot drop fails.

* The tablesync worker is now allowing multiple tx instead of single tx

* A new state (SUBREL_STATE_FINISHEDCOPY) is persisted after a
successful copy_table in tablesync's LogicalRepSyncTableStart.

* If a re-launched tablesync finds state SUBREL_STATE_FINISHEDCOPY
then it will bypass the initial copy_table phase.

* Now tablesync sets up replication origin tracking in
LogicalRepSyncTableStart (similar as done for the apply worker). The
origin is advanced when first created.

* The tablesync replication origin tracking is cleaned up during
DropSubscription and/or process_syncing_tables_for_apply.

* The DropSubscription cleanup code was enhanced (v7+) to take care of
any crashed tablesync workers.

* The AlterSubscription_refresh (v14+) is now more similar to
DropSubscription w.r.t to stopping tablesync workers for any "removed"
tables.

* Updates to PG docs.

TODO / Known Issues:

* The AlterSubscription_refresh tablesync cleanup code still has some
problems [1]
[1] = https://www.postgresql.org/message-id/CAA4eK1JuwZF7FHM%2BEPjWdVh%3DXaz-7Eo-G0TByMjWeUU32Xue3w%40mail.gmail.com

---

Kind Regards,
Peter Smith.
Fujitsu Australia

Commits

  1. Fix relcache reference leak introduced by ce0fdbfe97.

  2. Fix Subscription test added by commit ce0fdbfe97.

  3. Allow multiple xacts during table sync in logical replication.

  4. Logical replication support for initial data copy