Re: Single transaction in the tablesync worker?
Peter Smith <smithpb2250@gmail.com>
From: Peter Smith <smithpb2250@gmail.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Craig Ringer <craig.ringer@enterprisedb.com>, Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>, Ashutosh Bapat <ashutosh.bapat@enterprisedb.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Petr Jelinek <petr.jelinek@enterprisedb.com>,
Petr Jelinek <petr@2ndquadrant.com>, Peter Eisentraut <peter.eisentraut@2ndquadrant.com>,
Ajin Cherian <itsajin@gmail.com>
Date: 2020-12-18T13:11:03Z
Lists: pgsql-hackers
Attachments
- v4-0002-WIP-patch-for-Solution1.patch (application/octet-stream) patch v4-0002
- v4-0001-2PC-change-tablesync-slot-to-use-same-two_phase-m.patch (application/octet-stream) patch v4-0001
Hi Amit. PSA my v4 WIP patch for the Solution1. This patch applies onto the v30 patch set [1] from other 2PC thread: [1] https://www.postgresql.org/message-id/CAFPTHDYA8yE6tEmQ2USYS68kNt%2BkM%3DSwKgj%3Djy4AvFD5e9-UTQ%40mail.gmail.com Although incomplete it does still pass all the make check, and src/test/subscription TAP tests. ==== Coded / WIP: * tablesync slot is now permanent instead of temporary * the tablesync slot cleanup (drop) code is added for DropSubscription and for finish_sync_worker functions * tablesync worked now allowing multiple tx instead of single tx * a new state (SUBREL_STATE_COPYDONE) is persisted after a successful copy_table in LogicalRepSyncTableStart. * if a relaunched tablesync finds the state is SUBREL_STATE_COPYDONE then it will bypass the initial copy_table phase. * tablesync now sets up replication origin tracking in LogicalRepSyncTableStart (similar as done for apply worker) * tablesync replication origin tracking is cleaned up during DropSubscription and/or process_syncing_tables_for_apply TODO / Known Issues: * the current implementation of tablesync drop slot (e.g. from DropSubscription or finish_sync_worker) regenerates the tablesync slot name so it knows what slot to drop. The current code might be ok for normal use cases, but if there is an ALTER SUBSCRIPTION ... SET (slot_name = newname) it would fail to be able to find the tablesync slot. * I think if there are crashed tablesync workers then they are not known to DropSubscription. So this might be a problem to cleanup slots and/or origin tracking belonging to those unknown workers. * help / comments / cleanup * There is temporary "!!>>" excessive logging of mine scattered around which I added to help my testing during development --- Kind Regards, Peter Smith. Fujitsu Australia
Commits
-
Fix relcache reference leak introduced by ce0fdbfe97.
- 8a4f9522d0c7 14.0 landed
-
Fix Subscription test added by commit ce0fdbfe97.
- c8b21b037814 14.0 landed
-
Allow multiple xacts during table sync in logical replication.
- ce0fdbfe9722 14.0 landed
-
Logical replication support for initial data copy
- 7c4f52409a8c 10.0 cited