RE: Handle infinite recursion in logical replication setup

Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>

From: "kuroda.hayato@fujitsu.com" <kuroda.hayato@fujitsu.com>
To: 'vignesh C' <vignesh21@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2022-03-01T10:42:27Z
Lists: pgsql-hackers
Hi Vignesh,

> In logical replication, currently Walsender sends the data that is
> generated locally and the data that are replicated from other
> instances. This results in infinite recursion in circular logical
> replication setup.

Thank you for good explanation. I understand that this fix can be used
for a bidirectional replication.

> Here there are two problems for the user: a) incremental
> synchronization of table sending both local data and replicated data
> by walsender b) Table synchronization of table using copy command
> sending both local data and replicated data

So you wanted to solve these two problem and currently focused on
the first one, right? We can check one by one.

> For the first problem "Incremental synchronization of table by
> Walsender" can be solved by:
> Currently the locally generated data does not have replication origin
> associated and the data that has originated from another instance will
> have a replication origin associated. We could use this information to
> differentiate locally generated data and replicated data and send only
> the locally generated data. This "only_local" could be provided as an
> option while subscription is created:
> ex: CREATE SUBSCRIPTION sub1 CONNECTION 'dbname =postgres port=5433'
> PUBLICATION pub1 with (only_local = on);

Sounds good, but I cannot distinguish whether the assumption will keep.

I played with your patch, but it could not be applied to current master.
I tested from bd74c40 and I confirmed infinite loop was not appeared.

local_only could not be set from ALTER SUBSCRIPTION command.
Is it expected?

Best Regards,
Hayato Kuroda
FUJITSU LIMITED

Commits

  1. Improved CREATE SUBSCRIPTION message for clarity

  2. Fix the test case introduced by commit 8756930190.

  3. Raise a warning if there is a possibility of data from multiple origins.

  4. Add wait_for_subscription_sync for TAP tests.

  5. Bump catversion for commit d8cd0c6c95c0120168df93aae095df4e0682a08a.

  6. Allow users to skip logical replication of data having origin.

  7. Improve two comments related to a boolean DefElem's value

  8. doc: Fix man page whitespace issues