Horton

Re: Conflict detection for update_deleted in logical replication

Amit Kapila <amit.kapila16@gmail.com>

View thread View on postgr.es
From: Amit Kapila <amit.kapila16@gmail.com>
To: "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>
Cc: shveta malik <shveta.malik@gmail.com>, vignesh C <vignesh21@gmail.com>, Masahiko Sawada <sawada.mshk@gmail.com>, Nisha Moond <nisha.moond412@gmail.com>, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2025-06-19T09:32:20Z
Lists: pgsql-hackers 
On Tue, Jun 17, 2025 at 4:26 PM Zhijie Hou (Fujitsu)
<houzj.fnst@fujitsu.com> wrote:
>
> On Mon, Jun 16, 2025 at 7:37 PM Amit Kapila wrote:
> >
> >
> > 3. Isn't the new check for logical slots in
> > check_new_cluster_subscription_configuration() somewhat redundant with
> > the previous check done in check_new_cluster_logical_replication_slots()?
> > Can't we combine both?
>
> Merged as suggested.
>

Okay, it is better to update the comments atop
check_new_cluster_replication_slots() to reflect the new functionality
as well.

After the upgrade, there will be a window where the launcher could
take the time to create the conflict_slot if there exists any
subscription that has retain_conflict_info enabled, and during that
window, the update_delete conflicts won't be detected reliably. To
close that, we can probably create the conflict_slot during upgrade,
if required.

Now, because we don't copy commit_ts during upgrade, we still won't be
able to detect origin_differ conflicts reliably after upgrade. This
can happen in cases where we want to detect conflicts for the
transactions that were pending to replicate before the upgrade. Note,
we don't ensure that the subscriber receives and apply all the
transactions before the upgrade.

Similarly, no slot information after the upgrade helps to protect the
rows from being vacuumed after the upgrade of the subscriber, so
update_delete conflict also may not be detected reliably for the
transactions pending before the upgrade. I think we need to mention
this in the docs so that users can try to ensure that all pending
transactions have been applied before upgrading the subscriber, if
they want to detect all possible conflicts reliably.

--
With Regards,
Amit Kapila.

Commits

  1. Fix intermittent BF failures in 035_conflicts.

  2. Resume conflict-relevant data retention automatically.

  3. Fix intermittent test failure introduced in 6456c6e2c4.

  4. Fix Coverity issue reported in commit a850be2fe.

  5. Add test to prevent premature removal of conflict-relevant data.

  6. Post-commit review fixes for 228c370868.

  7. Add max_retention_duration option to subscriptions.

  8. Detect and report update_deleted conflicts.

  9. Preserve conflict-relevant data during logical replication.

  10. Integrate FullTransactionIds deeper into two-phase code

  11. Improve checks for GUC recovery_target_timeline

  12. Prevent excessive delays before launching new logrep workers.

  13. Keep WAL segments by slot's last saved restart LSN

  14. Rework some code handling pg_subscription data in psql and pg_dump

  15. Use generateClonedIndexStmt to propagate CREATE INDEX to partitions.