Horton

RE: Conflict detection for update_deleted in logical replication

Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>

View thread View on postgr.es
From: "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>
To: "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>, shveta malik <shveta.malik@gmail.com>
Cc: Dilip Kumar <dilipbalaut@gmail.com>, Amit Kapila <amit.kapila16@gmail.com>, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, vignesh C <vignesh21@gmail.com>, Nisha Moond <nisha.moond412@gmail.com>, Masahiko Sawada <sawada.mshk@gmail.com>
Date: 2025-08-07T04:39:53Z
Lists: pgsql-hackers

Attachments

On Tuesday, August 5, 2025 10:09 AM Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com> wrote:
> Here is V57 patch set which addressed most of comments.
> 
> In this version, I also fixed a bug that the apply worker continued to find dead
> tuples even if it has already stop retaining dead tuples.

Here is a V58 patch set which improved few things by internal review:

0001:

* Remove the slot invalidation.

Initially, we thought it would be convenient for users to determine if they can
reliably detect update_deleted by checking the validity of the conflict
detection slot. However, after re-thinking, even if the slot is valid, it
doesn't guarantee that each apply worker can reliably detect conflicts. Some
apply workers might have stopped retention, yet the slot remains valid due to
other active workers continuing retention.

Instead of querying the slot, users should verify the ability of a specific
apply worker to reliably detect conflicts by checking the view
pg_stat_subscription.retain_dead_tuples.

So, slot invalidation would be necessary. We could set slot.xmin to invalid
instead to allow dead tuples to be removed when all apply workers stop
retention. This approach simplifies implementation and avoids introducing a new
invalidation type solely for one internal slot.

* Fixed a bug that parallel apply worker continues to search dead tuples when
  the retention has already stopped. The parallel and table sync worker referred
  to its own stop_conflict_info_retention flag, but should refer to the
  retention flag of the leader instead because only leader mananges this flag.

0002:

* Allow the apply worker to wait for the slot to be recover after resuming the dead
  tuple retention instead of restarting the apply worker.

Best Regards,
Hou zj

Commits

  1. Fix intermittent BF failures in 035_conflicts.

  2. Resume conflict-relevant data retention automatically.

  3. Fix intermittent test failure introduced in 6456c6e2c4.

  4. Fix Coverity issue reported in commit a850be2fe.

  5. Add test to prevent premature removal of conflict-relevant data.

  6. Post-commit review fixes for 228c370868.

  7. Add max_retention_duration option to subscriptions.

  8. Detect and report update_deleted conflicts.

  9. Preserve conflict-relevant data during logical replication.

  10. Integrate FullTransactionIds deeper into two-phase code

  11. Improve checks for GUC recovery_target_timeline

  12. Prevent excessive delays before launching new logrep workers.

  13. Keep WAL segments by slot's last saved restart LSN

  14. Rework some code handling pg_subscription data in psql and pg_dump

  15. Use generateClonedIndexStmt to propagate CREATE INDEX to partitions.