Horton

RE: Conflict detection for update_deleted in logical replication

Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>

View thread View on postgr.es
From: "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>
To: shveta malik <shveta.malik@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Nisha Moond <nisha.moond412@gmail.com>, Masahiko Sawada <sawada.mshk@gmail.com>, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, vignesh C <vignesh21@gmail.com>, Dilip Kumar <dilipbalaut@gmail.com>
Date: 2025-08-27T03:29:54Z
Lists: pgsql-hackers

Attachments

On Tuesday, August 26, 2025 6:45 PM shveta malik <shveta.malik@gmail.com> wrote:
> 
> Please find some more comments:

Thanks for the comments!

> 
> 
> 3)
> CREATE SUBSCRIPTION sub CONNECTION '...' PUBLICATION pub WITH
> (connect = false, retain_dead_tuples = true, max_conflict_retention_duration
> = 5000);
> WARNING:  deleted rows to detect conflicts would not be removed until the
> subscription is enabled
> HINT:  Consider setting retain_dead_tuples to false.
> WARNING:  subscription was created, but is not connected
> HINT:  To initiate replication, you must manually create the replication slot,
> enable the subscription, and refresh the subscription.
> CREATE SUBSCRIPTION
> 
> With connect=false, we get above messages. Reverse order of WARNINGs will
> make more sense as 'not connected' WARNING and HINT clarifies a few things
> including that the sub is disabled and needs to be enabled.
> Can we attempt doing it provided it does not over-complicate code?

I agree it makes sense for reversing the messages. But since this behavior isn't
caused by the remaining patches, we can consider it a separate improvement after
the main patch is pushed.

All other comments look good to me, so addressed in the latest patches.

Here is V66 patch set which includes the following changes:

[0001]
* Enhanced documentation according to Dilip's comments [1].
* Simplified logic by directly using the leader's oldest_nonremovable_xid to
  locate deleted tuples, according to Amit's comments [2].
* Merged Amit's diff [2].
* Integrated the new NOTICE into the existing CheckSubDeadTupleRetention
  function according to Amit's feedback [3].
* Some other adjustments according to feedback from [2] [3].

[0002]
* Refactored code according to Shveta's comments [4].
* Removed an unnecessary variable according to Shveta's suggestions [5].
* Some other adjustments according to feedback from [4][5][6].

[1] https://www.postgresql.org/message-id/CAFiTN-v2-Jv3UFYQ48pbX%2Bjb%2BMXWoxrfsRXQ_J1s1xqPq8P_zg%40mail.gmail.com
[2] https://www.postgresql.org/message-id/CAA4eK1%2BHcrkKfXAwEsXK0waDK8VSx1qjBVj95SmZKPM0vMF%3DQg%40mail.gmail.com
[3] https://www.postgresql.org/message-id/CAA4eK1JhYwJhU4vYPGeh8Y46S%2BFS5ciATw5beJKPrkF5ZAu2AQ%40mail.gmail.com
[4] https://www.postgresql.org/message-id/CAJpy0uCrHtwN3wgnC26G8M4jQfaMJG3EUU3OY%2BzpwQPeifjmTg%40mail.gmail.com
[5] https://www.postgresql.org/message-id/CAJpy0uDpX6jQWC3-cyA38ANT0L_L_qQeWPy2cATzSpLNDha1%3DA%40mail.gmail.com
[6] https://www.postgresql.org/message-id/CAJpy0uBQ_v0D3ceuZfJrx%3DzH6-59ORLqj%2BaqZJ7AQnw3vRRcSA%40mail.gmail.com

Best Regards,
Hou zj

Commits

  1. Fix intermittent BF failures in 035_conflicts.

  2. Resume conflict-relevant data retention automatically.

  3. Fix intermittent test failure introduced in 6456c6e2c4.

  4. Fix Coverity issue reported in commit a850be2fe.

  5. Add test to prevent premature removal of conflict-relevant data.

  6. Post-commit review fixes for 228c370868.

  7. Add max_retention_duration option to subscriptions.

  8. Detect and report update_deleted conflicts.

  9. Preserve conflict-relevant data during logical replication.

  10. Integrate FullTransactionIds deeper into two-phase code

  11. Improve checks for GUC recovery_target_timeline

  12. Prevent excessive delays before launching new logrep workers.

  13. Keep WAL segments by slot's last saved restart LSN

  14. Rework some code handling pg_subscription data in psql and pg_dump

  15. Use generateClonedIndexStmt to propagate CREATE INDEX to partitions.