Horton

Re: Conflict detection for update_deleted in logical replication

shveta malik <shveta.malik@gmail.com>

View thread View on postgr.es
From: shveta malik <shveta.malik@gmail.com>
To: "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Nisha Moond <nisha.moond412@gmail.com>, Masahiko Sawada <sawada.mshk@gmail.com>, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, vignesh C <vignesh21@gmail.com>, Dilip Kumar <dilipbalaut@gmail.com>, shveta malik <shveta.malik@gmail.com>
Date: 2025-08-26T10:44:54Z
Lists: pgsql-hackers 
Please find some more comments:

1)
In CheckSubDeadTupleRetention(), shall we have below instead of
retain_dead_tuples check in all conditions?

if (retain_dead_tuples)
    guc checks (wal_level and tracl_commit)
else
   max retention check

2)
Currently stop and resume messages are:
~~
LOG:  logical replication worker for subscription "sub2" has stopped
retaining the information for detecting conflicts
DETAIL:  The retention duration for information used in conflict
detection has exceeded the maximum limit of 10000 ms.
HINT:  You might need to increase "max_conflict_retention_duration".
 --
LOG:  logical replication worker for subscription "sub2" will resume
retaining the information for detecting conflicts
DETAIL:  The retention duration for information used in conflict
detection is now within the acceptable limit of 10000 ms.
~~

Resume message does not mention GUC while stop does mention it in
HINT. Shall we have both stop and resume DETAIL msg mention GUC as:

Stop:
DETAIL: Retention of information used for conflict detection has
exceeded max_conflict_retention_duration of 10000 ms.

Resume:
DETAIL: Retention of information used for conflict detection is now
within the max_conflict_retention_duration of 1000 ms.

I think we should get rid of HINT in stop msg as that is not what we
actually should be suggesting without knowing the
system-workload/bloat condition. Hint seems oversimplified and thus
incomplete looking at the possibilities we may have here.


3)
CREATE SUBSCRIPTION sub CONNECTION '...' PUBLICATION pub WITH (connect
= false, retain_dead_tuples = true, max_conflict_retention_duration =
5000);
WARNING:  deleted rows to detect conflicts would not be removed until
the subscription is enabled
HINT:  Consider setting retain_dead_tuples to false.
WARNING:  subscription was created, but is not connected
HINT:  To initiate replication, you must manually create the
replication slot, enable the subscription, and refresh the
subscription.
CREATE SUBSCRIPTION

With connect=false, we get above messages. Reverse order of WARNINGs
will make more sense as 'not connected' WARNING and HINT clarifies a
few things including that the sub is disabled and needs to be enabled.
Can we attempt doing it provided it does not over-complicate code?

4)
postgres=# \dRs+
     List of subscriptions
.. | Retain dead tuples | Max conflict retention duration | Dead tuple
retention active |..

Here shall we have  'Max retention duration' and 'Retention Active'
instead of 'Max conflict retention duration' and 'Dead tuple retention
active'?

thanks
Shveta

Commits

  1. Fix intermittent BF failures in 035_conflicts.

  2. Resume conflict-relevant data retention automatically.

  3. Fix intermittent test failure introduced in 6456c6e2c4.

  4. Fix Coverity issue reported in commit a850be2fe.

  5. Add test to prevent premature removal of conflict-relevant data.

  6. Post-commit review fixes for 228c370868.

  7. Add max_retention_duration option to subscriptions.

  8. Detect and report update_deleted conflicts.

  9. Preserve conflict-relevant data during logical replication.

  10. Integrate FullTransactionIds deeper into two-phase code

  11. Improve checks for GUC recovery_target_timeline

  12. Prevent excessive delays before launching new logrep workers.

  13. Keep WAL segments by slot's last saved restart LSN

  14. Rework some code handling pg_subscription data in psql and pg_dump

  15. Use generateClonedIndexStmt to propagate CREATE INDEX to partitions.