Re: Conflict detection for update_deleted in logical replication

shveta malik <shveta.malik@gmail.com>

From: shveta malik <shveta.malik@gmail.com>
To: "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>
Cc: Dilip Kumar <dilipbalaut@gmail.com>, Amit Kapila <amit.kapila16@gmail.com>, Masahiko Sawada <sawada.mshk@gmail.com>, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, vignesh C <vignesh21@gmail.com>, Nisha Moond <nisha.moond412@gmail.com>, shveta malik <shveta.malik@gmail.com>
Date: 2025-08-14T03:45:33Z
Lists: pgsql-hackers
On Wed, Aug 13, 2025 at 4:15 PM shveta malik <shveta.malik@gmail.com> wrote:
>
> On Wed, Aug 13, 2025 at 10:41 AM Zhijie Hou (Fujitsu)
> <houzj.fnst@fujitsu.com> wrote:
> >
> >
> > Here is the V61 patch set which addressed above comments and the comment by Nisha[2].
> >
>
> Thank You for the patch. I tested the patch, please find a few comments:
>
> 1)
> Now when it stops-retention and later resumes it due to the fact that
> max_duration is meanwhile altered to 0, I get log:
>
> LOG:  logical replication worker for subscription "sub1" resumes
> retaining the information for detecting conflicts
> DETAIL:  The time spent applying changes up to LSN 0/17DD728 is now
> within the maximum limit of 0 ms.
>
> I did not get which lsn it is pointing to? Is it some dangling lsn
> from when it was retaining info?  Also the msg looks odd, when it says
> 'is now within the maximum limit of 0 ms.'
>
> 2)
> While stopping the message is:
> LOG:  logical replication worker for subscription "sub1" will stop
> retaining conflict information
> DETAIL:  The time spent advancing the non-removable transaction ID has
> exceeded the maximum limit of 1000 ms.
>
> And while resuming:
> logical replication worker for subscription "sub1" resumes retaining
> the information for detecting conflicts
> ----------
>
> We can make both similar. Both can have 'retaining the information for
> detecting conflicts' instead of 'conflict information' in first one.
>
> 3)
> I believe the tenses should also be updated. When stopping, we can say:
>
> Logical replication worker for subscription "sub1" has stopped...
>
> This is appropriate because it has already stopped by pre-setting
> oldest_nonremovable_xid to Invalid.
>
> When resuming, we can say:
> Logical replication worker for subscription "sub1" will resume...
>
> This is because it will begin resuming from the next cycle onward,
> specifically after the launcher sets its oldest_xid.
>
> 4)
> For the DETAIL part of resume and stop messages, how about these:
>
> The retention duration for information used in conflict detection has
> exceeded the limit of xx.
> The retention duration for information used in conflict detection is
> now within the acceptable limit of xx.
> The retention duration for information used in conflict detection is
> now indefinite.
>

5)
Say there 2-3 subs, all have stopped-retention and the slot is set to
have invalid xmin; now if I  create a new sub, it will start with
stopped-flag set to true due to the fact that slot has invalid xmin to
begin with. But then immediately, it will dump a resume message. It
looks odd, as at first, it has not even stopped, as it is a new sub.
Is there anything we can do to improve this situation?

Logs:
2025-08-13 15:13:01.197 IST [61926] LOG:  logical replication apply
worker for subscription "sub4" has started
2025-08-13 15:13:01.482 IST [61926] LOG:  logical replication worker
for subscription "sub4" resumes retaining the information for
detecting conflicts
2025-08-13 15:13:01.482 IST [61926] DETAIL:  The time spent applying
changes up to LSN 0/17C9348 is now within the maximum limit of 50000
ms.

thanks
Shveta



Commits

  1. Fix intermittent BF failures in 035_conflicts.

  2. Resume conflict-relevant data retention automatically.

  3. Fix intermittent test failure introduced in 6456c6e2c4.

  4. Fix Coverity issue reported in commit a850be2fe.

  5. Add test to prevent premature removal of conflict-relevant data.

  6. Post-commit review fixes for 228c370868.

  7. Add max_retention_duration option to subscriptions.

  8. Detect and report update_deleted conflicts.

  9. Preserve conflict-relevant data during logical replication.

  10. Integrate FullTransactionIds deeper into two-phase code

  11. Improve checks for GUC recovery_target_timeline

  12. Prevent excessive delays before launching new logrep workers.

  13. Keep WAL segments by slot's last saved restart LSN

  14. Rework some code handling pg_subscription data in psql and pg_dump

  15. Use generateClonedIndexStmt to propagate CREATE INDEX to partitions.