Horton

Re: Conflict detection for update_deleted in logical replication

shveta malik <shveta.malik@gmail.com>

View thread View on postgr.es
From: shveta malik <shveta.malik@gmail.com>
To: "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>
Cc: vignesh C <vignesh21@gmail.com>, Amit Kapila <amit.kapila16@gmail.com>, Masahiko Sawada <sawada.mshk@gmail.com>, Nisha Moond <nisha.moond412@gmail.com>, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, shveta malik <shveta.malik@gmail.com>
Date: 2025-04-17T04:19:10Z
Lists: pgsql-hackers 
On Wed, Apr 16, 2025 at 10:30 AM shveta malik <shveta.malik@gmail.com> wrote:
>
> On Wed, Mar 26, 2025 at 4:17 PM Zhijie Hou (Fujitsu)
> <houzj.fnst@fujitsu.com> wrote:
> >
> > Here's a rebased version of the patch series.
> >
>

Few comments for patch004:
Config.sgml:
1)
+       <para>
+        Maximum duration (in milliseconds) for which conflict
+        information can be retained for conflict detection by the apply worker.
+        The default value is <literal>0</literal>, indicating that conflict
+        information is retained until it is no longer needed for detection
+        purposes.
+       </para>

IIUC, the above is not entirely accurate. Suppose the subscriber
manages to catch up and sets oldest_nonremovable_xid to 100, which is
then updated in slot. After this, the apply worker takes a nap and
begins a new xid update cycle. Now, let’s say the next candidate_xid
is 200, but this time the subscriber fails to keep up and exceeds
max_conflict_retention_duration. As a result, it sets
oldest_nonremovable_xid to InvalidTransactionId, and the launcher
skips updating the slot’s xmin. However, the previous xmin value (100)
is still there in the slot, causing its data to be retained beyond the
max_conflict_retention_duration. The xid 200 which actually honors
max_conflict_retention_duration was never marked for retention. If my
understanding is correct, then the documentation doesn’t fully capture
this scenario.


2)
+        The replication slot
+        <quote><literal>pg_conflict_detection</literal></quote> that used to
+        retain conflict information will be invalidated if all apply workers
+        associated with the subscription, where

Subscription --> subscriptions

3)
Name stop_conflict_retention in MyLogicalRepWorker is confusing. Shall
it be stop_conflict_info_retention?

thanks
Shveta

Commits

  1. Fix intermittent BF failures in 035_conflicts.

  2. Resume conflict-relevant data retention automatically.

  3. Fix intermittent test failure introduced in 6456c6e2c4.

  4. Fix Coverity issue reported in commit a850be2fe.

  5. Add test to prevent premature removal of conflict-relevant data.

  6. Post-commit review fixes for 228c370868.

  7. Add max_retention_duration option to subscriptions.

  8. Detect and report update_deleted conflicts.

  9. Preserve conflict-relevant data during logical replication.

  10. Integrate FullTransactionIds deeper into two-phase code

  11. Improve checks for GUC recovery_target_timeline

  12. Prevent excessive delays before launching new logrep workers.

  13. Keep WAL segments by slot's last saved restart LSN

  14. Rework some code handling pg_subscription data in psql and pg_dump

  15. Use generateClonedIndexStmt to propagate CREATE INDEX to partitions.