Horton

Re: Conflict detection for update_deleted in logical replication

Michail Nikolaev <michail.nikolaev@gmail.com>

View thread View on postgr.es
From: Michail Nikolaev <michail.nikolaev@gmail.com>
To: "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>
Cc: "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>, Amit Kapila <amit.kapila16@gmail.com>, Masahiko Sawada <sawada.mshk@gmail.com>, shveta malik <shveta.malik@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, Nisha Moond <nisha.moond412@gmail.com>
Date: 2024-10-29T10:40:54Z
Lists: pgsql-hackers 
Hello Hayato,

> WaitForOlderSnapshots() waits other transactions which can access older
tuples
> than the specified (=current) transaction, right? I think it does not
solve our issue.

Oh, I actually described the idea a bit incorrectly. The goal isn’t simply
to call WaitForOlderSnapshots(slot.xmin);
rather, it’s to ensure that we wait for slot.xmin in the same way we wait
for regular snapshots (xmin).
The reason WaitForOlderSnapshots is used in ReindexConcurrently and
DefineIndex is to guarantee that any transaction
needing to view rows not included in the index has completed before the
index is marked as valid.
The same logic should apply here — we need to wait for the xmin of slot
used in conflict detection as well.

> From another point of view... this approach must fix REINDEX code, but we
should
> not modify other component of codes as much as possible. This feature is
related
> with the replication so that changes should be closed within the
replication subdir.

One possible solution here would be to register a snapshot with slot.xmin
for the worker backend.
This way, WaitForOlderSnapshots will account for it.

By the way, WaitForOlderSnapshots is also used in partitioning and other
areas for similar reasons,
so these might be good places to check for any related issues.

Best regards,
Mikhail,

Commits

  1. Fix intermittent BF failures in 035_conflicts.

  2. Resume conflict-relevant data retention automatically.

  3. Fix intermittent test failure introduced in 6456c6e2c4.

  4. Fix Coverity issue reported in commit a850be2fe.

  5. Add test to prevent premature removal of conflict-relevant data.

  6. Post-commit review fixes for 228c370868.

  7. Add max_retention_duration option to subscriptions.

  8. Detect and report update_deleted conflicts.

  9. Preserve conflict-relevant data during logical replication.

  10. Integrate FullTransactionIds deeper into two-phase code

  11. Improve checks for GUC recovery_target_timeline

  12. Prevent excessive delays before launching new logrep workers.

  13. Keep WAL segments by slot's last saved restart LSN

  14. Rework some code handling pg_subscription data in psql and pg_dump

  15. Use generateClonedIndexStmt to propagate CREATE INDEX to partitions.