Horton

Re: Conflict detection for update_deleted in logical replication

Dilip Kumar <dilipbalaut@gmail.com>

View thread View on postgr.es
From: Dilip Kumar <dilipbalaut@gmail.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Xuneng Zhou <xunengzhou@gmail.com>, "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>, shveta malik <shveta.malik@gmail.com>, vignesh C <vignesh21@gmail.com>, Masahiko Sawada <sawada.mshk@gmail.com>, Nisha Moond <nisha.moond412@gmail.com>, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, pgsql-hackers@lists.postgresql.org
Date: 2025-05-25T08:36:22Z
Lists: pgsql-hackers 
On Sat, May 24, 2025 at 4:46 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> This sounds reasonable to me. Let us see what others think.
>

I was looking into the for getting the transaction status from
publisher, what I would assume this patch should be doing is request
the publisher status first time, and if some transactions are still in
commit, then we need to wait for them to get completed.  But in the
current design its possible that while we are waiting for in-commit
transactions to get committed the old running transaction might come
in commit phase and then we wait for them again, is my understanding
not correct?

Maybe this is very corner case that there are thousands of old running
transaction and everytime we request the status we find some
transactions is in commit phase and the process keep running for long
time until all the old running transaction eventually get committed.

I am thinking can't we make it more deterministic such that when we
get the status first time if we find some transactions that are in
commit phase then we should just wait for those transaction to get
committed?  One idea is to get the list of xids in commit phase and
next time when we get the list we can just compare and in next status
if we don't get any xids in commit phase which were in commit phase
during previous status then we are done.  But not sure is this worth
the complexity?  Mabe not but shall we add some comment explaining the
case and also explaining why this corner case is not harmful?

-- 
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com

Commits

  1. Fix intermittent BF failures in 035_conflicts.

  2. Resume conflict-relevant data retention automatically.

  3. Fix intermittent test failure introduced in 6456c6e2c4.

  4. Fix Coverity issue reported in commit a850be2fe.

  5. Add test to prevent premature removal of conflict-relevant data.

  6. Post-commit review fixes for 228c370868.

  7. Add max_retention_duration option to subscriptions.

  8. Detect and report update_deleted conflicts.

  9. Preserve conflict-relevant data during logical replication.

  10. Integrate FullTransactionIds deeper into two-phase code

  11. Improve checks for GUC recovery_target_timeline

  12. Prevent excessive delays before launching new logrep workers.

  13. Keep WAL segments by slot's last saved restart LSN

  14. Rework some code handling pg_subscription data in psql and pg_dump

  15. Use generateClonedIndexStmt to propagate CREATE INDEX to partitions.