RE: Conflict detection and logging in logical replication
Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
From: "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>
To: shveta malik <shveta.malik@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Nisha Moond <nisha.moond412@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, Dilip Kumar <dilipbalaut@gmail.com>, Jan Wieck <jan@wi3ck.info>, Tomas Vondra <tomas.vondra@enterprisedb.com>, Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Date: 2024-07-31T02:10:24Z
Lists: pgsql-hackers
Attachments
- v8-0002-Add-a-detect_conflict-option-to-subscriptions.patch (application/octet-stream) patch v8-0002
- v8-0001-Detect-and-log-conflicts-in-logical-replication.patch (application/octet-stream) patch v8-0001
- v8-0003-Collect-statistics-about-conflicts-in-logical-rep.patch (application/octet-stream) patch v8-0003
On Tuesday, July 30, 2024 5:06 PM shveta malik <shveta.malik@gmail.com> wrote: > > On Mon, Jul 29, 2024 at 11:44 AM Zhijie Hou (Fujitsu) > <houzj.fnst@fujitsu.com> wrote: > > > > Here is the V7 patch set that addressed all the comments so far[1][2][3]. > > Thanks for the patch, few comments: Thanks for the comments ! > > 2) > apply_handle_delete_internal() > > --Do we need to check "(!edata->mtstate || edata->mtstate->operation != > CMD_UPDATE)" in the else part as well? Can there be a scenario where during > update flow, it is trying to delete from a partition and comes here, but till then > that row is deleted already and we end up raising 'delete_missing' additionally > instead of 'update_missing' > alone? I think this shouldn't happen because the row to be deleted should have been locked before entering the apply_handle_delete_internal(). Actually, calling apply_handle_delete_internal() for cross-partition update is a big buggy because the row to be deleted has already been found in apply_handle_tuple_routing(), so we could have avoid scanning the tuple again. I have posted another patch to fix this issue in thread[1]. Here is the V8 patch set. It includes the following changes: * Addressed the comments from Shveta. * Reported the origin name in the DETAIL instead of the origin id. * fixed the issue Dilip pointed[2]. * fixed one old issue[3] Nisha pointed that I missed to fix in previous version. * Improved the document a bit. [1] https://www.postgresql.org/message-id/CAA4eK1JsNPzFE8dgFOm-Tfk_CDZyg1R3zuuQWkUnef-N-vTkoA%40mail.gmail.com [2] https://www.postgresql.org/message-id/CAFiTN-tYdN63U%3Dd8V8rBfRtFmhZ%3DQQX7jEmj1cdWMe_NM%2B7%3DTQ%40mail.gmail.com [3] https://www.postgresql.org/message-id/CABdArM6%2BN1Xy_%2BtK%2Bu-H%3DsCB%2B92rAUh8qH6GDsB%2B1naKzgGKzQ%40mail.gmail.com Best Regards, Hou zj
Commits
-
Rename the conflict types for the origin differ cases.
- 640178c92e3f 18.0 landed
-
Doc: explain the log format of logical replication conflicts.
- edcb71258504 18.0 landed
-
Log the conflicts while applying changes in logical replication.
- 9758174e2e5c 18.0 landed
-
Avoid duplicate table scans for cross-partition updates during logical replication.
- a67da49e1d98 18.0 cited