RE: Conflict detection and logging in logical replication

Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>

From: "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>
To: Michail Nikolaev <michail.nikolaev@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, shveta malik <shveta.malik@gmail.com>, Nisha Moond <nisha.moond412@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, Dilip Kumar <dilipbalaut@gmail.com>, Jan Wieck <jan@wi3ck.info>, Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Date: 2024-08-12T03:33:05Z
Lists: pgsql-hackers
On Friday, August 9, 2024 7:45 PM Michail Nikolaev <michail.nikolaev@gmail.com>  wrote:
> There are some comments on this patch related to issue [0]. In short: any
> DirtySnapshot index scan may fail to find an existing tuple in the case of a
> concurrent update.
> 
> - FindConflictTuple may return false negative result in the case of concurrent update because > ExecCheckIndexConstraints uses SnapshotDirty.
> - As a result, CheckAndReportConflict may fail to report the conflict.
> - In apply_handle_update_internal we may get an CT_UPDATE_MISSING instead of CT_UPDATE_DIFFER
> - In apply_handle_update_internal we may get an CT_DELETE_MISSING instead of CT_DELETE_DIFFER
> - In apply_handle_tuple_routing we may get an CT_UPDATE_MISSING instead of CT_UPDATE_DIFFER
> 
> If you're interested, I could create a test to reproduce the issue within the
> context of logical replication. Issue [0] itself includes a test case to
> replicate the problem.
> 
> It also seems possible that a conflict could be resolved by a concurrent update
> before the call to CheckAndReportConflict, which means there's no guarantee
> that the conflict will be reported correctly. Should we be concerned about
> this?

Thanks for reporting.

I think this is an independent issue which can be discussed separately in the
original thread[1], and I have replied to that thread.

Best Regards,
Hou zj

Commits

  1. Rename the conflict types for the origin differ cases.

  2. Doc: explain the log format of logical replication conflicts.

  3. Log the conflicts while applying changes in logical replication.

  4. Avoid duplicate table scans for cross-partition updates during logical replication.