Horton

RE: Conflict detection for update_deleted in logical replication

Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>

View thread View on postgr.es
From: "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>
To: Peter Smith <smithpb2250@gmail.com>, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, Nisha Moond <nisha.moond412@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Masahiko Sawada <sawada.mshk@gmail.com>, shveta malik <shveta.malik@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2024-10-28T03:06:34Z
Lists: pgsql-hackers

Attachments

On Thursday, October 24, 2024 1:00 PM Peter Smith <smithpb2250@gmail.com> wrote:

Thanks Peter, Kuroda-san and Nisha for the comments.

> 
> Hi Hou-san, here are my review comments for patch v5-0001.
> 
> 15.
> + /*
> + * Exit early if the user has disabled sending messages to the
> + * publisher.
> + */
> + if (wal_receiver_status_interval <= 0) return;
> 
> What are the implications of this early exit? If the update request is not possible,
> then I guess the update status is never received, but then I suppose that means
> none of this update_deleted logic is possible. If that is correct, then will there
> be some documented warning/caution about conflict-handling implications by
> disabling that GUC?

The detection of update_deleted is still possible in this case, but the dead tuple
cannot be cleaned up without communicating with the publisher. I have documented
this in the 0004 patch where the subscription option is added.

On Friday, October 25, 2024 2:17 PM Kuroda, Hayato/黒田 隼人 <kuroda.hayato@fujitsu.com> wrote:
> > Here is the V5 patch set which addressed above comments.
> 
> Thanks for updating the patch! While reviewing yours, I found a corner case
> that
> a recently deleted tuple cannot be detected when index scan is chosen.
> This can happen when indices are re-built during the replication.
> Unfortunately, I don't have any solutions for it.

Thanks for reporting. I changed the patch to use table scan in all cases, since
It is used only in conflict cases, so it could be OK, but I will
think if there are better solutions for it based on Mikhail's reply[1].


On Friday, October 25, 2024 4:51 PM Nisha Moond <nisha.moond412@gmail.com> wrote:
> 1) In FindMostRecentlyDeletedTupleInfo(),
> 
> + /* Try to find the tuple */
> + while (index_getnext_slot(scan, ForwardScanDirection, scanslot)) {
> + Assert(tuples_equal(scanslot, searchslot, eq));
> + update_recent_dead_tuple_info(scanslot, oldestXmin, delete_xid,
> +   delete_time, delete_origin);
> + }
> 
> In my tests, I found that the above assert() triggers during unidirectional
> replication of an update on a table. While doing the replica identity index scan,
> it can only ensure to match the indexed columns value, but the current Assert()
> assumes all the column values should match, which seems wrong.

Thanks for reporting. These codes have been removed in V6 patch set.

> 
> 2) Since update_deleted requires both 'track_commit_timestamp' and the
> 'detect_update_deleted' to be enabled, should we raise an error in the CREATE
> and ALTER subscription commands when track_commit_timestamp=OFF but
> the user specifies detect_update_deleted=true?

Added.

Attach the V6 patch set which addressed all the comments so far.

[1] https://www.postgresql.org/message-id/CANtu0ohzMB38tcf2%3Dq1HVCdz%2BeEpBBauHhMoATz_%3DYFEjDd7FQ%40mail.gmail.com.

Best Regards,
Hou zj

Commits

  1. Fix intermittent BF failures in 035_conflicts.

  2. Resume conflict-relevant data retention automatically.

  3. Fix intermittent test failure introduced in 6456c6e2c4.

  4. Fix Coverity issue reported in commit a850be2fe.

  5. Add test to prevent premature removal of conflict-relevant data.

  6. Post-commit review fixes for 228c370868.

  7. Add max_retention_duration option to subscriptions.

  8. Detect and report update_deleted conflicts.

  9. Preserve conflict-relevant data during logical replication.

  10. Integrate FullTransactionIds deeper into two-phase code

  11. Improve checks for GUC recovery_target_timeline

  12. Prevent excessive delays before launching new logrep workers.

  13. Keep WAL segments by slot's last saved restart LSN

  14. Rework some code handling pg_subscription data in psql and pg_dump

  15. Use generateClonedIndexStmt to propagate CREATE INDEX to partitions.