Horton

Re: Conflict detection for update_deleted in logical replication

Masahiko Sawada <sawada.mshk@gmail.com>

View thread View on postgr.es
From: Masahiko Sawada <sawada.mshk@gmail.com>
To: "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>, shveta malik <shveta.malik@gmail.com>, vignesh C <vignesh21@gmail.com>, Nisha Moond <nisha.moond412@gmail.com>, Dilip Kumar <dilipbalaut@gmail.com>, "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>, Amit Kapila <amit.kapila16@gmail.com>
Date: 2025-07-07T19:00:45Z
Lists: pgsql-hackers 
On Mon, Jul 7, 2025 at 12:29 PM Hayato Kuroda (Fujitsu)
<kuroda.hayato@fujitsu.com> wrote:
>
> Dear Sawada-san,
>
> > What does each duration mean in these results? Can we interpret the
> > test case of max_conflict_retention_duration=120s that when 7 clients
> > and 15 clients are working on the publisher and the subscriber
> > respectively, the TPS on the subscriber was about one fourth (17835.3
> > vs. 4707)?
>
> Firstly, this workload is done to prove that users can tune their workload to keep
> enabling the update_deleted detections. Let me describe what happened there with
> the timetable since the test starts.
>
> 0-162s:
> Number of clients on both publisher/subscriber was 15. TPS was 17835.3 on the
> publisher and 4571.8 on the subscriber. This means that retained dead tuples on
> the subscriber may reduce the performance to around 1/4 compared with publisher,
> and the workload on the publisher is too heavy to keep working the update_deleted
> detection.
>
> 163-314s:
> Number of clients was 7 on publisher, and 15 on subscriber. TPS was 9503.8 on
> the publisher and 4707 on the subscriber. This means that N=7 on the publisher
> was still too many thus conflict slot must be invalidated.
>
> 315-597s:
> Number of clients was 3 on publisher, and 15 on subscriber. TPS was 4243.9 on
> the publisher and 19568.4 on the subscriber. Here the conflict slot could survive
> during the benchmark because concurrency on the publisher was reduced.
> Performance could be improved on the subscriber side because dead tuples can be
> reduced here.

Thank you for your explanation!

Since this feature is designed to reliably detect conflicts on the
subscriber side, this scenario, where both the publisher and
subscriber are under load, represents a typical use case.

The fact that the subscriber can withstand the case where N=7 on the
publisher and N=15 on the subscriber with retain_conflict_info =
false, but fails to do so when retain_conflict_info = true, might
suggests a significant performance impact from enabling this feature.
In these test cases, was autovacuum disabled? I'm curious whether
users would experience permanently reduced transaction throughput, or
if this performance drop is temporary and recovers after autovacuum
runs.

Regards,

-- 
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com

Commits

  1. Fix intermittent BF failures in 035_conflicts.

  2. Resume conflict-relevant data retention automatically.

  3. Fix intermittent test failure introduced in 6456c6e2c4.

  4. Fix Coverity issue reported in commit a850be2fe.

  5. Add test to prevent premature removal of conflict-relevant data.

  6. Post-commit review fixes for 228c370868.

  7. Add max_retention_duration option to subscriptions.

  8. Detect and report update_deleted conflicts.

  9. Preserve conflict-relevant data during logical replication.

  10. Integrate FullTransactionIds deeper into two-phase code

  11. Improve checks for GUC recovery_target_timeline

  12. Prevent excessive delays before launching new logrep workers.

  13. Keep WAL segments by slot's last saved restart LSN

  14. Rework some code handling pg_subscription data in psql and pg_dump

  15. Use generateClonedIndexStmt to propagate CREATE INDEX to partitions.