Horton

Re: Conflict detection for update_deleted in logical replication

Dilip Kumar <dilipbalaut@gmail.com>

View thread View on postgr.es
From: Dilip Kumar <dilipbalaut@gmail.com>
To: shveta malik <shveta.malik@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Masahiko Sawada <sawada.mshk@gmail.com>, "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, vignesh C <vignesh21@gmail.com>, Nisha Moond <nisha.moond412@gmail.com>
Date: 2025-07-18T02:21:22Z
Lists: pgsql-hackers 
On Thu, Jul 17, 2025 at 4:44 PM shveta malik <shveta.malik@gmail.com> wrote:
>
> On Thu, Jul 17, 2025 at 9:56 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
> >
> > I was just thinking about what are the most practical use cases where
> > a user would need multiple active writer nodes. Most applications
> > typically function well with a single active writer node. While it's
> > beneficial to have multiple nodes capable of writing for immediate
> > failover (e.g., if the current writer goes down), or they select a
> > primary writer via consensus algorithms like Raft/Paxos, I rarely
> > encounter use cases where users require multiple active writer nodes
> > for scaling write workloads.
>
> Thank you for the feedback. In the scenario with a single writer node
> and a subscriber with RCI enabled, we have not observed any
> regression.  Please refer to the test report at [1], specifically test
> cases 1 and 2, which involve a single writer node. Next, we can test a
> scenario with multiple (2-3) writer nodes publishing changes, and a
> subscriber node subscribing to those writers with RCI enabled, which
> can even serve as a good use case of the conflict detection we are
> targeting through RCI enabling.

+1

> > One common use case for multiple active writer nodes is in
> > geographically distributed systems. Here, a dedicated writer in each
> > zone can significantly reduce write latency by sending writes to the
> > nearest zone.
> >
> > In a multi-zone replication setup with an active writer in each zone
> > and data replicated across all zones, performance can be impacted by
> > factors like network latency. However, if such configurations are
> > implemented wisely and subscriptions are managed effectively, this
> > performance impact can be minimized.
> >
> > IMHO, the same principle applies to this case when
> > ‘retain_conflict_info’ is set to ON. If this setting is enabled, it
> > should only be used where absolutely essential. Additionally, the user
> > or DBA must carefully consider other factors. For instance, if they
> > use a single subscriber in each zone and subscribe to everything
> > across all zones, performance will significantly degrade. However, if
> > managed properly by subscribing only to data relevant to each zone and
> > using multiple subscribers for parallel apply of different
> > tables/partitions to reduce delay, it should work fine.
> >
>
> Strongly agree with this. We tested scenarios involving multiple
> subscribers, each subscribing to exclusive data,  as well as
> publishers using row filters. In both cases, no regressions were
> observed. Please refer to the test results at [2] and [3].

Right, thanks for pointing me to the related test cases.

-- 
Regards,
Dilip Kumar
Google

Commits

  1. Fix intermittent BF failures in 035_conflicts.

  2. Resume conflict-relevant data retention automatically.

  3. Fix intermittent test failure introduced in 6456c6e2c4.

  4. Fix Coverity issue reported in commit a850be2fe.

  5. Add test to prevent premature removal of conflict-relevant data.

  6. Post-commit review fixes for 228c370868.

  7. Add max_retention_duration option to subscriptions.

  8. Detect and report update_deleted conflicts.

  9. Preserve conflict-relevant data during logical replication.

  10. Integrate FullTransactionIds deeper into two-phase code

  11. Improve checks for GUC recovery_target_timeline

  12. Prevent excessive delays before launching new logrep workers.

  13. Keep WAL segments by slot's last saved restart LSN

  14. Rework some code handling pg_subscription data in psql and pg_dump

  15. Use generateClonedIndexStmt to propagate CREATE INDEX to partitions.