Horton

Re: Conflict detection for update_deleted in logical replication

shveta malik <shveta.malik@gmail.com>

View thread View on postgr.es
From: shveta malik <shveta.malik@gmail.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>, vignesh C <vignesh21@gmail.com>, Masahiko Sawada <sawada.mshk@gmail.com>, Nisha Moond <nisha.moond412@gmail.com>, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2025-05-16T05:45:20Z
Lists: pgsql-hackers 
On Thu, May 15, 2025 at 6:02 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Fri, Apr 25, 2025 at 4:06 PM shveta malik <shveta.malik@gmail.com> wrote:
> >
> > >
> > > Here is V30 patch set includes the following changes:
> > >
> >
> > Thank You for the patch, please find few concerns:
> >
> > 1)
> > By looking at code of ApplyLauncherMain(), it appears that we stopped
> > advancing shared-slot's xmin if any of the subscriptions with
> > retain_conflict_info is disabled. If a subscription is not being used
> > and is disabled forever (or for quite long), that means xmin will
> > never be advanced and we will keep accumulating dead-rows even if
> > other subscribers with retain_conflict_info are actively setting their
> > oldest_xmin. This could be problematic. Here too, there should be some
> > way to set stop-conflict-rettention for such a subscription like we do
> > for 'subscriber not able to catch-up case'.
> > But I understand it can be complex to implement as we do not know for
> > how long a subscription is disabled. If we do not find a simpler way
> > to implement it, then at least we can document  such cases and the
> > risks associated with disabled subscription which has
> > 'retain_conflict_info' enabled. Thoughts?
> >
>
> Yeah, I agree that this is the case to be worried about but OTOH, in
> such cases, even the corresponding logical_slot on the primary won't
> be advanced and lead to bloat on the primary as well. So, we can
> probably give WARNING when user tries to disable a subscription or
> create a disabled subscription with retention flag true. We may need
> to  write a LOG or raise a WARNING when while exiting apply worker
> disables the subscription due disable_on_error option. In addition to
> that, we can even document this case. Will that address your concern?
>

Yes, it should solve the purpose. Thanks.

thanks
Shveta

Commits

  1. Fix intermittent BF failures in 035_conflicts.

  2. Resume conflict-relevant data retention automatically.

  3. Fix intermittent test failure introduced in 6456c6e2c4.

  4. Fix Coverity issue reported in commit a850be2fe.

  5. Add test to prevent premature removal of conflict-relevant data.

  6. Post-commit review fixes for 228c370868.

  7. Add max_retention_duration option to subscriptions.

  8. Detect and report update_deleted conflicts.

  9. Preserve conflict-relevant data during logical replication.

  10. Integrate FullTransactionIds deeper into two-phase code

  11. Improve checks for GUC recovery_target_timeline

  12. Prevent excessive delays before launching new logrep workers.

  13. Keep WAL segments by slot's last saved restart LSN

  14. Rework some code handling pg_subscription data in psql and pg_dump

  15. Use generateClonedIndexStmt to propagate CREATE INDEX to partitions.