Horton

Re: Conflict detection for update_deleted in logical replication

Amit Kapila <amit.kapila16@gmail.com>

View thread View on postgr.es
From: Amit Kapila <amit.kapila16@gmail.com>
To: vignesh C <vignesh21@gmail.com>
Cc: "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, Nisha Moond <nisha.moond412@gmail.com>, Masahiko Sawada <sawada.mshk@gmail.com>, shveta malik <shveta.malik@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2025-01-03T10:47:57Z
Lists: pgsql-hackers 
On Fri, Jan 3, 2025 at 2:34 PM vignesh C <vignesh21@gmail.com> wrote:
>
> Few comments:
> 1) In case there are no logical replication workers, the launcher
> process just logs a warning "out of logical replication worker slots"
> and continues. Whereas in case of "pg_conflict_detection" replication
> slot creation launcher throw an error and the launcher exits, can we
> throw a warning in this case too:
> 2025-01-02 10:24:41.899 IST [4280] ERROR:  all replication slots are in use
> 2025-01-02 10:24:41.899 IST [4280] HINT:  Free one or increase
> "max_replication_slots".
> 2025-01-02 10:24:42.148 IST [4272] LOG:  background worker "logical
> replication launcher" (PID 4280) exited with exit code 1
>

This case is not the same because if give just WARNING and allow to
proceed then we won't be able to protect dead rows from removal. We
don't want the apply workers to keep working and making progress till
this slot is created. Am, I missing something? If not, we probably
need to ensure this if not already ensured. Also, we should mention in
the docs that the 'max_replication_slots' setting should consider this
additional slot.

> 2) Currently, we do not detect when the track_commit_timestamp setting
> is disabled for a subscription immediately after the worker starts.
> Instead, it is detected later during conflict detection.
>

I am not sure if an ERROR is required in the first place. Shouldn't we
simply not detect the update_delete in that case? It should be
documented that to detect this conflict 'track_commit_timestamp'
should be enabled. Don't we do the same thing for *_origin_differs
type of conflicts?

-- 
With Regards,
Amit Kapila.

Commits

  1. Fix intermittent BF failures in 035_conflicts.

  2. Resume conflict-relevant data retention automatically.

  3. Fix intermittent test failure introduced in 6456c6e2c4.

  4. Fix Coverity issue reported in commit a850be2fe.

  5. Add test to prevent premature removal of conflict-relevant data.

  6. Post-commit review fixes for 228c370868.

  7. Add max_retention_duration option to subscriptions.

  8. Detect and report update_deleted conflicts.

  9. Preserve conflict-relevant data during logical replication.

  10. Integrate FullTransactionIds deeper into two-phase code

  11. Improve checks for GUC recovery_target_timeline

  12. Prevent excessive delays before launching new logrep workers.

  13. Keep WAL segments by slot's last saved restart LSN

  14. Rework some code handling pg_subscription data in psql and pg_dump

  15. Use generateClonedIndexStmt to propagate CREATE INDEX to partitions.