Horton

Re: Conflict detection for update_deleted in logical replication

shveta malik <shveta.malik@gmail.com>

View thread View on postgr.es
From: shveta malik <shveta.malik@gmail.com>
To: "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>
Cc: Dilip Kumar <dilipbalaut@gmail.com>, Amit Kapila <amit.kapila16@gmail.com>, Masahiko Sawada <sawada.mshk@gmail.com>, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, vignesh C <vignesh21@gmail.com>, Nisha Moond <nisha.moond412@gmail.com>, shveta malik <shveta.malik@gmail.com>
Date: 2025-08-12T08:36:30Z
Lists: pgsql-hackers 
On Mon, Aug 11, 2025 at 2:40 PM Zhijie Hou (Fujitsu)
<houzj.fnst@fujitsu.com> wrote:
>
> I agree. So, following the above points and some off-list discussions, I have
> revised the option to be a subscription option in the V60 version.
>

Thank You for the patches. Tried to test the new sub-level parameter,
have few comments:


1)
Let's say commit on pub is taking time and worker has stopped
retention and meanwhile we alter max_conflict_retention_duration=0,
then the expectation is immediately worker should resume retention.
But it does not happen, it does not restart conflict-retention until
the pub's commit is finished. The 'dead_tuple_retention_active'
remains 'f' till then.

postgres=# select subname, subretaindeadtuples, maxconflretention from
pg_subscription order by subname;
 subname | subretaindeadtuples | maxconflretention
---------+---------------------+-------------------
 sub1       | t                                |                 0


postgres=# select subname, worker_type, dead_tuple_retention_active
from pg_stat_subscription order by subname;
 subname | worker_type | dead_tuple_retention_active
---------+-------------+-----------------------------
 sub1      | apply              | f

I think we shall reset 'stop_conflict_info_retention' flag in
should_stop_conflict_info_retention() if maxconflretention is 0 and
the flag is originally true.


2)
postgres=# create subscription sub2  connection 'dbname=postgres
host=localhost user=shveta port=5433' publication pub2 WITH
(retain_dead_tuples = false, max_conflict_retention_duration=1000);
NOTICE:  created replication slot "sub2" on publisher
CREATE SUBSCRIPTION

Shall we give notice that max_conflict_retention_duration is ignored
as retain_dead_tuples is false.


3)
When worker stops retention, it gives message:

LOG:  logical replication worker for subscription "sub1" will stop
retaining the information for detecting conflicts
DETAIL:  The time spent advancing the non-removable transaction ID has
exceeded the maximum limit of 100 ms.

Will it be more informative if we mention the parameter name
'max_conflict_retention_duration' either in DETAIL or in additional
HINT, as then the user can easily map this behaviour to the parameter
configured.

thanks
Shveta

Commits

  1. Fix intermittent BF failures in 035_conflicts.

  2. Resume conflict-relevant data retention automatically.

  3. Fix intermittent test failure introduced in 6456c6e2c4.

  4. Fix Coverity issue reported in commit a850be2fe.

  5. Add test to prevent premature removal of conflict-relevant data.

  6. Post-commit review fixes for 228c370868.

  7. Add max_retention_duration option to subscriptions.

  8. Detect and report update_deleted conflicts.

  9. Preserve conflict-relevant data during logical replication.

  10. Integrate FullTransactionIds deeper into two-phase code

  11. Improve checks for GUC recovery_target_timeline

  12. Prevent excessive delays before launching new logrep workers.

  13. Keep WAL segments by slot's last saved restart LSN

  14. Rework some code handling pg_subscription data in psql and pg_dump

  15. Use generateClonedIndexStmt to propagate CREATE INDEX to partitions.