Horton

Re: Conflict detection and logging in logical replication

Peter Smith <smithpb2250@gmail.com>

View thread View on postgr.es
From: Peter Smith <smithpb2250@gmail.com>
To: "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>
Cc: shveta malik <shveta.malik@gmail.com>, Amit Kapila <amit.kapila16@gmail.com>, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, Nisha Moond <nisha.moond412@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, Dilip Kumar <dilipbalaut@gmail.com>, Jan Wieck <jan@wi3ck.info>, Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Date: 2024-08-22T08:02:52Z
Lists: pgsql-hackers 
Hi Hou-san.

I was experimenting with some conflict logging and found that large
column values are truncated in the log DETAIL.

E.g. Below I have a table where I inserted a 3000 character text value
'bigbigbig..."

Then I caused a replication conflict.

test_sub=# delete fr2024-08-22 17:50:17.181 AEST [14901] LOG:  logical
replication apply worker for subscription "sub1" has started
2024-08-22 17:50:17.193 AEST [14901] ERROR:  conflict detected on
relation "public.t1": conflict=insert_exists
2024-08-22 17:50:17.193 AEST [14901] DETAIL:  Key already exists in
unique index "t1_pkey", modified in transaction 780.
Key (a)=(k3); existing local tuple (k3,
bigbigbigbigbigbigbigbigbigbigbigbigbigbigbigbigbigbigbigbigbigb...);
remote tuple (k3, this will clash).

~

Do you think the documentation for the 'column_value' parameter of the
conflict logging should say that the displayed value might be
truncated?

======
Kind Regards,
Peter Smith.
Fujitsu Australia

Commits

  1. Rename the conflict types for the origin differ cases.

  2. Doc: explain the log format of logical replication conflicts.

  3. Log the conflicts while applying changes in logical replication.

  4. Avoid duplicate table scans for cross-partition updates during logical replication.