Re: Data is copied twice when specifying both child and parent table in publication
Greg Nancarrow <gregn4422@gmail.com>
From: Greg Nancarrow <gregn4422@gmail.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Amit Langote <amitlangote09@gmail.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>, "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>
Date: 2021-10-20T07:14:07Z
Lists: pgsql-hackers
On Mon, Oct 18, 2021 at 5:00 PM Amit Kapila <amit.kapila16@gmail.com> wrote: > > > I have not debugged it yet to find out why, but with the patch > > applied, the original double-publish problem that I reported > > (converted to just use TABLE rather than ALL TABLES IN SCHEMA) still > > occurs. > > > > Yeah, I think this is a variant of the problem being fixed by > Hou-San's patch. I think one possible idea to investigate is that on > the subscriber-side, after fetching tables, we check the already > subscribed tables and if the child tables already exist then we ignore > the parent table and vice versa. We might want to consider the case > where a user has toggled the "publish_via_partition_root" parameter. > > It seems both these behaviours/problems exist since commit 17b9e7f9 > (Support adding partitioned tables to publication). Adding Amit L and > Peter E (people involved in this work) to know their opinion? > Actually, at least with the scenario I gave steps for, after looking at it again and debugging, I think that the behavior is understandable and not a bug. The reason is that the INSERTed data is first published though the partitions, since initially there is no partitioned table in the publication (so publish_via_partition_root=true doesn't have any effect). But then adding the partitioned table to the publication and refreshing the publication in the subscriber, the data is then published "using the identity and schema of the partitioned table" due to publish_via_partition_root=true. Note that the corresponding table in the subscriber may well be a non-partitioned table (or the partitions arranged differently) so the data does need to be replicated again. Regards, Greg Nancarrow Fujitsu Australia
Commits
-
Avoid syncing data twice for the 'publish_via_partition_root' option.
- 062a84442424 16.0 landed
-
Fix partition table's REPLICA IDENTITY checking on the subscriber.
- 52d5ea9adb01 14.5 cited
-
Prohibit combining publications with different column lists.
- fd0b9dcebda7 15.0 cited
-
Fix double publish of child table's data.
- 3f06c00cf6dc 13.6 landed
- 614b77d65a38 14.2 landed
- 5e97905a2c76 15.0 landed
-
Support adding partitioned tables to publication
- 17b9e7f9fe23 13.0 cited