Re: Data is copied twice when specifying both child and parent table in publication
Greg Nancarrow <gregn4422@gmail.com>
From: Greg Nancarrow <gregn4422@gmail.com>
To: Dilip Kumar <dilipbalaut@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>,
Amit Langote <amitlangote09@gmail.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>, "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>
Date: 2021-10-20T09:00:18Z
Lists: pgsql-hackers
On Wed, Oct 20, 2021 at 7:02 PM Dilip Kumar <dilipbalaut@gmail.com> wrote: > > > Actually, at least with the scenario I gave steps for, after looking > > at it again and debugging, I think that the behavior is understandable > > and not a bug. > > The reason is that the INSERTed data is first published though the > > partitions, since initially there is no partitioned table in the > > publication (so publish_via_partition_root=true doesn't have any > > effect). But then adding the partitioned table to the publication and > > refreshing the publication in the subscriber, the data is then > > published "using the identity and schema of the partitioned table" due > > to publish_via_partition_root=true. Note that the corresponding table > > in the subscriber may well be a non-partitioned table (or the > > partitions arranged differently) so the data does need to be > > replicated again. > > I don't think this behavior is consistent, I mean for the initial sync > we will replicate the duplicate data, whereas for later streaming we > will only replicate it once. From the user POW, this behavior doesn't > look correct. > The scenario I gave steps for didn't have any table data when the subscription was made, so the initial sync did not replicate any data. I was referring to the double-publish that occurs when publish_via_partition_root=true and then the partitioned table is added to the publication and the subscriber does ALTER SUBSCRIPTION ... REFRESH PUBLICATION. If I modify my example to include both the partitioned table and (explicitly) its child partitions in the publication, and insert some data on the publisher side prior to the subscription, then I am seeing duplicate data on the initial sync on the subscriber side, and I would agree that this doesn't seem correct. Regards, Greg Nancarrow Fujitsu Australia
Commits
-
Avoid syncing data twice for the 'publish_via_partition_root' option.
- 062a84442424 16.0 landed
-
Fix partition table's REPLICA IDENTITY checking on the subscriber.
- 52d5ea9adb01 14.5 cited
-
Prohibit combining publications with different column lists.
- fd0b9dcebda7 15.0 cited
-
Fix double publish of child table's data.
- 3f06c00cf6dc 13.6 landed
- 614b77d65a38 14.2 landed
- 5e97905a2c76 15.0 landed
-
Support adding partitioned tables to publication
- 17b9e7f9fe23 13.0 cited