Re: Data is copied twice when specifying both child and parent table in publication
Jacob Champion <jchampion@timescale.com>
From: Jacob Champion <jchampion@timescale.com>
To: Peter Smith <smithpb2250@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>,
"wangw.fnst@fujitsu.com" <wangw.fnst@fujitsu.com>,
Andres Freund <andres@anarazel.de>, vignesh C <vignesh21@gmail.com>,
"Takamichi Osumi (Fujitsu)" <osumi.takamichi@fujitsu.com>,
"shiy.fnst@fujitsu.com" <shiy.fnst@fujitsu.com>,
"houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>,
Amit Langote <amitlangote09@gmail.com>,
Peter Eisentraut <peter.eisentraut@enterprisedb.com>,
"pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>,
Dilip Kumar <dilipbalaut@gmail.com>, Greg Nancarrow <gregn4422@gmail.com>
Date: 2023-03-31T23:04:37Z
Lists: pgsql-hackers
On 3/30/23 20:01, Peter Smith wrote: > For example, Just imagine if logic could be made smarter to recognize > that since there was already the 'part_def' being subscribed so it > should NOT use the default 'copy_data=true' when the REFRESH launches > the ancestor table 'part'... > > Even if that logic was implemented, I have a feeling you could *still* > run into problems if the 'part' table was made of multiple partitions. > I think you might get to a situation where you DO want some partition > data copied (because you did not have it yet but now you are > subscribing to the root you want it) while at the same time, you DON'T > want to get duplicated data from other partitions (because you already > knew about those ones -- like your example does). Hm, okay. My interest here is mainly because my logical-roots proposal generalizes the problem (and therefore makes it worse). For what it's worth, that patchset introduces the ability for the subscriber to sync multiple tables into one. I wonder if that could be used somehow to help fix this problem too? > At least, we need to check there are sufficient "BE CAREFUL" warnings > in the documentation for scenarios like this. Agreed. These are sharp edges. Thanks, --Jacob
Commits
-
Avoid syncing data twice for the 'publish_via_partition_root' option.
- 062a84442424 16.0 landed
-
Fix partition table's REPLICA IDENTITY checking on the subscriber.
- 52d5ea9adb01 14.5 cited
-
Prohibit combining publications with different column lists.
- fd0b9dcebda7 15.0 cited
-
Fix double publish of child table's data.
- 3f06c00cf6dc 13.6 landed
- 614b77d65a38 14.2 landed
- 5e97905a2c76 15.0 landed
-
Support adding partitioned tables to publication
- 17b9e7f9fe23 13.0 cited