Re: Data is copied twice when specifying both child and parent table in publication
Amit Kapila <amit.kapila16@gmail.com>
From: Amit Kapila <amit.kapila16@gmail.com>
To: "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>
Cc: Greg Nancarrow <gregn4422@gmail.com>,
Amit Langote <amitlangote09@gmail.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>,
Dilip Kumar <dilipbalaut@gmail.com>
Date: 2021-11-12T04:27:39Z
Lists: pgsql-hackers
On Thu, Nov 11, 2021 at 12:22 PM houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com> wrote: > > On Friday, November 5, 2021 11:20 AM Greg Nancarrow <gregn4422@gmail.com> wrote: > >On Thu, Nov 4, 2021 at 7:10 PM Amit Kapila <mailto:amit.kapila16@gmail.com> wrote: > > > >Almost. > >The patch does seem to solve that first problem (double publish on tablesync). > >I used the following test (taken from [2]), and variations of it: > > > >However, there did still seem to be a problem, if publish_via_partition_root is then set to false; it seems that can result in > >duplicate partition entries in the pg_publication_tables view, see below (this follows on from the test scenario given above): > > > >postgres=# select * from pg_publication_tables; > > pubname | schemaname | tablename > >---------+------------+----------- > > pub1 | sch1 | tbl1 > > pub1 | sch3 | t1 > >(2 rows) > > > >postgres=# alter publication pub1 set (publish_via_partition_root=false); > >ALTER PUBLICATION > >postgres=# select * from pg_publication_tables; > > pubname | schemaname | tablename > >---------+------------+------------ > > pub1 | sch2 | tbl1_part1 > > pub1 | sch2 | tbl1_part2 > > pub1 | sch2 | tbl1_part1 > > pub1 | sch3 | t1 > >(4 rows) > > > >So I think the patch would need to be updated to prevent that. > > Thanks for testing the patch. > > The reason of the duplicate output is that: > The existing function GetPublicationRelations doesn't de-duplicate the output > oid list. So, when adding both child and parent table to the > publication(pubviaroot = false), the pg_publication_tables view will output > duplicate partition. > > Attach the fix patch. > 0001 fix data double publish(first issue in this thread) > 0002 fix duplicate partition in view pg_publication_tables(reported by greg when testing the 0001 patch) > Can we start a separate thread to discuss the 0002 patch as that doesn't seem directly to duplicate data issues being discussed here? Please specify the exact test in the email as that would make it easier to understand the problem. -- With Regards, Amit Kapila.
Commits
-
Avoid syncing data twice for the 'publish_via_partition_root' option.
- 062a84442424 16.0 landed
-
Fix partition table's REPLICA IDENTITY checking on the subscriber.
- 52d5ea9adb01 14.5 cited
-
Prohibit combining publications with different column lists.
- fd0b9dcebda7 15.0 cited
-
Fix double publish of child table's data.
- 3f06c00cf6dc 13.6 landed
- 614b77d65a38 14.2 landed
- 5e97905a2c76 15.0 landed
-
Support adding partitioned tables to publication
- 17b9e7f9fe23 13.0 cited