Re: Data is copied twice when specifying both child and parent table in publication

Amit Kapila <amit.kapila16@gmail.com>

From: Amit Kapila <amit.kapila16@gmail.com>
To: Greg Nancarrow <gregn4422@gmail.com>
Cc: "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>, Amit Langote <amitlangote09@gmail.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>, Dilip Kumar <dilipbalaut@gmail.com>
Date: 2021-11-12T04:11:01Z
Lists: pgsql-hackers
On Thu, Nov 11, 2021 at 1:44 PM Greg Nancarrow <gregn4422@gmail.com> wrote:
>
> On Thu, Nov 11, 2021 at 5:52 PM houzj.fnst@fujitsu.com
> <houzj.fnst@fujitsu.com> wrote:
> >
> > When looking into how to fix the second issue, I have a question:
> >
> > After changing publish_via_partition_root from false to true, the
> > subcriber will fetch the partitioned table from publisher when refreshing.
> >
> > In subsriber side, If all the child tables of the partitioned table already
> > subscribed, then we can just skip the table sync for the partitioned table. But
> > if only some of the child tables(not all child tables) were already subscribed,
> > should we skip the partitioned table's table sync ? I am not sure about the
> > appropriate behavior here.
> >
> > What do you think ?
> >
>
> I'm not sure you can skip the partitioned table's table sync as you
> are suggesting, because on the subscriber side, the tables are mapped
> by name, so what is a partitioned table on the publisher side might
> not be a partitioned table on the subscriber side (e.g. might be an
> ordinary table; and similarly for the partitions) or it might be
> partitioned differently to that on the publisher side.
>

Sure, we don't know about that, or at least there is no such mapping
that is recorded. So, I think we should skip it even if any one of the
child table is present.

> (I might be
> wrong here, and I don't have a good solution, but I can see the
> potential for inconsistent data resulting in this case, unless say,
> the subscriber "child tables" are first truncated on the refresh, if
> they are in fact partitions of the root, and then the table sync
> publishes the existing data via the root)
>

Do you want to say current behavior for this case where data is copied
twice is okay? I think we need to find a way to handle this and
document it to set the expectations right.


-- 
With Regards,
Amit Kapila.



Commits

  1. Avoid syncing data twice for the 'publish_via_partition_root' option.

  2. Fix partition table's REPLICA IDENTITY checking on the subscriber.

  3. Prohibit combining publications with different column lists.

  4. Fix double publish of child table's data.

  5. Support adding partitioned tables to publication