RE: Data is copied twice when specifying both child and parent table in publication

Wei Wang (Fujitsu) <wangw.fnst@fujitsu.com>

From: "wangw.fnst@fujitsu.com" <wangw.fnst@fujitsu.com>
To: vignesh C <vignesh21@gmail.com>
Cc: Peter Smith <smithpb2250@gmail.com>, "Takamichi Osumi (Fujitsu)" <osumi.takamichi@fujitsu.com>, Amit Kapila <amit.kapila16@gmail.com>, "shiy.fnst@fujitsu.com" <shiy.fnst@fujitsu.com>, "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>, Amit Langote <amitlangote09@gmail.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>, Dilip Kumar <dilipbalaut@gmail.com>, Greg Nancarrow <gregn4422@gmail.com>
Date: 2022-11-16T08:58:31Z
Lists: pgsql-hackers

Attachments

On Mon, Nov 14, 2022 at 0:56 AM vignesh C <vignesh21@gmail.com> wrote:
> >
> > Attach new patches.
> 

Thanks for your comments.

> Here we are having tables list to store the relids and table_infos
> list which stores pubid along with relid. Here tables list acts as a
> temporary list to get filter_partitions and then delete the
> published_rel from table_infos. Will it be possible to directly
> operate on table_infos list and remove the temporary tables list used.
> We might have to implement comparator, deduplication functions and
> change filter_partitions function to work directly on published_rel
> type list.
> +                       /
> +                        * Record the published table and the
> corresponding publication so
> +                        * that we can get row filters and column list later.
> +                        *
> +                        * When a table is published by multiple
> publications, to obtain
> +                        * all row filters and column list, the
> structure related to this
> +                        * table will be recorded multiple times.
> +                        */
> +                       foreach(lc, pub_elem_tables)
> +                       {
> +                               published_rel *table_info =
> (published_rel *) malloc(sizeof(published_rel));
> +
> +                               table_info->relid = lfirst_oid(lc);
> +                               table_info->pubid = pub_elem->oid;
> +                               table_infos = lappend(table_infos, table_info);
> +                       }
> +
> +                       tables = list_concat(tables, pub_elem_tables);
> 
> Thoughts?

I think we could only deduplicate published tables per publication to get all
row filters and column lists for each published table later.
I removed the temporary list 'tables' and modified the API of the function
filter_partitions to handle published_rel type list.

Attach the new patch set.

Regards,
Wang wei

Commits

  1. Avoid syncing data twice for the 'publish_via_partition_root' option.

  2. Fix partition table's REPLICA IDENTITY checking on the subscriber.

  3. Prohibit combining publications with different column lists.

  4. Fix double publish of child table's data.

  5. Support adding partitioned tables to publication