RE: Data is copied twice when specifying both child and parent table in publication
Wei Wang (Fujitsu) <wangw.fnst@fujitsu.com>
From: "wangw.fnst@fujitsu.com" <wangw.fnst@fujitsu.com>
To: vignesh C <vignesh21@gmail.com>
Cc: Peter Smith <smithpb2250@gmail.com>, "Takamichi Osumi (Fujitsu)" <osumi.takamichi@fujitsu.com>, Amit Kapila <amit.kapila16@gmail.com>, "shiy.fnst@fujitsu.com" <shiy.fnst@fujitsu.com>, "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>, Amit Langote <amitlangote09@gmail.com>, Peter
Eisentraut <peter.eisentraut@enterprisedb.com>, "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>, Dilip Kumar <dilipbalaut@gmail.com>, Greg Nancarrow <gregn4422@gmail.com>
Date: 2022-11-16T08:58:31Z
Lists: pgsql-hackers
Attachments
- HEAD_v15-0001-Fix-data-replicated-twice-when-specifying-publis.patch (application/octet-stream) patch v15-0001
- HEAD_v15-0002-Add-clarification-for-the-behaviour-of-the-publi.patch (application/octet-stream) patch v15-0002
- REL14_v15-0001-Fix-data-replicated-twice-when-specifying-publis_patch (application/octet-stream)
- REL15_v15-0001-Fix-data-replicated-twice-when-specifying-publis_patch (application/octet-stream)
On Mon, Nov 14, 2022 at 0:56 AM vignesh C <vignesh21@gmail.com> wrote:
> >
> > Attach new patches.
>
Thanks for your comments.
> Here we are having tables list to store the relids and table_infos
> list which stores pubid along with relid. Here tables list acts as a
> temporary list to get filter_partitions and then delete the
> published_rel from table_infos. Will it be possible to directly
> operate on table_infos list and remove the temporary tables list used.
> We might have to implement comparator, deduplication functions and
> change filter_partitions function to work directly on published_rel
> type list.
> + /
> + * Record the published table and the
> corresponding publication so
> + * that we can get row filters and column list later.
> + *
> + * When a table is published by multiple
> publications, to obtain
> + * all row filters and column list, the
> structure related to this
> + * table will be recorded multiple times.
> + */
> + foreach(lc, pub_elem_tables)
> + {
> + published_rel *table_info =
> (published_rel *) malloc(sizeof(published_rel));
> +
> + table_info->relid = lfirst_oid(lc);
> + table_info->pubid = pub_elem->oid;
> + table_infos = lappend(table_infos, table_info);
> + }
> +
> + tables = list_concat(tables, pub_elem_tables);
>
> Thoughts?
I think we could only deduplicate published tables per publication to get all
row filters and column lists for each published table later.
I removed the temporary list 'tables' and modified the API of the function
filter_partitions to handle published_rel type list.
Attach the new patch set.
Regards,
Wang wei
Commits
-
Avoid syncing data twice for the 'publish_via_partition_root' option.
- 062a84442424 16.0 landed
-
Fix partition table's REPLICA IDENTITY checking on the subscriber.
- 52d5ea9adb01 14.5 cited
-
Prohibit combining publications with different column lists.
- fd0b9dcebda7 15.0 cited
-
Fix double publish of child table's data.
- 3f06c00cf6dc 13.6 landed
- 614b77d65a38 14.2 landed
- 5e97905a2c76 15.0 landed
-
Support adding partitioned tables to publication
- 17b9e7f9fe23 13.0 cited