RE: Replica Identity check of partition table on subscriber
Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
From: "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Amit Langote <amitlangote09@gmail.com>, "shiy.fnst@fujitsu.com" <shiy.fnst@fujitsu.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2022-06-13T07:33:18Z
Lists: pgsql-hackers
Attachments
- v4-0004-fix-memory-leak-about-attrmap.patch (application/octet-stream) patch v4-0004
- v4-0001-Fix-partition-map-cache-invalidation-on-subscriber.patch (application/octet-stream) patch v4-0001
- v4-0003-Check-partition-table-replica-identity-on-subscriber.patch (application/octet-stream) patch v4-0003
- v4-0002-Reset-partition-map-cache-when-receiving-new-relatio.patch (application/octet-stream) patch v4-0002
On Monday, June 13, 2022 1:53 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Sat, Jun 11, 2022 at 2:36 PM houzj.fnst@fujitsu.com
> <houzj.fnst@fujitsu.com> wrote:
> >
> > On Saturday, June 11, 2022 9:36 AM Amit Kapila <amit.kapila16@gmail.com>
> wrote:
> > >
> > > On Fri, Jun 10, 2022 at 2:26 PM Amit Langote
> > > <amitlangote09@gmail.com>
> > > wrote:
> > > >
> > > > +logicalrep_partmap_invalidate
> > > >
> > > > I wonder why not call this logicalrep_partmap_update() to go with
> > > > logicalrep_relmap_update()? It seems confusing to have
> > > > logicalrep_partmap_invalidate() right next to
> > > > logicalrep_partmap_invalidate_cb().
> > > >
> > >
> > > I am thinking about why we need to update the relmap in this new
> > > function logicalrep_partmap_invalidate()? I think it may be better
> > > to do it in
> > > logicalrep_partition_open() when actually required, otherwise, we
> > > end up doing a lot of work that may not be of use unless the
> > > corresponding partition is accessed. Also, it seems awkward to me
> > > that we do the same thing in this new function
> > > logicalrep_partmap_invalidate() and then also in
> > > logicalrep_partition_open() under different conditions.
> > >
> > > One more point about the 0001, it doesn't seem to have a test that
> > > validates
> > > logicalrep_partmap_invalidate_cb() functionality. I think for that
> > > we need to Alter the local table (table on the subscriber side). Can we try to
> write a test for it?
> >
> >
> > Thanks for Amit. L and Amit. K for your comments ! I agree with this point.
> > Here is the version patch set which try to address all these comments.
> >
> > In addition, when reviewing the code, I found some other related
> > problems in the code.
> >
> > 1)
> > entry->attrmap = make_attrmap(map->maplen);
> > for (attno = 0; attno < entry->attrmap->maplen; attno++)
> > {
> > AttrNumber root_attno =
> map->attnums[attno];
> >
> > entry->attrmap->attnums[attno] =
> attrmap->attnums[root_attno - 1];
> > }
> >
> > In this case, It’s possible that 'attno' points to a dropped column in
> > which case the root_attno would be '0'. I think in this case we should
> > just set the
> > entry->attrmap->attnums[attno] to '-1' instead of accessing the
> > attrmap->attnums[]. I included this change in 0001 because the
> > attrmap->testcase which
> > can reproduce these problems are related(we need to ALTER the
> > partition on subscriber to reproduce it).
> >
>
> Hmm, this appears to be a different issue. Can we separate out the bug-fix
> code for the subscriber-side issue caused by the DDL on the subscriber?
>
> Few other comments:
> + * Note that we don't update the remoterel information in the entry
> +here,
> + * we will update the information in logicalrep_partition_open to save
> + * unnecessary work.
> + */
> +void
> +logicalrep_partmap_invalidate(LogicalRepRelation *remoterel)
>
> /to save/to avoid
>
> Also, I agree with Amit L. that it is confusing to have
> logicalrep_partmap_invalidate() right next to
> logicalrep_partmap_invalidate_cb() and both have somewhat different kinds of
> logic. So, we can either name it as
> logicalrep_partmap_reset_relmap() or logicalrep_partmap_update() unless you
> have any other better suggestions? Accordingly, change the comment atop
> this function.
Thanks for the comments.
I have separated out the bug-fix for the subscriber-side.
And fix the typo and function name.
Attach the new version patch set.
Best regards,
Hou zj
Commits
-
Fix memory leak due to LogicalRepRelMapEntry.attrmap.
- ac0e2d387a04 15.0 landed
- 3238b5c330bc 14.5 landed
- 3a6ef0cdf311 13.8 landed
- 9e0d9a24ef88 12.12 landed
- ed2a7a6bf67d 11.17 landed
- d873b5a5a340 10.22 landed
-
Fix stale values in partition map entries on subscribers.
- 75bfe7434d41 15.0 landed
- f0022a77d011 14.5 landed
- 419c72715172 13.8 landed
-
Fix partition table's REPLICA IDENTITY checking on the subscriber.
- 26b3455afaa9 15.0 landed
- 5f113d60e998 13.8 landed
- 52d5ea9adb01 14.5 landed
-
Fix data inconsistency between publisher and subscriber.
- b7658c24c7c1 15.0 landed
- 0980adfd4d02 14.5 landed
- 1f9a7738eb84 13.8 landed
-
Fix cache look-up failures while applying changes in logical replication.
- 16f5a8da76f2 13.8 landed
- d457cb4e8a5e 14.5 landed
- 5a97b1325453 15.0 landed
-
Add logical replication support to replicate into partitioned tables
- f1ac27bfda6c 13.0 cited