Re: Replica Identity check of partition table on subscriber
Amit Kapila <amit.kapila16@gmail.com>
From: Amit Kapila <amit.kapila16@gmail.com>
To: Amit Langote <amitlangote09@gmail.com>
Cc: "shiy.fnst@fujitsu.com" <shiy.fnst@fujitsu.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>
Date: 2022-06-14T12:56:51Z
Lists: pgsql-hackers
Attachments
- v7-0001-Fix-cache-look-up-failures-while-applying-changes.patch (application/octet-stream) patch v7-0001
- v7-0002-Fix-data-inconsistency-between-publisher-and-subs.patch (application/octet-stream) patch v7-0002
On Tue, Jun 14, 2022 at 1:02 PM Amit Langote <amitlangote09@gmail.com> wrote:
>
> On Tue, Jun 14, 2022 at 3:31 PM Amit Langote <amitlangote09@gmail.com> wrote:
> > On Mon, Jun 13, 2022 at 6:14 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > > I think we can do that way as well but do you see any benefit in it?
> > > The way I am suggesting will avoid the effort of updating the remote
> > > rel copy till we try to access that particular partition.
> >
> > I don't see any benefit as such to doing it the way the patch does,
> > it's just that that seems to be the only way to go given the way
> > things are.
>
> Oh, I see that v4-0002 has this:
>
> +/*
> + * Reset the entries in the partition map that refer to remoterel
> + *
> + * Called when new relation mapping is sent by the publisher to update our
> + * expected view of incoming data from said publisher.
> + *
> + * Note that we don't update the remoterel information in the entry here,
> + * we will update the information in logicalrep_partition_open to avoid
> + * unnecessary work.
> + */
> +void
> +logicalrep_partmap_reset_relmap(LogicalRepRelation *remoterel)
> +{
> + HASH_SEQ_STATUS status;
> + LogicalRepPartMapEntry *part_entry;
> + LogicalRepRelMapEntry *entry;
> +
> + if (LogicalRepPartMap == NULL)
> + return;
> +
> + hash_seq_init(&status, LogicalRepPartMap);
> + while ((part_entry = (LogicalRepPartMapEntry *)
> hash_seq_search(&status)) != NULL)
> + {
> + entry = &part_entry->relmapentry;
> +
> + if (entry->remoterel.remoteid != remoterel->remoteid)
> + continue;
> +
> + logicalrep_relmap_free_entry(entry);
> +
> + memset(entry, 0, sizeof(LogicalRepRelMapEntry));
> + }
> +}
>
> The previous versions would also call logicalrep_relmap_update() on
> the entry after the memset, which is no longer done, so that is indeed
> saving useless work. I also see that both logicalrep_relmap_update()
> and the above function basically invalidate the whole
> LogicalRepRelMapEntry before setting the new remote relation info so
> that the next logicaprep_rel_open() or logicalrep_partition_open()
> have to refill the other members too.
>
> Though, I thought maybe you were saying that we shouldn't need this
> function for resetting partitions in the first place, which I guess
> you weren't.
>
Right.
> v4-0002 looks good btw, except the bitpick about test comment similar
> to my earlier comment regarding v5-0001:
>
> +# Change the column order of table on publisher
>
> I think it might be better to say something specific to describe the
> test intent, like:
>
> Test that replication into partitioned target table continues to works
> correctly when the published table is altered
>
Okay changed this and slightly modify the comments and commit message.
I am just attaching the HEAD patches for the first two issues.
--
With Regards,
Amit Kapila.
Commits
-
Fix memory leak due to LogicalRepRelMapEntry.attrmap.
- ac0e2d387a04 15.0 landed
- 3238b5c330bc 14.5 landed
- 3a6ef0cdf311 13.8 landed
- 9e0d9a24ef88 12.12 landed
- ed2a7a6bf67d 11.17 landed
- d873b5a5a340 10.22 landed
-
Fix stale values in partition map entries on subscribers.
- 75bfe7434d41 15.0 landed
- f0022a77d011 14.5 landed
- 419c72715172 13.8 landed
-
Fix partition table's REPLICA IDENTITY checking on the subscriber.
- 26b3455afaa9 15.0 landed
- 5f113d60e998 13.8 landed
- 52d5ea9adb01 14.5 landed
-
Fix data inconsistency between publisher and subscriber.
- b7658c24c7c1 15.0 landed
- 0980adfd4d02 14.5 landed
- 1f9a7738eb84 13.8 landed
-
Fix cache look-up failures while applying changes in logical replication.
- 16f5a8da76f2 13.8 landed
- d457cb4e8a5e 14.5 landed
- 5a97b1325453 15.0 landed
-
Add logical replication support to replicate into partitioned tables
- f1ac27bfda6c 13.0 cited