Re: Column Filtering in Logical Replication
Amit Kapila <amit.kapila16@gmail.com>
On Sat, Sep 25, 2021 at 1:15 PM vignesh C <vignesh21@gmail.com> wrote:
>
> On Fri, Sep 24, 2021 at 6:55 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> >
> > On 2021-Sep-23, Amit Kapila wrote:
> >
> > > Alvaro, do you have any thoughts on these proposed grammar changes?
> >
> > Yeah, I think pubobj_name remains a problem in that you don't know its
> > return type -- could be a String or a RangeVar, and the user of that
> > production can't distinguish. So you're still (unnecessarily, IMV)
> > stashing an object of undetermined type into ->object.
> >
> > I think you should get rid of both pubobj_name and pubobj_expr and do
> > somethine like this:
> >
> > /* FOR TABLE and FOR ALL TABLES IN SCHEMA specifications */
> > PublicationObjSpec: TABLE ColId
> > {
> > $$ = makeNode(PublicationObjSpec);
> > $$->pubobjtype = PUBLICATIONOBJ_TABLE;
> > $$->rangevar = makeRangeVarFromQualifiedName($1, NULL, @1, yyscanner);
> > }
> > | TABLE ColId indirection
> > {
> > $$ = makeNode(PublicationObjSpec);
> > $$->pubobjtype = PUBLICATIONOBJ_TABLE;
> > $$->rangevar = makeRangeVarFromQualifiedName($1, $2, @1, yyscanner);
> > }
> > | ALL TABLES IN_P SCHEMA ColId
> > {
> > $$ = makeNode(PublicationObjSpec);
> > $$->pubobjtype = PUBLICATIONOBJ_REL_IN_SCHEMA;
> > $$->name = $4;
> > }
> > | ALL TABLES IN_P SCHEMA CURRENT_SCHEMA /* XXX should this be "IN_P CURRENT_SCHEMA"? */
> > {
> > $$ = makeNode(PublicationObjSpec);
> > $$->pubobjtype = PUBLICATIONOBJ_CURRSCHEMA;
> > $$->name = $4;
> > }
> > | ColId
> > {
> > $$ = makeNode(PublicationObjSpec);
> > $$->name = $1;
> > $$->pubobjtype = PUBLICATIONOBJ_CONTINUATION;
> > }
> > | ColId indirection
> > {
> > $$ = makeNode(PublicationObjSpec);
> > $$->rangevar = makeRangeVarFromQualifiedName($1, $2, @1, yyscanner);
> > $$->pubobjtype = PUBLICATIONOBJ_CONTINUATION;
> > }
> > | CURRENT_SCHEMA
> > {
> > $$ = makeNode(PublicationObjSpec);
> > $$->pubobjtype = PUBLICATIONOBJ_CURRSCHEMA;
> > }
> > ;
>
> Apart from the issue that Hou San pointed, I found one issue with
> introduction of PUBLICATIONOBJ_CURRSCHEMA, I was not able to
> differentiate if it is table or schema in the following cases:
> CREATE PUBLICATION pub1 FOR ALL TABLES IN SCHEMA CURRENT_SCHEMA;
> CREATE PUBLICATION pub1 FOR ALL TABLES IN SCHEMA sch1, CURRENT_SCHEMA;
> CREATE PUBLICATION pub1 FOR table t1, CURRENT_SCHEMA;
> The differentiation is required to differentiate and add a schema or a table.
>
I am not sure what makes you say that we can't distinguish the above
cases when there is already a separate rule for CURRENT_SCHEMA? I
think you can distinguish by tracking the previous objects as we are
already doing in the patch. But one thing that is not clear to me is
is the reason to introduce a new type PUBLICATIONOBJ_CURRSCHEMA when
we use PUBLICATIONOBJ_REL_IN_SCHEMA and PUBLICATIONOBJ_CONTINUATION to
distinguish all cases of CURRENT_SCHEMA. Alvaro might have something
in mind for this which is not apparent and that might have caused
confusion to you as well?
--
With Regards,
Amit Kapila.
Commits
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Doc: Explain about Column List feature.
- f98d07424523 16.0 landed
- 25996a82a446 15.0 landed
-
Doc: fix column list vs. replica identity rules.
- 376af686111e 16.0 landed
- ab3131880d8a 15.0 landed
-
Prohibit combining publications with different column lists.
- fd0b9dcebda7 15.0 cited
-
Fix the check to limit sync workers.
- be46985bed8a 10.21 landed
- a90de822e483 11.16 landed
- 59348fbdeb7b 12.11 landed
- 82d4a17a1750 13.7 landed
- c9dea58e2702 14.3 landed
- dd4ab6fd6528 15.0 landed
-
Wait for subscription to sync in t/031_column_list.sql
- 404f49338fef 15.0 landed
-
Move prattrs to the pg_publication_rel section in docs
- 41b00f8e601f 15.0 landed
-
Allow specifying column lists for logical replication
- 923def9a533a 15.0 landed
-
Fix row filters with multiple publications
- 5a079662256e 15.0 landed
-
Fix publish_as_relid with multiple publications
- 27fafee72d17 13.7 landed
- 677a1dc0ca0f 14.3 landed
- c91f71b9dc91 15.0 landed
-
Add some additional tests for row filters in logical replication.
- ceb57afd3ce1 15.0 cited
-
Add index on pg_publication_rel.prpubid
- 025b920a3d45 15.0 landed
-
Avoid using DefElemAction in AlterPublicationStmt
- 9623d8999603 15.0 landed
-
Small cleanups related to PUBLICATION framework code
- c9105dd3660f 15.0 landed
-
Add PublicationTable and PublicationRelInfo structs
- 0c6828fa987b 15.0 landed
-
Fix various concurrency issues in logical replication worker launching
- de4389712206 10.0 cited