RE: row filtering for logical replication
Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
Commits
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Release cache tuple when no longer needed
- ed0fbc8e5ac9 15.0 landed
-
Add some additional tests for row filters in logical replication.
- ceb57afd3ce1 15.0 landed
-
Fix one of the tests introduced in commit 52e4f0cd47.
- cfb4e209ec15 15.0 landed
-
Allow specifying row filters for logical replication of tables.
- 52e4f0cd472d 15.0 landed
-
Move scanint8() to numutils.c
- cfc7191dfea3 15.0 cited
-
Replace Test::More plans with done_testing
- 549ec201d613 15.0 cited
-
Reduce relcache access in WAL sender streaming logical changes
- 6ce16088bfed 15.0 cited
-
Small cleanups related to PUBLICATION framework code
- c9105dd3660f 15.0 cited
-
Add a view to show the stats of subscription workers.
- 8d74fc96db5f 15.0 cited
-
Allow publishing the tables of schema.
- 5a2832465fd8 15.0 cited
-
Doc: improve documentation of CREATE/ALTER SUBSCRIPTION.
- 1882d6cca161 15.0 cited
-
Add PublicationTable and PublicationRelInfo structs
- 0c6828fa987b 15.0 cited
-
Remove unused argument "txn" in maybe_send_schema().
- 93d573d86571 15.0 cited
-
Add prepare API support for streaming transactions in logical replication.
- 63cf61cdeb7b 15.0 cited
-
Unify PostgresNode's new() and get_new_node() methods
- 201a76183e20 15.0 cited
-
Use l*_node() family of functions where appropriate
- 2b00db4fb0c7 15.0 cited
-
Add support for prepared transactions to built-in logical replication.
- a8fd13cab0ba 15.0 cited
-
Restore the portal-level snapshot after procedure COMMIT/ROLLBACK.
- ef9480509622 11.13 cited
-
Rename a parse node to be more general
- 91d1f2d30210 14.0 landed
-
Remove unused column atttypmod from initial tablesync query
- 4ad31bb2ef25 14.0 landed
-
SEARCH and CYCLE clauses
- 3696a600e229 14.0 cited
On Wed, Nov 10, 2021 10:48 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
> On Tue, Nov 9, 2021 at 2:22 PM houzj.fnst@fujitsu.com wrote:
> >
> > On Fri, Nov 5, 2021 4:49 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > > On Fri, Nov 5, 2021 at 10:44 AM Peter Smith <smithpb2250@gmail.com> wrote:
> > > >
> > > > PSA new set of v37* patches.
> > > 3.
> > > - | ColId
> > > + | ColId OptWhereClause
> > > {
> > > $$ = makeNode(PublicationObjSpec);
> > > $$->pubobjtype = PUBLICATIONOBJ_CONTINUATION;
> > > - $$->name = $1;
> > > + if ($2)
> > > + {
> > > + $$->pubtable = makeNode(PublicationTable); $$->pubtable->relation
> > > + = makeRangeVar(NULL, $1, @1); $$->pubtable->whereClause = $2; }
> > > + else { $$->name = $1; }
> > >
> > > Again this doesn't appear to be the right way. I think this should
> > > be handled at a later point.
> >
> > I think the difficulty to handle this at a later point is that we need
> > to make sure we don't lose the whereclause. Currently, we can only
> > save the whereclause in PublicationTable structure and the
> > PublicationTable is only used for TABLE, but '| ColId' can be used for
> > either a SCHEMA or TABLE. We cannot distinguish the actual type at
> > this stage, so we always need to save the whereclause if it's NOT NULL.
> >
>
> I see your point. But, I think we can add some comments here indicating that
> the user might have mistakenly given where clause with some schema which we
> will identify later and give an appropriate error. Then, in
> preprocess_pubobj_list(), identify if the user has given the where clause with
> schema name and give an appropriate error.
>
OK, IIRC, in this approach, we need to set both $$->name and $$->pubtable in
'| ColId OptWhereClause'. And In preprocess_pubobj_list, we can add some check
if both name and pubtable is NOT NULL.
the grammar code could be:
| ColId OptWhereClause
{
$$ = makeNode(PublicationObjSpec);
$$->pubobjtype = PUBLICATIONOBJ_CONTINUATION;
$$->name = $1;
+ /* xxx */
+ $$->pubtable = makeNode(PublicationTable);
+ $$->pubtable->relation = makeRangeVar(NULL, $1, @1);
+ $$->pubtable->whereClause = $2;
$$->location = @1;
}
preprocess_pubobj_list
...
else if (pubobj->pubobjtype == PUBLICATIONOBJ_REL_IN_SCHEMA ||
pubobj->pubobjtype == PUBLICATIONOBJ_CURRSCHEMA)
{
...
+ if (pubobj->name &&
+ (!pubobj->pubtable || !pubobj->pubtable->whereClause))
pubobj->pubobjtype = PUBLICATIONOBJ_REL_IN_SCHEMA;
else if (!pubobj->name && !pubobj->pubtable)
pubobj->pubobjtype = PUBLICATIONOBJ_CURRSCHEMA;
else
ereport(ERROR,
errcode(ERRCODE_SYNTAX_ERROR),
errmsg("invalid schema name at or near"),
parser_errposition(pubobj->location));
}
Best regards,
Hou zj