Re: row filtering for logical replication
Amit Kapila <amit.kapila16@gmail.com>
Commits
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Release cache tuple when no longer needed
- ed0fbc8e5ac9 15.0 landed
-
Add some additional tests for row filters in logical replication.
- ceb57afd3ce1 15.0 landed
-
Fix one of the tests introduced in commit 52e4f0cd47.
- cfb4e209ec15 15.0 landed
-
Allow specifying row filters for logical replication of tables.
- 52e4f0cd472d 15.0 landed
-
Move scanint8() to numutils.c
- cfc7191dfea3 15.0 cited
-
Replace Test::More plans with done_testing
- 549ec201d613 15.0 cited
-
Reduce relcache access in WAL sender streaming logical changes
- 6ce16088bfed 15.0 cited
-
Small cleanups related to PUBLICATION framework code
- c9105dd3660f 15.0 cited
-
Add a view to show the stats of subscription workers.
- 8d74fc96db5f 15.0 cited
-
Allow publishing the tables of schema.
- 5a2832465fd8 15.0 cited
-
Doc: improve documentation of CREATE/ALTER SUBSCRIPTION.
- 1882d6cca161 15.0 cited
-
Add PublicationTable and PublicationRelInfo structs
- 0c6828fa987b 15.0 cited
-
Remove unused argument "txn" in maybe_send_schema().
- 93d573d86571 15.0 cited
-
Add prepare API support for streaming transactions in logical replication.
- 63cf61cdeb7b 15.0 cited
-
Unify PostgresNode's new() and get_new_node() methods
- 201a76183e20 15.0 cited
-
Use l*_node() family of functions where appropriate
- 2b00db4fb0c7 15.0 cited
-
Add support for prepared transactions to built-in logical replication.
- a8fd13cab0ba 15.0 cited
-
Restore the portal-level snapshot after procedure COMMIT/ROLLBACK.
- ef9480509622 11.13 cited
-
Rename a parse node to be more general
- 91d1f2d30210 14.0 landed
-
Remove unused column atttypmod from initial tablesync query
- 4ad31bb2ef25 14.0 landed
-
SEARCH and CYCLE clauses
- 3696a600e229 14.0 cited
On Mon, Jul 12, 2021 at 7:19 AM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> Hi
>
> Andres complained about the safety of doing general expression
> evaluation in pgoutput; that was first in
>
> https://postgr.es/m/20210128022032.eq2qqc6zxkqn5syt@alap3.anarazel.de
> where he described a possible approach to handle it by restricting
> expressions to have limited shape; and later in
> http://postgr.es/m/20210331191710.kqbiwe73lur7jo2e@alap3.anarazel.de
>
> I was just scanning the patch trying to see if some sort of protection
> had been added for this, but I couldn't find anything. (Some functions
> are under-commented, though). So, is it there already, and if so what
> is it?
>
I think the patch is trying to prohibit arbitrary expressions in the
WHERE clause via
transformWhereClause(..EXPR_KIND_PUBLICATION_WHERE..). You can notice
that at various places the expressions are prohibited via
EXPR_KIND_PUBLICATION_WHERE. I am not sure that the checks are correct
and sufficient but I think there is some attempt to do it. For
example, the below sort of ad-hoc check for func_call doesn't seem to
be good idea.
@@ -119,6 +119,13 @@ transformExprRecurse(ParseState *pstate, Node *expr)
/* Guard against stack overflow due to overly complex expressions */
check_stack_depth();
+ /* Functions are not allowed in publication WHERE clauses */
+ if (pstate->p_expr_kind == EXPR_KIND_PUBLICATION_WHERE &&
nodeTag(expr) == T_FuncCall)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("functions are not allowed in publication WHERE expressions"),
+ parser_errposition(pstate, exprLocation(expr))));
Now, the other idea I had in mind was to traverse the WHERE clause
expression in publication_add_relation and identify if it contains
anything other than the ANDed list of 'foo.bar op constant'
expressions. OTOH, for index where clause expressions or policy check
expressions, we use a technique similar to what we have in the patch
to prohibit certain kinds of expressions.
Do you have any preference on how this should be addressed?
--
With Regards,
Amit Kapila.