Re: row filtering for logical replication
Amit Kapila <amit.kapila16@gmail.com>
Commits
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Release cache tuple when no longer needed
- ed0fbc8e5ac9 15.0 landed
-
Add some additional tests for row filters in logical replication.
- ceb57afd3ce1 15.0 landed
-
Fix one of the tests introduced in commit 52e4f0cd47.
- cfb4e209ec15 15.0 landed
-
Allow specifying row filters for logical replication of tables.
- 52e4f0cd472d 15.0 landed
-
Move scanint8() to numutils.c
- cfc7191dfea3 15.0 cited
-
Replace Test::More plans with done_testing
- 549ec201d613 15.0 cited
-
Reduce relcache access in WAL sender streaming logical changes
- 6ce16088bfed 15.0 cited
-
Small cleanups related to PUBLICATION framework code
- c9105dd3660f 15.0 cited
-
Add a view to show the stats of subscription workers.
- 8d74fc96db5f 15.0 cited
-
Allow publishing the tables of schema.
- 5a2832465fd8 15.0 cited
-
Doc: improve documentation of CREATE/ALTER SUBSCRIPTION.
- 1882d6cca161 15.0 cited
-
Add PublicationTable and PublicationRelInfo structs
- 0c6828fa987b 15.0 cited
-
Remove unused argument "txn" in maybe_send_schema().
- 93d573d86571 15.0 cited
-
Add prepare API support for streaming transactions in logical replication.
- 63cf61cdeb7b 15.0 cited
-
Unify PostgresNode's new() and get_new_node() methods
- 201a76183e20 15.0 cited
-
Use l*_node() family of functions where appropriate
- 2b00db4fb0c7 15.0 cited
-
Add support for prepared transactions to built-in logical replication.
- a8fd13cab0ba 15.0 cited
-
Restore the portal-level snapshot after procedure COMMIT/ROLLBACK.
- ef9480509622 11.13 cited
-
Rename a parse node to be more general
- 91d1f2d30210 14.0 landed
-
Remove unused column atttypmod from initial tablesync query
- 4ad31bb2ef25 14.0 landed
-
SEARCH and CYCLE clauses
- 3696a600e229 14.0 cited
On Tue, Dec 7, 2021 at 6:31 PM Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> wrote: > > On Tue, Dec 7, 2021 at 12:18 PM tanghy.fnst@fujitsu.com > <tanghy.fnst@fujitsu.com> wrote: > > > > I have another problem with your patch. The document says: > > > > ... If the subscription has several publications in > > + which the same table has been published with different filters, those > > + expressions get OR'ed together so that rows satisfying any of the expressions > > + will be replicated. Notice this means if one of the publications has no filter > > + at all then all other filters become redundant. > > > > Then, what if one of the publications is specified as 'FOR ALL TABLES' or 'FOR > > ALL TABLES IN SCHEMA'. > > > > For example: > > create table tbl (a int primary key);" > > create publication p1 for table tbl where (a > 10); > > create publication p2 for all tables; > > create subscription sub connection 'dbname=postgres port=5432' publication p1, p2; > > Thanks for the example. I was wondering about this case myself. > I think we should handle this case. > > > > I think for "FOR ALL TABLE" publication(p2 in my case), table tbl should be > > treated as no filter, and table tbl should have no filter in subscription sub. Thoughts? > > > > But for now, the filter(a > 10) works both when copying initial data and later changes. > > > > To fix it, I think we can check if the table is published in a 'FOR ALL TABLES' > > publication or published as part of schema in function pgoutput_row_filter_init > > (which was introduced in v44-0003 patch), also we need to make some changes in > > tablesync.c. > > In order to check "FOR ALL_TABLES", we might need to fetch publication > metadata. > Do we really need to perform a separate fetch for this? In get_rel_sync_entry(), we already have this information, can't we someway stash that in the corresponding RelationSyncEntry so that same can be used later for row filtering. > Instead of that can we add a "TRUE" filter on all the tables > which are part of FOR ALL TABLES publication? > How? We won't have an entry for such tables in pg_publication_rel where we store row_filter information. -- With Regards, Amit Kapila.