Re: row filtering for logical replication
Amit Kapila <amit.kapila16@gmail.com>
Commits
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Release cache tuple when no longer needed
- ed0fbc8e5ac9 15.0 landed
-
Add some additional tests for row filters in logical replication.
- ceb57afd3ce1 15.0 landed
-
Fix one of the tests introduced in commit 52e4f0cd47.
- cfb4e209ec15 15.0 landed
-
Allow specifying row filters for logical replication of tables.
- 52e4f0cd472d 15.0 landed
-
Move scanint8() to numutils.c
- cfc7191dfea3 15.0 cited
-
Replace Test::More plans with done_testing
- 549ec201d613 15.0 cited
-
Reduce relcache access in WAL sender streaming logical changes
- 6ce16088bfed 15.0 cited
-
Small cleanups related to PUBLICATION framework code
- c9105dd3660f 15.0 cited
-
Add a view to show the stats of subscription workers.
- 8d74fc96db5f 15.0 cited
-
Allow publishing the tables of schema.
- 5a2832465fd8 15.0 cited
-
Doc: improve documentation of CREATE/ALTER SUBSCRIPTION.
- 1882d6cca161 15.0 cited
-
Add PublicationTable and PublicationRelInfo structs
- 0c6828fa987b 15.0 cited
-
Remove unused argument "txn" in maybe_send_schema().
- 93d573d86571 15.0 cited
-
Add prepare API support for streaming transactions in logical replication.
- 63cf61cdeb7b 15.0 cited
-
Unify PostgresNode's new() and get_new_node() methods
- 201a76183e20 15.0 cited
-
Use l*_node() family of functions where appropriate
- 2b00db4fb0c7 15.0 cited
-
Add support for prepared transactions to built-in logical replication.
- a8fd13cab0ba 15.0 cited
-
Restore the portal-level snapshot after procedure COMMIT/ROLLBACK.
- ef9480509622 11.13 cited
-
Rename a parse node to be more general
- 91d1f2d30210 14.0 landed
-
Remove unused column atttypmod from initial tablesync query
- 4ad31bb2ef25 14.0 landed
-
SEARCH and CYCLE clauses
- 3696a600e229 14.0 cited
On Mon, Sep 20, 2021 at 3:17 PM Ajin Cherian <itsajin@gmail.com> wrote: > > On Wed, Sep 8, 2021 at 7:59 PM Ajin Cherian <itsajin@gmail.com> wrote: > > > > On Wed, Sep 1, 2021 at 9:23 PM Euler Taveira <euler@eulerto.com> wrote: > > > > > > Somehow this approach of either new_tuple or old_tuple doesn't seem to > > be very fruitful if the user requires that his replica is up-to-date > > based on the filter condition. For that, I think you will need to > > convert UPDATES to either INSERTS or DELETES if only new_tuple or > > old_tuple matches the filter condition but not both matches the filter > > condition. > > > > UPDATE > > old-row (match) new-row (no match) -> DELETE > > old-row (no match) new row (match) -> INSERT > > old-row (match) new row (match) -> UPDATE > > old-row (no match) new-row (no match) -> (drop change) > > > > Adding a patch that strives to do the logic that I described above. > For updates, the row filter is applied on both old_tuple > and new_tuple. This patch assumes that the row filter only uses > columns that are part of the REPLICA IDENTITY. (the current patch-set > only > restricts this for row-filters that are delete only) > The old_tuple only has columns that are part of the old_tuple and have > been changed, which is a problem while applying the row-filter. Since > unchanged REPLICA IDENTITY columns > are not present in the old_tuple, this patch creates a temporary > old_tuple by getting such column values from the new_tuple and then > applies the filter on this hand-created temp old_tuple. The way the > old_tuple is created can be better optimised in future versions. > Yeah, this is the kind of idea which can work. One thing you might want to check is the overhead of the additional deform/form cycle. You might want to use Peter's tests above. I think you need to only form old/new tuples when you have changed something in it but on a quick look, it seems you are always re-forming both the tuples. -- With Regards, Amit Kapila.