Re: row filtering for logical replication

Japin Li <japinli@hotmail.com>

From: japin <japinli@hotmail.com>

To: Euler Taveira <euler@eulerto.com>

Cc: Önder Kalacı <onderkalaci@gmail.com>, David Steele <david@pgmasters.net>, Craig Ringer <craig@2ndquadrant.com>, Tomas Vondra <tomas.vondra@2ndquadrant.com>, Michael Paquier <michael@paquier.xyz>, Amit Langote <amitlangote09@gmail.com>, movead li <movead.li@highgo.ca>, pgsql-hackers@lists.postgresql.org

Date: 2021-02-02T05:37:31Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Release cache tuple when no longer needed
- ed0fbc8e5ac9 15.0 landed
Add some additional tests for row filters in logical replication.
- ceb57afd3ce1 15.0 landed
Fix one of the tests introduced in commit 52e4f0cd47.
- cfb4e209ec15 15.0 landed
Allow specifying row filters for logical replication of tables.
- 52e4f0cd472d 15.0 landed
Move scanint8() to numutils.c
- cfc7191dfea3 15.0 cited
Replace Test::More plans with done_testing
- 549ec201d613 15.0 cited
Reduce relcache access in WAL sender streaming logical changes
- 6ce16088bfed 15.0 cited
Small cleanups related to PUBLICATION framework code
- c9105dd3660f 15.0 cited
Add a view to show the stats of subscription workers.
- 8d74fc96db5f 15.0 cited
Allow publishing the tables of schema.
- 5a2832465fd8 15.0 cited
Doc: improve documentation of CREATE/ALTER SUBSCRIPTION.
- 1882d6cca161 15.0 cited
Add PublicationTable and PublicationRelInfo structs
- 0c6828fa987b 15.0 cited
Remove unused argument "txn" in maybe_send_schema().
- 93d573d86571 15.0 cited
Add prepare API support for streaming transactions in logical replication.
- 63cf61cdeb7b 15.0 cited
Unify PostgresNode's new() and get_new_node() methods
- 201a76183e20 15.0 cited
Use l*_node() family of functions where appropriate
- 2b00db4fb0c7 15.0 cited
Add support for prepared transactions to built-in logical replication.
- a8fd13cab0ba 15.0 cited
Restore the portal-level snapshot after procedure COMMIT/ROLLBACK.
- ef9480509622 11.13 cited
Rename a parse node to be more general
- 91d1f2d30210 14.0 landed
Remove unused column atttypmod from initial tablesync query
- 4ad31bb2ef25 14.0 landed
SEARCH and CYCLE clauses
- 3696a600e229 14.0 cited

On Tue, 02 Feb 2021 at 03:11, Euler Taveira <euler@eulerto.com> wrote:
> On Mon, Feb 1, 2021, at 6:11 AM, japin wrote:
>> Thanks for updating the patch.  Here are some comments:
> Thanks for your review. I updated the documentation accordingly.
>
>> The documentation says:
>> 
>> >  Columns used in the <literal>WHERE</literal> clause must be part of the
>> >  primary key or be covered by <literal>REPLICA IDENTITY</literal> otherwise
>> >  <command>UPDATE</command> and <command>DELETE</command> operations will not
>> >  be replicated.
> The UPDATE is an oversight from a previous version.
>
>> 
>> Does the publication only load the REPLICA IDENTITY columns into oldtuple when we
>> execute DELETE? So the pgoutput_row_filter() cannot find non REPLICA IDENTITY
>> columns, which cause it return false, right?  If that's right, the UPDATE might
>> not be limitation by REPLICA IDENTITY, because all columns are in newtuple,
>> isn't it?
> No. oldtuple could possibly be available for UPDATE and DELETE. However, row
> filter consider only one tuple for filtering. INSERT has only newtuple; row
> filter uses it.  UPDATE has newtuple and optionally oldtuple (if it has PK or
> REPLICA IDENTITY); row filter uses newtuple. DELETE optionally has only
> oldtuple; row filter uses it (if available). Keep in mind, if the expression
> evaluates to NULL, it returns false and the row won't be replicated.
>

Thanks for your clarification.

-- 
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.