Re: row filtering for logical replication

Euler Taveira <euler@eulerto.com>

From: "Euler Taveira" <euler@eulerto.com>
To: "Amit Kapila" <amit.kapila16@gmail.com>, "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>
Cc: "Peter Smith" <smithpb2250@gmail.com>, "Alvaro Herrera" <alvherre@alvh.no-ip.org>, "Greg Nancarrow" <gregn4422@gmail.com>, "vignesh C" <vignesh21@gmail.com>, "Ajin Cherian" <itsajin@gmail.com>, "tanghy.fnst@fujitsu.com" <tanghy.fnst@fujitsu.com>, "Dilip Kumar" <dilipbalaut@gmail.com>, "Rahila Syed" <rahilasyed90@gmail.com>, "Peter Eisentraut" <peter.eisentraut@enterprisedb.com>, Önder Kalacı <onderkalaci@gmail.com>, japin <japinli@hotmail.com>, "Michael Paquier" <michael@paquier.xyz>, "David Steele" <david@pgmasters.net>, "Craig Ringer" <craig@2ndquadrant.com>, "Amit Langote" <amitlangote09@gmail.com>, "PostgreSQL Hackers" <pgsql-hackers@lists.postgresql.org>
Date: 2022-02-21T23:16:33Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →
  1. Release cache tuple when no longer needed

  2. Add some additional tests for row filters in logical replication.

  3. Fix one of the tests introduced in commit 52e4f0cd47.

  4. Allow specifying row filters for logical replication of tables.

  5. Move scanint8() to numutils.c

  6. Replace Test::More plans with done_testing

  7. Reduce relcache access in WAL sender streaming logical changes

  8. Small cleanups related to PUBLICATION framework code

  9. Add a view to show the stats of subscription workers.

  10. Allow publishing the tables of schema.

  11. Doc: improve documentation of CREATE/ALTER SUBSCRIPTION.

  12. Add PublicationTable and PublicationRelInfo structs

  13. Remove unused argument "txn" in maybe_send_schema().

  14. Add prepare API support for streaming transactions in logical replication.

  15. Unify PostgresNode's new() and get_new_node() methods

  16. Use l*_node() family of functions where appropriate

  17. Add support for prepared transactions to built-in logical replication.

  18. Restore the portal-level snapshot after procedure COMMIT/ROLLBACK.

  19. Rename a parse node to be more general

  20. Remove unused column atttypmod from initial tablesync query

  21. SEARCH and CYCLE clauses

On Thu, Feb 17, 2022, at 3:36 AM, Amit Kapila wrote:
> As there is a new version, I would like to wait for a few more days
> before committing. I am planning to commit this early next week (by
> Tuesday) unless others or I see any more things that can be improved.
Amit, I don't have additional comments or suggestions. Let's move on. Next
topic. :-)

> I would once like to mention the replica identity handling of the
> patch. Right now, (on HEAD) we are not checking the replica identity
> combination at DDL time, they are checked at execution time in
> CheckCmdReplicaIdentity(). This patch follows the same scheme and
> gives an error at the time of update/delete if the table publishes
> update/delete and the publication(s) has a row filter that contains
> non-replica-identity columns. We had earlier thought of handling it at
> DDL time but that won't follow the existing scheme and has a lot of
> complications as explained in emails [1][2]. Do let me know if you see
> any problem here?
IMO it is not an issue that this patch needs to solve. The conclusion of
checking the RI at the DDL time vs execution time is that:

* the current patch just follows the same pattern used in the current logical
  replication implementation;
* it is easier to check during execution time (a central point) versus a lot of
  combinations for DDL commands;
* the check during DDL time might eventually break if new subcommands are
  added;
* the execution time does not have the maintenance burden imposed by new DDL
  subcommands;
* we might change the RI check to execute at DDL time if the current
  implementation imposes a significant penalty in certain workloads.

Again, it is material for another patch.

Thanks for taking care of a feature that has been discussed for 4 years [1].

[1] https://www.postgresql.org/message-id/CAHE3wggb715X%2BmK_DitLXF25B%3DjE6xyNCH4YOwM860JR7HarGQ%40mail.gmail.com


--
Euler Taveira
EDB   https://www.enterprisedb.com/