Horton
Menu

Re: Column Filtering in Logical Replication

Alvaro Herrera <alvherre@alvh.no-ip.org>

View thread
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Peter Eisentraut <peter.eisentraut@enterprisedb.com>, Rahila Syed <rahilasyed90@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2022-01-06T12:29:31Z
Lists: pgsql-hackers 
On 2022-Jan-06, Amit Kapila wrote:

> Considering this, don't we need to deal with "For All Tables" and "For
> All Tables In Schema .." Publications in this query? The row filter
> patch deal with such cases. The row filter patch handles the NULL case
> via C code which makes the query relatively simpler.

Yes.  I realized after sending that email that the need to handle schema
publications would make a single query very difficult, so I ended up
splitting it again in two queries, which is what you see in the latest
version submitted.

> I am not sure if the same logic can be used here but having a simple
> query here have merit that if we want to use a single query to fetch
> both column and row filters then we should be able to enhance it
> without making it further complicated.

I have looked the row filter code a couple of times to make sure we're
somewhat compatible, but didn't look closely enough to see if we can
make the queries added by both patches into a single one.

> Shouldn't we try to have a behavior similar to the row filter patch
> for this case? The row filter patch behavior is as follows: "If your
> publication contains a partitioned table, the publication parameter
> publish_via_partition_root determines if it uses the partition row
> filter (if the parameter is false, the default) or the root
> partitioned table row filter. During initial tablesync, it doesn't do
> any special handling for partitions.

I'll have a look.

Thanks for looking!

-- 
Álvaro Herrera              Valdivia, Chile  —  https://www.EnterpriseDB.com/

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Doc: Explain about Column List feature.

  2. Doc: fix column list vs. replica identity rules.

  3. Prohibit combining publications with different column lists.

  4. Fix the check to limit sync workers.

  5. Wait for subscription to sync in t/031_column_list.sql

  6. Move prattrs to the pg_publication_rel section in docs

  7. Allow specifying column lists for logical replication

  8. Fix row filters with multiple publications

  9. Fix publish_as_relid with multiple publications

  10. Add some additional tests for row filters in logical replication.

  11. Add index on pg_publication_rel.prpubid

  12. Avoid using DefElemAction in AlterPublicationStmt

  13. Small cleanups related to PUBLICATION framework code

  14. Add PublicationTable and PublicationRelInfo structs

  15. Fix various concurrency issues in logical replication worker launching