Horton
Menu

Re: Column Filtering in Logical Replication

Peter Eisentraut <peter.eisentraut@enterprisedb.com>

View thread
From: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
To: Tomas Vondra <tomas.vondra@enterprisedb.com>, Amit Kapila <amit.kapila16@gmail.com>
Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>, Justin Pryzby <pryzby@telsasoft.com>, "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>, Rahila Syed <rahilasyed90@gmail.com>, Peter Smith <smithpb2250@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, shiy.fnst@fujitsu.com
Date: 2022-03-09T09:20:57Z
Lists: pgsql-hackers

Attachments

On 07.03.22 16:18, Tomas Vondra wrote:
> AFAICS these issues should be resolved by the adoption of the row-filter
> approach (i.e. it should fail the same way as for row filter).

The first two patches (additional testing for row filtering feature) 
look okay to me.

Attached is a fixup patch for your main feature patch (the third one).

It's a bit of code and documentation cleanup, but mainly I removed the 
term "column filter" from the patch.  Half the code was using "column 
list" or similar and half the code "column filter", which was confusing. 
  Also, there seemed to be a bit of copy-and-pasting from row-filter 
code going on, with some code comments not quite sensible, so I rewrote 
some of them.  Also some code used "rf" and "cf" symbols which were a 
bit hard to tell apart.  A few more letters can increase readability.

Note in publicationcmds.c OpenTableList() the wrong if condition was used.

I'm still confused about the intended replica identity handling.  This 
patch still checks whether the column list contains the replica identity 
at DDL time.  And then it also checks at execution time.  I thought the 
latest understanding was that the DDL-time checking would be removed.  I 
think it's basically useless now, since as the test cases show, you can 
subvert those checks by altering the replica identity later.

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Doc: Explain about Column List feature.

  2. Doc: fix column list vs. replica identity rules.

  3. Prohibit combining publications with different column lists.

  4. Fix the check to limit sync workers.

  5. Wait for subscription to sync in t/031_column_list.sql

  6. Move prattrs to the pg_publication_rel section in docs

  7. Allow specifying column lists for logical replication

  8. Fix row filters with multiple publications

  9. Fix publish_as_relid with multiple publications

  10. Add some additional tests for row filters in logical replication.

  11. Add index on pg_publication_rel.prpubid

  12. Avoid using DefElemAction in AlterPublicationStmt

  13. Small cleanups related to PUBLICATION framework code

  14. Add PublicationTable and PublicationRelInfo structs

  15. Fix various concurrency issues in logical replication worker launching