Horton
Menu

Re: Column Filtering in Logical Replication

vignesh C <vignesh21@gmail.com>

View thread
From: vignesh C <vignesh21@gmail.com>
To: Alvaro Herrera <alvherre@alvh.no-ip.org>
Cc: Peter Smith <smithpb2250@gmail.com>, Rahila Syed <rahilasyed90@gmail.com>, Amit Kapila <amit.kapila16@gmail.com>, Tomas Vondra <tomas.vondra@enterprisedb.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2021-12-03T09:17:39Z
Lists: pgsql-hackers 
On Fri, Dec 3, 2021 at 12:45 AM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> On 2021-Sep-16, Peter Smith wrote:
>
> > I noticed that the latest v5 no longer includes the TAP test which was
> > in the v4 patch.
> >
> > (src/test/subscription/t/021_column_filter.pl)
> >
> > Was that omission deliberate?
>
> Somehow I not only failed to notice the omission, but also your email
> where you told us about it.  I have since posted a version of the patch
> that again includes it.

Thanks for the patch, Few comments:
I had a look at the patch, I felt the following should be handled:
1) Dump changes to include the column filters while adding table to
publication in dumpPublicationTable
2) Documentation changes for column filtering in create_publication.sgml
3) describe publication changes to support \dRp command in describePublications
4) I felt we need not allow specifying columns in case of "alter
publication drop table" as currently dropping column filter is not
allowed.
5) We should check if the column specified is present in the table,
currently we are able to specify non existent column for column
filtering
+       foreach(lc, targetrel->columns)
+       {
+               char       *colname;
+
+               colname = strVal(lfirst(lc));
+               target_cols = lappend(target_cols, colname);
+       }
+       check_publication_add_relation(targetrel->relation, target_cols);

Regards,
Vignesh

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Doc: Explain about Column List feature.

  2. Doc: fix column list vs. replica identity rules.

  3. Prohibit combining publications with different column lists.

  4. Fix the check to limit sync workers.

  5. Wait for subscription to sync in t/031_column_list.sql

  6. Move prattrs to the pg_publication_rel section in docs

  7. Allow specifying column lists for logical replication

  8. Fix row filters with multiple publications

  9. Fix publish_as_relid with multiple publications

  10. Add some additional tests for row filters in logical replication.

  11. Add index on pg_publication_rel.prpubid

  12. Avoid using DefElemAction in AlterPublicationStmt

  13. Small cleanups related to PUBLICATION framework code

  14. Add PublicationTable and PublicationRelInfo structs

  15. Fix various concurrency issues in logical replication worker launching