Horton
Menu

Re: Column Filtering in Logical Replication

vignesh C <vignesh21@gmail.com>

View thread
From: vignesh C <vignesh21@gmail.com>
To: Peter Smith <smithpb2250@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2022-08-16T17:04:26Z
Lists: pgsql-hackers 
On Mon, Aug 8, 2022 at 2:08 PM Peter Smith <smithpb2250@gmail.com> wrote:
>
> PSA patch version v1* for a new "Column Lists" pgdocs section
>
> This is just a first draft, but I wanted to post it as-is, with the
> hope that I can get some feedback while continuing to work on it.

Few comments:
1) Row filters mentions that "It has no effect on TRUNCATE commands.",
the same is not present in case of column filters. We should keep the
changes similarly for consistency.
--- a/doc/src/sgml/ref/create_publication.sgml
+++ b/doc/src/sgml/ref/create_publication.sgml
@@ -90,8 +90,7 @@ CREATE PUBLICATION <replaceable
class="parameter">name</replaceable>
      <para>
       When a column list is specified, only the named columns are replicated.
       If no column list is specified, all columns of the table are replicated
-      through this publication, including any columns added later.  If a column
-      list is specified, it must include the replica identity columns.
+      through this publication, including any columns added later.

2) The document says that "if the table uses REPLICA IDENTITY FULL,
specifying a column list is not allowed.":
+   publishes only <command>INSERT</command> operations. Furthermore, if the
+   table uses <literal>REPLICA IDENTITY FULL</literal>, specifying a column
+   list is not allowed.
+  </para>

Did you mean specifying a column list during create publication for
REPLICA IDENTITY FULL table like below scenario:
postgres=# create table t2(c1 int, c2 int, c3 int);
CREATE TABLE
postgres=# alter table t2 replica identity full ;
ALTER TABLE
postgres=# create publication pub1 for table t2(c1,c2);
CREATE PUBLICATION

If so, the document says specifying column list is not allowed, but
creating a publication with column list on replica identity full was
successful.

Regards,
Vignesh

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Doc: Explain about Column List feature.

  2. Doc: fix column list vs. replica identity rules.

  3. Prohibit combining publications with different column lists.

  4. Fix the check to limit sync workers.

  5. Wait for subscription to sync in t/031_column_list.sql

  6. Move prattrs to the pg_publication_rel section in docs

  7. Allow specifying column lists for logical replication

  8. Fix row filters with multiple publications

  9. Fix publish_as_relid with multiple publications

  10. Add some additional tests for row filters in logical replication.

  11. Add index on pg_publication_rel.prpubid

  12. Avoid using DefElemAction in AlterPublicationStmt

  13. Small cleanups related to PUBLICATION framework code

  14. Add PublicationTable and PublicationRelInfo structs

  15. Fix various concurrency issues in logical replication worker launching