Horton
Menu

Re: Column Filtering in Logical Replication

Peter Smith <smithpb2250@gmail.com>

View thread
From: Peter Smith <smithpb2250@gmail.com>
To: vignesh C <vignesh21@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2022-08-26T02:02:57Z
Lists: pgsql-hackers

Attachments

On Thu, Aug 25, 2022 at 7:38 PM vignesh C <vignesh21@gmail.com> wrote:
>
...
> > PSA the v3* patch set.
>
> Thanks for the updated patch.
> Few comments:
> 1) We can shuffle the columns in publisher table and subscriber to
> show that the order of the column does not matter
> +   <para>
> +    Create a publication <literal>p1</literal>. A column list is defined for
> +    table <literal>t1</literal> to reduce the number of columns that will be
> +    replicated.
> +<programlisting>
> +test_pub=# CREATE PUBLICATION p1 FOR TABLE t1 (id, a, b, c);
> +CREATE PUBLICATION
> +test_pub=#
> +</programlisting></para>
>

OK. I made the following changes to the example.
- now the subscriber table defines cols in a different order than that
of the publisher table
- now the publisher column list defines col names in a different order
than that of the table
- now the column list avoids using only adjacent column names

> 2) We can try to keep the line content to less than 80 chars
> +   <para>
> +    A column list is specified per table following the tablename, and
> enclosed by
> +    parenthesis. See <xref linkend="sql-createpublication"/> for details.
> +   </para>
>

OK. Modified to use < 80 chars

> 3) tablename should be table name like it is used in other places
> +   <para>
> +    A column list is specified per table following the tablename, and
> enclosed by
> +    parenthesis. See <xref linkend="sql-createpublication"/> for details.
> +   </para>
>

Sorry, I don't see this problem. AFAIK this same issue was already
fixed in the v3* patches.  Notice in the cited fragment that
'parenthesis' is misspelt but that was also fixed in v3. Maybe you are
looking at an old patch file (??)

> 4a) In the below, you could include mentioning "Only the column list
> data of publication <literal>p1</literal> are replicated."
> +    <para>
> +     Insert some rows to table <literal>t1</literal>.
> +<programlisting>
> +test_pub=# INSERT INTO t1 VALUES(1, 'a-1', 'b-1', 'c-1', 'd-1', 'e-1');
> +INSERT 0 1
>

OK. Modified to say this.

> 4b) In the above, we could mention that the insert should be done on
> the "publisher side" as the previous statements are executed on the
> subscriber side.

OK. Modified to say this.

~~~

Thanks for the feedback.

PSA patch set v4* where all of the above comments are now addressed.

------
Kind Regards,
Peter Smith.
Fujitsu Australia

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Doc: Explain about Column List feature.

  2. Doc: fix column list vs. replica identity rules.

  3. Prohibit combining publications with different column lists.

  4. Fix the check to limit sync workers.

  5. Wait for subscription to sync in t/031_column_list.sql

  6. Move prattrs to the pg_publication_rel section in docs

  7. Allow specifying column lists for logical replication

  8. Fix row filters with multiple publications

  9. Fix publish_as_relid with multiple publications

  10. Add some additional tests for row filters in logical replication.

  11. Add index on pg_publication_rel.prpubid

  12. Avoid using DefElemAction in AlterPublicationStmt

  13. Small cleanups related to PUBLICATION framework code

  14. Add PublicationTable and PublicationRelInfo structs

  15. Fix various concurrency issues in logical replication worker launching