Horton
Menu

Re: Column Filtering in Logical Replication

Alvaro Herrera <alvherre@alvh.no-ip.org>

View thread
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Tomas Vondra <tomas.vondra@enterprisedb.com>, "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, Justin Pryzby <pryzby@telsasoft.com>, Rahila Syed <rahilasyed90@gmail.com>, Peter Smith <smithpb2250@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, "shiy.fnst@fujitsu.com" <shiy.fnst@fujitsu.com>
Date: 2022-03-23T10:21:23Z
Lists: pgsql-hackers 
On 2022-Mar-23, Amit Kapila wrote:

> On Wed, Mar 23, 2022 at 12:54 AM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> >
> > On 2022-Mar-19, Tomas Vondra wrote:
> >
> > > @@ -174,7 +182,13 @@ ALTER PUBLICATION noinsert SET (publish = 'update, delete');
> > >    <para>
> > >     Add some tables to the publication:
> > >  <programlisting>
> > > -ALTER PUBLICATION mypublication ADD TABLE users, departments;
> > > +ALTER PUBLICATION mypublication ADD TABLE users (user_id, firstname), departments;
> > > +</programlisting></para>
> > > +
> > > +  <para>
> > > +   Change the set of columns published for a table:
> > > +<programlisting>
> > > +ALTER PUBLICATION mypublication SET TABLE users (user_id, firstname, lastname), TABLE departments;
> > >  </programlisting></para>
> > >
> > >    <para>
> >
> > Hmm, it seems to me that if you've removed the feature to change the set
> > of columns published for a table, then the second example should be
> > removed as well.
> 
> As per my understanding, the removed feature is "Alter Publication ...
> Alter Table ...". The example here "Alter Publication ... Set Table
> .." should still work as mentioned in my email[1].

Ah, I see.  Yeah, that makes sense.  In that case, the leading text
seems a bit confusing.  I would suggest "Change the set of tables in the
publication, specifying a different set of columns for one of them:"

I think it would make the example more useful if we table for which the
columns are changing is a different one.  Maybe do this:

    Add some tables to the publication:
 <programlisting>
-ALTER PUBLICATION mypublication ADD TABLE users, departments;
+ALTER PUBLICATION mypublication ADD TABLE users (user_id, firstname), departments;
+</programlisting></para>
+
+  <para>
+   Change the set of tables in the publication, keeping the column list
+   in the users table and specifying a different column list for the
+   departments table. Note that previously published tables not mentioned
+   in this command are removed from the publication:
+
+<programlisting>
+ALTER PUBLICATION mypublication SET TABLE users (user_id, firstname), TABLE departments (dept_id, deptname);
 </programlisting></para>

so that it is clear that if you want to keep the column list unchanged
in one table, you are forced to specify it again.

(Frankly, this ALTER PUBLICATION SET command seems pretty useless from a
user PoV.)

-- 
Álvaro Herrera         PostgreSQL Developer  —  https://www.EnterpriseDB.com/
"Investigación es lo que hago cuando no sé lo que estoy haciendo"
(Wernher von Braun)

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Doc: Explain about Column List feature.

  2. Doc: fix column list vs. replica identity rules.

  3. Prohibit combining publications with different column lists.

  4. Fix the check to limit sync workers.

  5. Wait for subscription to sync in t/031_column_list.sql

  6. Move prattrs to the pg_publication_rel section in docs

  7. Allow specifying column lists for logical replication

  8. Fix row filters with multiple publications

  9. Fix publish_as_relid with multiple publications

  10. Add some additional tests for row filters in logical replication.

  11. Add index on pg_publication_rel.prpubid

  12. Avoid using DefElemAction in AlterPublicationStmt

  13. Small cleanups related to PUBLICATION framework code

  14. Add PublicationTable and PublicationRelInfo structs

  15. Fix various concurrency issues in logical replication worker launching