Horton
Menu

Re: Column Filtering in Logical Replication

Alvaro Herrera <alvherre@alvh.no-ip.org>

View thread
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
To: Rahila Syed <rahilasyed90@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Tomas Vondra <tomas.vondra@enterprisedb.com>, Peter Smith <smithpb2250@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2021-12-17T15:58:50Z
Lists: pgsql-hackers 
On 2021-Dec-17, Rahila Syed wrote:

> > This means that we need to support changing the column list of a
> > table in a publication.  I'm looking at implementing some form of
> > ALTER PUBLICATION for that.
>
> I think right now the patch contains support only for ALTER
> PUBLICATION..  ADD TABLE with column filters.  In order to achieve
> changing the column lists of a published table, I think we can extend
> the ALTER TABLE ..SET TABLE syntax to support specification of column
> list.

Yeah, that's what I was thinking too.

> > So this whole thing can be reduced to just this:
> 
> > if (att_map != NULL && !bms_is_member(att->attnum, att_map))
> >        continue;        /* that is, don't send this attribute */
> 
> I agree the condition can be shortened now. The long if condition was
> included because initially the feature allowed specifying filters
> without replica identity columns(sent those columns internally without
> user having to specify).

Ah, true, I had forgotten that.  Thanks.

> >  900 +        * the table is partitioned.  Run a recursive query to iterate through all
> >  901 +        * the parents of the partition and retreive the record for the parent
> >  902 +        * that exists in pg_publication_rel.
> >  903 +        */
> 
> The above comment in fetch_remote_table_info() can be changed as the
> recursive query is no longer used.

Oh, of course.

I'll finish some loose ends and submit a v10, but it's still not final.

-- 
Álvaro Herrera           39°49'30"S 73°17'W  —  https://www.EnterpriseDB.com/
"Right now the sectors on the hard disk run clockwise, but I heard a rumor that
you can squeeze 0.2% more throughput by running them counterclockwise.
It's worth the effort. Recommended."  (Gerry Pourwelle)

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Doc: Explain about Column List feature.

  2. Doc: fix column list vs. replica identity rules.

  3. Prohibit combining publications with different column lists.

  4. Fix the check to limit sync workers.

  5. Wait for subscription to sync in t/031_column_list.sql

  6. Move prattrs to the pg_publication_rel section in docs

  7. Allow specifying column lists for logical replication

  8. Fix row filters with multiple publications

  9. Fix publish_as_relid with multiple publications

  10. Add some additional tests for row filters in logical replication.

  11. Add index on pg_publication_rel.prpubid

  12. Avoid using DefElemAction in AlterPublicationStmt

  13. Small cleanups related to PUBLICATION framework code

  14. Add PublicationTable and PublicationRelInfo structs

  15. Fix various concurrency issues in logical replication worker launching