Horton
Menu

Re: Column Filtering in Logical Replication

Alvaro Herrera <alvherre@alvh.no-ip.org>

View thread
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
To: Rahila Syed <rahilasyed90@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Tomas Vondra <tomas.vondra@enterprisedb.com>, Peter Smith <smithpb2250@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2021-12-13T17:44:11Z
Lists: pgsql-hackers

Attachments

On 2021-Dec-10, Alvaro Herrera wrote:

> Actually it's not so easy to implement.

So I needed to add "sub object id" support for pg_publication_rel
objects in pg_depend / dependency.c.  What I have now is partial (the
describe routines need patched) but it's sufficient to show what's
needed.  In essence, we now set these depend entries with column
numbers, so that they can be dropped independently; when the drop comes,
the existing pg_publication_rel row is modified to cover the remaining
columns.  As far as I can tell, it works correctly.

There is one policy decision to make: what if ALTER TABLE drops the last
remaining column in the publication?  I opted to raise a specific error
in this case, though we could just the same opt to drop the relation
from the publication.  Are there opinions on this?

This version incorporates the fixups Peter submitted, plus some other
fixes of my own.  Notably, as Peter also mentioned, I changed
pg_publication_rel.prattrs to store int2vector rather than an array of
column names.  This makes for better behavior if columns are renamed and
things like that, and also we don't need to be so cautious about
quoting.  It does mean we need a slightly more complicated query in a
couple of spots, but that should be okay.

-- 
Álvaro Herrera           39°49'30"S 73°17'W  —  https://www.EnterpriseDB.com/
"Always assume the user will do much worse than the stupidest thing
you can imagine."                                (Julien PUYDT)

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Doc: Explain about Column List feature.

  2. Doc: fix column list vs. replica identity rules.

  3. Prohibit combining publications with different column lists.

  4. Fix the check to limit sync workers.

  5. Wait for subscription to sync in t/031_column_list.sql

  6. Move prattrs to the pg_publication_rel section in docs

  7. Allow specifying column lists for logical replication

  8. Fix row filters with multiple publications

  9. Fix publish_as_relid with multiple publications

  10. Add some additional tests for row filters in logical replication.

  11. Add index on pg_publication_rel.prpubid

  12. Avoid using DefElemAction in AlterPublicationStmt

  13. Small cleanups related to PUBLICATION framework code

  14. Add PublicationTable and PublicationRelInfo structs

  15. Fix various concurrency issues in logical replication worker launching