Horton
Menu

Re: Column Filtering in Logical Replication

Tomas Vondra <tomas.vondra@enterprisedb.com>

View thread
From: Tomas Vondra <tomas.vondra@enterprisedb.com>
To: Alvaro Herrera <alvherre@alvh.no-ip.org>, Rahila Syed <rahilasyed90@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Peter Smith <smithpb2250@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2021-12-14T19:28:53Z
Lists: pgsql-hackers 
On 12/14/21 17:43, Alvaro Herrera wrote:
> On 2021-Dec-13, Alvaro Herrera wrote:
> 
>> I think this means we need a new OBJECT_PUBLICATION_REL_COLUMN value in
>> the ObjectType (paralelling OBJECT_COLUMN), and no new ObjectClass
>> value.  Looking now to confirm.
> 
> After working on this a little bit more, I realized that this is a bad
> idea overall.  It causes lots of complications and it's just not worth
> it.  So I'm back at my original thought that we need to throw an ERROR
> at ALTER TABLE .. DROP COLUMN time if the column is part of a
> replication column filter, and suggest the user to remove the column
> from the filter first and reattempt the DROP COLUMN.
> 
> This means that we need to support changing the column list of a table
> in a publication.  I'm looking at implementing some form of ALTER
> PUBLICATION for that.
> 

Yeah. I think it's not clear if this should behave more like an index or 
a view. When an indexed column gets dropped we simply drop the index. 
But if you drop a column referenced by a view, we fail with an error. I 
think we should handle this more like a view, because publications are 
externally visible objects too (while indexes are pretty much just an 
implementation detail).

But why would it be easier not to add new object type? We still need to 
check there is no publication referencing the column - either you do 
that automatically through a dependency, or you do that by custom code. 
Using a dependency seems better to me, but I don't know what are the 
complications you mentioned.

regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Doc: Explain about Column List feature.

  2. Doc: fix column list vs. replica identity rules.

  3. Prohibit combining publications with different column lists.

  4. Fix the check to limit sync workers.

  5. Wait for subscription to sync in t/031_column_list.sql

  6. Move prattrs to the pg_publication_rel section in docs

  7. Allow specifying column lists for logical replication

  8. Fix row filters with multiple publications

  9. Fix publish_as_relid with multiple publications

  10. Add some additional tests for row filters in logical replication.

  11. Add index on pg_publication_rel.prpubid

  12. Avoid using DefElemAction in AlterPublicationStmt

  13. Small cleanups related to PUBLICATION framework code

  14. Add PublicationTable and PublicationRelInfo structs

  15. Fix various concurrency issues in logical replication worker launching