Horton
Menu

Re: Column Filtering in Logical Replication

Tomas Vondra <tomas.vondra@enterprisedb.com>

View thread
From: Tomas Vondra <tomas.vondra@enterprisedb.com>
To: Alvaro Herrera <alvherre@alvh.no-ip.org>
Cc: Rahila Syed <rahilasyed90@gmail.com>, Amit Kapila <amit.kapila16@gmail.com>, Peter Smith <smithpb2250@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2021-12-14T21:13:49Z
Lists: pgsql-hackers 

On 12/14/21 20:35, Alvaro Herrera wrote:
> On 2021-Dec-14, Tomas Vondra wrote:
> 
>> Yeah. I think it's not clear if this should behave more like an index or a
>> view. When an indexed column gets dropped we simply drop the index. But if
>> you drop a column referenced by a view, we fail with an error. I think we
>> should handle this more like a view, because publications are externally
>> visible objects too (while indexes are pretty much just an implementation
>> detail).
> 
> I agree -- I think it's more like a view than like an index.  (The
> original proposal was that if you dropped a column that was part of the
> column list of a relation in a publication, the entire relation is
> dropped from the view, but that doesn't seem very friendly behavior --
> you break the replication stream immediately if you do that, and the
> only way to fix it is to send a fresh copy of the remaining subset of
> columns.)
> 

Right, that's my reasoning too.

>> But why would it be easier not to add new object type? We still need to
>> check there is no publication referencing the column - either you do that
>> automatically through a dependency, or you do that by custom code. Using a
>> dependency seems better to me, but I don't know what are the complications
>> you mentioned.
> 
> The problem is that we need a way to represent the object "column of a
> table in a publication".  I found myself adding a lot of additional code
> to support OBJECT_PUBLICATION_REL_COLUMN and that seemed like too much.
> 

My experience with dependencies is pretty limited, but can't we simply 
make a dependency between the whole publication and the column?

regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Doc: Explain about Column List feature.

  2. Doc: fix column list vs. replica identity rules.

  3. Prohibit combining publications with different column lists.

  4. Fix the check to limit sync workers.

  5. Wait for subscription to sync in t/031_column_list.sql

  6. Move prattrs to the pg_publication_rel section in docs

  7. Allow specifying column lists for logical replication

  8. Fix row filters with multiple publications

  9. Fix publish_as_relid with multiple publications

  10. Add some additional tests for row filters in logical replication.

  11. Add index on pg_publication_rel.prpubid

  12. Avoid using DefElemAction in AlterPublicationStmt

  13. Small cleanups related to PUBLICATION framework code

  14. Add PublicationTable and PublicationRelInfo structs

  15. Fix various concurrency issues in logical replication worker launching