Horton
Menu

Re: Column Filtering in Logical Replication

Peter Eisentraut <peter.eisentraut@enterprisedb.com>

View thread
From: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
To: Alvaro Herrera <alvherre@alvh.no-ip.org>, Rahila Syed <rahilasyed90@gmail.com>
Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2021-12-20T13:53:46Z
Lists: pgsql-hackers 
On 17.12.21 22:07, Alvaro Herrera wrote:
> So I've been thinking about this as a "security" item (you can see my
> comments to that effect sprinkled all over this thread), in the sense
> that if a publication "hides" some column, then the replica just won't
> get access to it.  But in reality that's mistaken: the filtering that
> this patch implements is done based on the queries that *the replica*
> executes at its own volition; if the replica decides to ignore the list
> of columns, it'll be able to get all columns.  All it takes is an
> uncooperative replica in order for the lot of data to be exposed anyway.

During normal replication, the publisher should only send the columns 
that are configured to be part of the publication.  So I don't see a 
problem there.

During the initial table sync, the subscriber indeed can construct any 
COPY command.  We could maybe replace this with a more customized COPY 
command variant, like COPY table OF publication TO STDOUT.

But right now the subscriber is sort of assumed to have access to 
everything on the publisher anyway, so I doubt that this is the only 
problem.  But it's worth considering.

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Doc: Explain about Column List feature.

  2. Doc: fix column list vs. replica identity rules.

  3. Prohibit combining publications with different column lists.

  4. Fix the check to limit sync workers.

  5. Wait for subscription to sync in t/031_column_list.sql

  6. Move prattrs to the pg_publication_rel section in docs

  7. Allow specifying column lists for logical replication

  8. Fix row filters with multiple publications

  9. Fix publish_as_relid with multiple publications

  10. Add some additional tests for row filters in logical replication.

  11. Add index on pg_publication_rel.prpubid

  12. Avoid using DefElemAction in AlterPublicationStmt

  13. Small cleanups related to PUBLICATION framework code

  14. Add PublicationTable and PublicationRelInfo structs

  15. Fix various concurrency issues in logical replication worker launching