Re: bogus: logical replication rows/cols combinations

Peter Eisentraut <peter.eisentraut@enterprisedb.com>

From: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
To: Tomas Vondra <tomas.vondra@enterprisedb.com>, Amit Kapila <amit.kapila16@gmail.com>
Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>, Pg Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2022-05-02T20:34:48Z
Lists: pgsql-hackers
On 01.05.22 23:42, Tomas Vondra wrote:
> Imagine have a table with customers from different regions, and you want
> to replicate the data somewhere else, but for some reason you can only
> replicate details for one particular region, and subset of columns for
> everyone else. So you'd do something like this:
> 
> CREATE PUBLICATION p1 FOR TABLE customers (... all columns ...)
>   WHERE region = 'USA';
> 
> CREATE PUBLICATION p1 FOR TABLE customers (... subset of columns ...)
>   WHERE region != 'USA';
> 
> I think ignoring the row filters and just merging the column lists makes
> no sense for this use case.

I'm thinking now the underlying problem is that we shouldn't combine 
column lists at all.  Examples like the above where you want to redact 
values somehow are better addressed with something like triggers or an 
actual "column filter" that works dynamically or some other mechanism.

The main purpose, in my mind, of column lists is if the tables 
statically have different shapes on publisher and subscriber.  Perhaps 
for space reasons or regulatory reasons you don't want to replicate 
everything.  But then it doesn't make sense to combine column lists.  If 
you decide over here that the subscriber table has this shape and over 
there that the subscriber table has that other shape, then the 
combination of the two will be a table that has neither shape and so 
will not work for anything.

I think in general we should be much more restrictive in how we combine 
publications.  Unless we are really sure it makes sense, we should 
disallow it.  Users can always make a new publication with different 
settings and subscribe to that directly.



Commits

  1. Prohibit combining publications with different column lists.

  2. Extend pg_publication_tables to display column list and row filter.

  3. Remove dynamic_shared_memory_type=none