v14-0003-Skip-publishing-the-columns-specified-in-FOR-TAB.patch
application/octet-stream
Filename: v14-0003-Skip-publishing-the-columns-specified-in-FOR-TAB.patch
Type: application/octet-stream
Part: 1
Patch
Same data as JSON:
GET /api/v1/attachments/:id/patch
the parsed metadata as JSON — format, series position, per-file stats; never the diff bytes.
API reference →
Format: format-patch
Series: patch v14-0003
Subject: Skip publishing the columns specified in FOR TABLE EXCEPT
| File | + | − |
|---|---|---|
| doc/src/sgml/catalogs.sgml | 4 | 1 |
| doc/src/sgml/logical-replication.sgml | 72 | 25 |
| doc/src/sgml/ref/alter_publication.sgml | 9 | 1 |
| doc/src/sgml/ref/create_publication.sgml | 16 | 1 |
| src/backend/catalog/pg_publication.c | 80 | 12 |
| src/backend/commands/publicationcmds.c | 28 | 7 |
| src/backend/parser/gram.y | 65 | 0 |
| src/backend/replication/pgoutput/pgoutput.c | 46 | 6 |
| src/bin/pg_dump/pg_dump.c | 11 | 5 |
| src/bin/pg_dump/pg_dump.h | 1 | 0 |
| src/bin/psql/describe.c | 59 | 26 |
| src/include/catalog/pg_publication.h | 7 | 3 |
| src/include/catalog/pg_publication_rel.h | 3 | 2 |
| src/test/regress/expected/publication.out | 67 | 0 |
| src/test/regress/sql/publication.sql | 46 | 0 |
| src/test/subscription/t/036_rep_changes_except_table.pl | 93 | 1 |
From 176ad491a5ea6d7eee8220a7e9810503445d8443 Mon Sep 17 00:00:00 2001
From: Shlok Kyal <shlok.kyal.oss@gmail.com>
Date: Tue, 24 Jun 2025 01:35:55 +0530
Subject: [PATCH v14 3/3] Skip publishing the columns specified in FOR TABLE
EXCEPT
A new "TABLE table_name EXCEPT (column_list)" clause for CREATE/ALTER
PUBLICATION allows one or more columns to be excluded. The publisher
will not send the data of excluded columns to the subscriber.
The new syntax allows specifying excluded column list when creating or
altering a publication. For example:
CREATE PUBLICATION pubname FOR TABLE tabname EXCEPT (col1, col2, col3)
or
ALTER PUBLICATION pubname ADD TABLE tabname EXCEPT (col1, col2, col3)
Column list specifed with EXCEPT is stored in column "prattrs" in table
"pg_publication_rel" and also column "prexcept" is set to "true", to maintain
the column list that user wants to exclude from the publication.
pg_dump is updated to identify and dump the excluded column list of the
publication.
The psql \d family of commands can now display excluded column list. e.g.
psql \dRp+ variant will now display associated "EXCEPT (column_list)" if
any.
---
doc/src/sgml/catalogs.sgml | 5 +-
doc/src/sgml/logical-replication.sgml | 97 ++++++++++++++-----
doc/src/sgml/ref/alter_publication.sgml | 10 +-
doc/src/sgml/ref/create_publication.sgml | 17 +++-
src/backend/catalog/pg_publication.c | 92 +++++++++++++++---
src/backend/commands/publicationcmds.c | 35 +++++--
src/backend/parser/gram.y | 65 +++++++++++++
src/backend/replication/pgoutput/pgoutput.c | 52 ++++++++--
src/bin/pg_dump/pg_dump.c | 16 ++-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/psql/describe.c | 85 +++++++++++-----
src/include/catalog/pg_publication.h | 10 +-
src/include/catalog/pg_publication_rel.h | 5 +-
src/test/regress/expected/publication.out | 67 +++++++++++++
src/test/regress/sql/publication.sql | 46 +++++++++
.../t/036_rep_changes_except_table.pl | 94 +++++++++++++++++-
16 files changed, 607 insertions(+), 90 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 4e37c928b44..5bbd95aaebb 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -6573,7 +6573,10 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<structfield>prexcept</structfield> <type>bool</type>
</para>
<para>
- True if the relation must be excluded
+ True if the relation or column list must be excluded. If publication is
+ created <literal>FOR ALL TABLES</literal> and it is specified as true,
+ the relation should be excluded. Else if it is true the columns in
+ <literal>prattrs</literal> should be excluded from being published.
</para></entry>
</row>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 3d0d29cf8b1..2d01e679691 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -1340,10 +1340,10 @@ Publications:
<title>Column Lists</title>
<para>
- Each publication can optionally specify which columns of each table are
- replicated to subscribers. The table on the subscriber side must have at
- least all the columns that are published. If no column list is specified,
- then all columns on the publisher are replicated.
+ Each publication can optionally specify which columns of each table should be
+ replicated or excluded from replication. On the subscriber side, the table
+ must include at least all the columns that are published. If no column list
+ is provided, all columns from the publisher are replicated by default.
See <xref linkend="sql-createpublication"/> for details on the syntax.
</para>
@@ -1356,9 +1356,11 @@ Publications:
</para>
<para>
- If no column list is specified, any columns added to the table later are
- automatically replicated. This means that having a column list which names
- all columns is not the same as having no column list at all.
+ If no column list or a column list with EXCEPT is specified, any columns
+ added to the table later are automatically replicated. This means that having
+ a column list which names all columns is not the same as having no
+ column list at all. If an column list is specified, any columns added to the
+ table later are automatically replicated.
</para>
<para>
@@ -1370,8 +1372,13 @@ Publications:
Generated columns can also be specified in a column list. This allows
generated columns to be published, regardless of the publication parameter
<link linkend="sql-createpublication-params-with-publish-generated-columns">
- <literal>publish_generated_columns</literal></link>. See
- <xref linkend="logical-replication-gencols"/> for details.
+ <literal>publish_generated_columns</literal></link>. Generated columns can
+ be included in column list specified with EXCEPT clause if publication
+ parameter
+ <link linkend="sql-createpublication-params-with-publish-generated-columns">
+ <literal>publish_generated_columns</literal></link> is not set to
+ <literal>none</literal>. Specified generated columns will not be published.
+ See <xref linkend="logical-replication-gencols"/> for details.
</para>
<para>
@@ -1391,11 +1398,13 @@ Publications:
<para>
If a publication publishes <command>UPDATE</command> or
- <command>DELETE</command> operations, any column list must include the
- table's replica identity columns (see
+ <command>DELETE</command> operations, any column list must include table's
+ replica identity columns and any column list specified with EXCEPT clause
+ must not include the table's replica identity columns (see
<xref linkend="sql-altertable-replica-identity"/>).
If a publication publishes only <command>INSERT</command> operations, then
- the column list may omit replica identity columns.
+ the column list may omit replica identity columns and the column list
+ specified with EXCEPT clause may include replica identity columns.
</para>
<para>
@@ -1440,18 +1449,20 @@ Publications:
<title>Examples</title>
<para>
- Create a table <literal>t1</literal> to be used in the following example.
+ Create tables <literal>t1</literal>, <literal>t2</literal> to be used in the
+ following example.
<programlisting>
/* pub # */ CREATE TABLE t1(id int, a text, b text, c text, d text, e text, PRIMARY KEY(id));
+/* pub # */ CREATE TABLE t2(id int, a text, b text, c text, d text, e text, PRIMARY KEY(id));
</programlisting></para>
<para>
Create a publication <literal>p1</literal>. A column list is defined for
- table <literal>t1</literal> to reduce the number of columns that will be
- replicated. Notice that the order of column names in the column list does
- not matter.
+ table <literal>t1</literal> and a column list is defined for table
+ <literal>t2</literal> with EXCEPT clause to reduce the number of columns that will be
+ replicated. Notice that the order of column names in the column lists does not matter.
<programlisting>
-/* pub # */ CREATE PUBLICATION p1 FOR TABLE t1 (id, b, a, d);
+/* pub # */ CREATE PUBLICATION p1 FOR TABLE t1 (id, b, a, d), t2 EXCEPT (d, a);
</programlisting></para>
<para>
@@ -1459,12 +1470,13 @@ Publications:
for each publication.
<programlisting>
/* pub # */ \dRp+
- Publication p1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
-----------+------------+---------+---------+---------+-----------+----------
- postgres | f | t | t | t | t | f
+ Publication p1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Generated columns | Via root
+--------+------------+---------+---------+---------+-----------+-------------------+----------
+ ubuntu | f | t | t | t | t | none | f
Tables:
"public.t1" (id, a, b, d)
+ "public.t2" EXCEPT (a, d)
</programlisting></para>
<para>
@@ -1485,23 +1497,41 @@ Indexes:
"t1_pkey" PRIMARY KEY, btree (id)
Publications:
"p1" (id, a, b, d)
+
+/* pub # */ \d t2
+ Table "public.t2"
+ Column | Type | Collation | Nullable | Default
+--------+---------+-----------+----------+---------
+ id | integer | | not null |
+ a | text | | |
+ b | text | | |
+ c | text | | |
+ d | text | | |
+ e | text | | |
+Indexes:
+ "t2_pkey" PRIMARY KEY, btree (id)
+Publications:
+ "p1" EXCEPT (a, d)
</programlisting></para>
<para>
- On the subscriber node, create a table <literal>t1</literal> which now
- only needs a subset of the columns that were on the publisher table
- <literal>t1</literal>, and also create the subscription
+ On the subscriber node, create tables <literal>t1</literal> and
+ <literal>t2</literal> which now only needs a subset of the columns that
+ were on the publisher tables <literal>t1</literal> and
+ <literal>t2</literal>, and also create the subscription
<literal>s1</literal> that subscribes to the publication
<literal>p1</literal>.
<programlisting>
/* sub # */ CREATE TABLE t1(id int, b text, a text, d text, PRIMARY KEY(id));
+/* sub # */ CREATE TABLE t2(id int, b text, c text, e text, PRIMARY KEY(id));
/* sub # */ CREATE SUBSCRIPTION s1
/* sub - */ CONNECTION 'host=localhost dbname=test_pub application_name=s1'
/* sub - */ PUBLICATION p1;
</programlisting></para>
<para>
- On the publisher node, insert some rows to table <literal>t1</literal>.
+ On the publisher node, insert some rows to tables <literal>t1</literal>
+ and <literal>t2</literal>
<programlisting>
/* pub # */ INSERT INTO t1 VALUES(1, 'a-1', 'b-1', 'c-1', 'd-1', 'e-1');
/* pub # */ INSERT INTO t1 VALUES(2, 'a-2', 'b-2', 'c-2', 'd-2', 'e-2');
@@ -1513,6 +1543,16 @@ Publications:
2 | a-2 | b-2 | c-2 | d-2 | e-2
3 | a-3 | b-3 | c-3 | d-3 | e-3
(3 rows)
+/* pub # */ INSERT INTO t2 VALUES(1, 'a-1', 'b-1', 'c-1', 'd-1', 'e-1');
+/* pub # */ INSERT INTO t2 VALUES(2, 'a-2', 'b-2', 'c-2', 'd-2', 'e-2');
+/* pub # */ INSERT INTO t2 VALUES(3, 'a-3', 'b-3', 'c-3', 'd-3', 'e-3');
+/* pub # */ SELECT * FROM t2 ORDER BY id;
+ id | a | b | c | d | e
+----+-----+-----+-----+-----+-----
+ 1 | a-1 | b-1 | c-1 | d-1 | e-1
+ 2 | a-2 | b-2 | c-2 | d-2 | e-2
+ 3 | a-3 | b-3 | c-3 | d-3 | e-3
+(3 rows)
</programlisting></para>
<para>
@@ -1526,6 +1566,13 @@ Publications:
2 | b-2 | a-2 | d-2
3 | b-3 | a-3 | d-3
(3 rows)
+/* sub # */ SELECT * FROM t2 ORDER BY id;
+ id | b | c | e
+----+-----+-----+-----
+ 1 | b-1 | c-1 | e-1
+ 2 | b-2 | c-2 | e-2
+ 3 | b-3 | c-3 | e-3
+(3 rows)
</programlisting></para>
</sect2>
diff --git a/doc/src/sgml/ref/alter_publication.sgml b/doc/src/sgml/ref/alter_publication.sgml
index 37e2c84bc10..10b249710bc 100644
--- a/doc/src/sgml/ref/alter_publication.sgml
+++ b/doc/src/sgml/ref/alter_publication.sgml
@@ -32,7 +32,7 @@ ALTER PUBLICATION <replaceable class="parameter">name</replaceable> RESET
<phrase>where <replaceable class="parameter">publication_object</replaceable> is one of:</phrase>
- TABLE [ ONLY ] <replaceable class="parameter">table_name</replaceable> [ * ] [ ( <replaceable class="parameter">column_name</replaceable> [, ... ] ) ] [ WHERE ( <replaceable class="parameter">expression</replaceable> ) ] [, ... ]
+ TABLE [ ONLY ] <replaceable class="parameter">table_name</replaceable> [ * ] [ [ EXCEPT ] ( column_name [, ... ] ) ] [ WHERE ( <replaceable class="parameter">expression</replaceable> ) ] [, ... ]
TABLES IN SCHEMA { <replaceable class="parameter">schema_name</replaceable> | CURRENT_SCHEMA } [, ... ]
<phrase>where <replaceable class="parameter">exception_object</replaceable> is:</phrase>
@@ -260,6 +260,14 @@ ALTER PUBLICATION production_publication ADD ALL TABLES EXCEPT users, department
<structname>production_publication</structname>:
<programlisting>
ALTER PUBLICATION production_publication ADD TABLE users, departments, TABLES IN SCHEMA production;
+</programlisting></para>
+
+ <para>
+ Alter publication <structname>mypublication</structname> to add table
+ <structname>users</structname> except column
+ <structname>security_pin</structname>:
+<programlisting>
+ALTER PUBLICATION production_publication ADD TABLE users EXCEPT (security_pin);
</programlisting></para>
<para>
diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml
index 7fd8872db5f..adee7d6d3d1 100644
--- a/doc/src/sgml/ref/create_publication.sgml
+++ b/doc/src/sgml/ref/create_publication.sgml
@@ -28,7 +28,7 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
<phrase>where <replaceable class="parameter">publication_object</replaceable> is one of:</phrase>
- TABLE [ ONLY ] <replaceable class="parameter">table_name</replaceable> [ * ] [ ( <replaceable class="parameter">column_name</replaceable> [, ... ] ) ] [ WHERE ( <replaceable class="parameter">expression</replaceable> ) ] [, ... ]
+ TABLE [ ONLY ] <replaceable class="parameter">table_name</replaceable> [ * ] [ [ EXCEPT ] ( column_name [, ... ] ) ] [ WHERE ( <replaceable class="parameter">expression</replaceable> ) ] [, ... ]
TABLES IN SCHEMA { <replaceable class="parameter">schema_name</replaceable> | CURRENT_SCHEMA } [, ... ]
<phrase>where <replaceable class="parameter">exception_object</replaceable> is:</phrase>
@@ -103,6 +103,12 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable>
lists.
</para>
+ <para>
+ When a column list is specified with EXCEPT, the named columns are not
+ replicated. Specifying a column list has no effect on
+ <literal>TRUNCATE</literal> commands.
+ </para>
+
<para>
Only persistent base tables and partitioned tables can be part of a
publication. Temporary tables, unlogged tables, foreign tables,
@@ -474,6 +480,15 @@ CREATE PUBLICATION mypublication FOR ALL TABLES EXCEPT users, departments;
<programlisting>
CREATE PUBLICATION users_filtered FOR TABLE users (user_id, firstname);
</programlisting></para>
+
+ <para>
+ Create a publication that publishes all changes for table
+ <structname>users</structname> except changes for column
+ <structname>security_pin</structname>:
+<programlisting>
+CREATE PUBLICATION users_safe FOR TABLE users EXCEPT (security_pin);
+</programlisting>
+ </para>
</refsect1>
<refsect1>
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index ec580e3b050..e4591cb0b35 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -263,13 +263,17 @@ is_schema_publication(Oid pubid)
* If a column list is found, the corresponding bitmap is returned through the
* cols parameter, if provided. The bitmap is constructed within the given
* memory context (mcxt).
+ *
+ * If a column list is found specified with EXCEPT clause, except_columns is set
+ * to true.
*/
bool
check_and_fetch_column_list(Publication *pub, Oid relid, MemoryContext mcxt,
- Bitmapset **cols)
+ Bitmapset **cols, bool *except_columns)
{
HeapTuple cftuple;
bool found = false;
+ bool except = false;
if (pub->alltables)
return false;
@@ -296,6 +300,15 @@ check_and_fetch_column_list(Publication *pub, Oid relid, MemoryContext mcxt,
found = true;
}
+ /* except found? */
+ cfdatum = SysCacheGetAttr(PUBLICATIONRELMAP, cftuple,
+ Anum_pg_publication_rel_prexcept, &isnull);
+
+ if (!isnull)
+ except = DatumGetBool(cfdatum);
+
+ *except_columns = except && !pub->alltables;
+
ReleaseSysCache(cftuple);
}
@@ -479,7 +492,9 @@ publication_add_relation(Oid pubid, PublicationRelInfo *pri,
check_publication_add_relation(targetrel);
/* Validate and translate column names into a Bitmapset of attnums. */
- attnums = pub_collist_validate(pri->relation, pri->columns);
+ attnums = pub_collist_validate(pri->relation, pri->columns,
+ pri->except && !pub->alltables,
+ pub->pubgencols_type);
/* Form a tuple. */
memset(values, 0, sizeof(values));
@@ -568,7 +583,8 @@ publication_add_relation(Oid pubid, PublicationRelInfo *pri,
* corresponding attnums.
*/
Bitmapset *
-pub_collist_validate(Relation targetrel, List *columns)
+pub_collist_validate(Relation targetrel, List *columns, bool except_columns,
+ PublishGencolsType pubgencols_type)
{
Bitmapset *set = NULL;
ListCell *lc;
@@ -591,6 +607,19 @@ pub_collist_validate(Relation targetrel, List *columns)
errmsg("cannot use system column \"%s\" in publication column list",
colname));
+ /*
+ * Check if column list specified with EXCEPT have any stored
+ * generated column and 'publish_generated_columns' is not set to
+ * 'stored'.
+ */
+ if (except_columns &&
+ TupleDescAttr(tupdesc, attnum - 1)->attgenerated == ATTRIBUTE_GENERATED_STORED &&
+ pubgencols_type != PUBLISH_GENCOLS_STORED)
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
+ errmsg("cannot use stored generated column \"%s\" in publication column list specified with EXCEPT when \"%s\" set to \"%s\"",
+ colname, "publish_generated_columns", "stored"));
+
if (TupleDescAttr(tupdesc, attnum - 1)->attgenerated == ATTRIBUTE_GENERATED_VIRTUAL)
ereport(ERROR,
errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
@@ -646,10 +675,12 @@ pub_collist_to_bitmapset(Bitmapset *columns, Datum pubcols, MemoryContext mcxt)
* Returns a bitmap representing the columns of the specified table.
*
* Generated columns are included if include_gencols_type is
- * PUBLISH_GENCOLS_STORED.
+ * PUBLISH_GENCOLS_STORED. Columns that are in the excludecols are excluded from
+ * the column list.
*/
Bitmapset *
-pub_form_cols_map(Relation relation, PublishGencolsType include_gencols_type)
+pub_form_cols_map(Relation relation, PublishGencolsType include_gencols_type,
+ Bitmapset *excludecols)
{
Bitmapset *result = NULL;
TupleDesc desc = RelationGetDescr(relation);
@@ -672,6 +703,9 @@ pub_form_cols_map(Relation relation, PublishGencolsType include_gencols_type)
continue;
}
+ if (excludecols && bms_is_member(att->attnum, excludecols))
+ continue;
+
result = bms_add_member(result, att->attnum);
}
@@ -776,9 +810,14 @@ GetRelationPublications(Oid relid, bool except_flag)
{
HeapTuple tup = &pubrellist->members[i]->tuple;
Oid pubid = ((Form_pg_publication_rel) GETSTRUCT(tup))->prpubid;
+ HeapTuple pubtup = SearchSysCache1(PUBLICATIONOID, ObjectIdGetDatum(pubid));
+ bool is_table_excluded = ((Form_pg_publication) GETSTRUCT(pubtup))->puballtables &&
+ ((Form_pg_publication_rel) GETSTRUCT(tup))->prexcept;
- if (except_flag == ((Form_pg_publication_rel) GETSTRUCT(tup))->prexcept)
+ if (except_flag == is_table_excluded)
result = lappend_oid(result, pubid);
+
+ ReleaseSysCache(pubtup);
}
ReleaseSysCacheList(pubrellist);
@@ -1263,6 +1302,11 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
Oid schemaid = get_rel_namespace(relid);
Datum values[NUM_PUBLICATION_TABLES_ELEM] = {0};
bool nulls[NUM_PUBLICATION_TABLES_ELEM] = {0};
+ Datum columnsDatum;
+ Datum exceptDatum;
+ Bitmapset *columns = NULL;
+ bool except = false;
+ bool isnull;
/*
* Form tuple with appropriate data.
@@ -1287,10 +1331,22 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
if (HeapTupleIsValid(pubtuple))
{
- /* Lookup the column list attribute. */
- values[2] = SysCacheGetAttr(PUBLICATIONRELMAP, pubtuple,
- Anum_pg_publication_rel_prattrs,
- &(nulls[2]));
+ exceptDatum = SysCacheGetAttr(PUBLICATIONRELMAP, pubtuple,
+ Anum_pg_publication_rel_prexcept,
+ &isnull);
+
+ if (!isnull)
+ except = DatumGetBool(exceptDatum);
+
+ columnsDatum = SysCacheGetAttr(PUBLICATIONRELMAP, pubtuple,
+ Anum_pg_publication_rel_prattrs,
+ &(nulls[2]));
+
+ /* if column list is specified with EXCEPT */
+ if (!pub->alltables && except)
+ columns = pub_collist_to_bitmapset(NULL, columnsDatum, NULL);
+ else
+ values[2] = columnsDatum;
/* Null indicates no filter. */
values[3] = SysCacheGetAttr(PUBLICATIONRELMAP, pubtuple,
@@ -1303,8 +1359,12 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
nulls[3] = true;
}
- /* Show all columns when the column list is not specified. */
- if (nulls[2])
+ /*
+ * Construct column list to show all columns when no column list is
+ * specified or to show remaining columns when a column list is
+ * provided with EXCEPT.
+ */
+ if ((!pub->alltables && except) || nulls[2])
{
Relation rel = table_open(relid, AccessShareLock);
int nattnums = 0;
@@ -1335,6 +1395,14 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
continue;
}
+ /*
+ * Skip columns that are part of column list specified with
+ * EXCEPT.
+ */
+ if ((!pub->alltables && except) &&
+ bms_is_member(att->attnum, columns))
+ continue;
+
attnums[nattnums++] = att->attnum;
}
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 5194b2fb6e2..8513e579df8 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -204,7 +204,6 @@ ObjectsInPublicationToOids(List *pubobjspec_list, ParseState *pstate,
switch (pubobj->pubobjtype)
{
case PUBLICATIONOBJ_TABLE:
- pubobj->pubtable->except = false;
*rels = lappend(*rels, pubobj->pubtable);
break;
case PUBLICATIONOBJ_EXCEPT_TABLE:
@@ -358,7 +357,8 @@ pub_rf_contains_invalid_column(Oid pubid, Relation relation, List *ancestors,
* This function evaluates two conditions:
*
* 1. Ensures that all columns referenced in the REPLICA IDENTITY are covered
- * by the column list. If any column is missing, *invalid_column_list is set
+ * by the column list and are not part of column list specified with EXCEPT.
+ * If any column is missing, *invalid_column_list is set
* to true.
* 2. Ensures that all the generated columns referenced in the REPLICA IDENTITY
* are published, either by being explicitly named in the column list or, if
@@ -381,6 +381,7 @@ pub_contains_invalid_column(Oid pubid, Relation relation, List *ancestors,
TupleDesc desc = RelationGetDescr(relation);
Publication *pub;
int x;
+ bool except_columns = false;
*invalid_column_list = false;
*invalid_gen_col = false;
@@ -404,7 +405,8 @@ pub_contains_invalid_column(Oid pubid, Relation relation, List *ancestors,
/* Fetch the column list */
pub = GetPublication(pubid);
- check_and_fetch_column_list(pub, publish_as_relid, NULL, &columns);
+ check_and_fetch_column_list(pub, publish_as_relid, NULL, &columns,
+ &except_columns);
if (relation->rd_rel->relreplident == REPLICA_IDENTITY_FULL)
{
@@ -494,8 +496,14 @@ pub_contains_invalid_column(Oid pubid, Relation relation, List *ancestors,
attnum = get_attnum(publish_as_relid, colname);
}
- /* replica identity column, not covered by the column list */
- *invalid_column_list |= !bms_is_member(attnum, columns);
+ /*
+ * Replica identity column, not covered by the column list or is part
+ * of column list specified with EXCEPT.
+ */
+ if (except_columns)
+ *invalid_column_list |= bms_is_member(attnum, columns);
+ else
+ *invalid_column_list |= !bms_is_member(attnum, columns);
if (*invalid_column_list && *invalid_gen_col)
break;
@@ -1443,6 +1451,7 @@ AlterPublicationTables(AlterPublicationStmt *stmt, HeapTuple tup,
HeapTuple rftuple;
Node *oldrelwhereclause = NULL;
Bitmapset *oldcolumns = NULL;
+ bool oldexcept = false;
/* look up the cache for the old relmap */
rftuple = SearchSysCache2(PUBLICATIONRELMAP,
@@ -1458,6 +1467,7 @@ AlterPublicationTables(AlterPublicationStmt *stmt, HeapTuple tup,
bool isnull = true;
Datum whereClauseDatum;
Datum columnListDatum;
+ Datum exceptDatum;
/* Load the WHERE clause for this table. */
whereClauseDatum = SysCacheGetAttr(PUBLICATIONRELMAP, rftuple,
@@ -1474,6 +1484,13 @@ AlterPublicationTables(AlterPublicationStmt *stmt, HeapTuple tup,
if (!isnull)
oldcolumns = pub_collist_to_bitmapset(NULL, columnListDatum, NULL);
+ exceptDatum = SysCacheGetAttr(PUBLICATIONRELMAP, rftuple,
+ Anum_pg_publication_rel_prexcept,
+ &isnull);
+
+ if (!isnull)
+ oldexcept = DatumGetBool(exceptDatum);
+
ReleaseSysCache(rftuple);
}
@@ -1493,7 +1510,9 @@ AlterPublicationTables(AlterPublicationStmt *stmt, HeapTuple tup,
* is cheap enough that that seems harmless.
*/
newcolumns = pub_collist_validate(newpubrel->relation,
- newpubrel->columns);
+ newpubrel->columns,
+ newpubrel->except && !pubform->puballtables,
+ pubform->pubgencols);
/*
* Check if any of the new set of relations matches with the
@@ -1505,7 +1524,8 @@ AlterPublicationTables(AlterPublicationStmt *stmt, HeapTuple tup,
if (newrelid == oldrelid)
{
if (equal(oldrelwhereclause, newpubrel->whereClause) &&
- bms_equal(oldcolumns, newcolumns))
+ bms_equal(oldcolumns, newcolumns) &&
+ oldexcept == newpubrel->except)
{
found = true;
break;
@@ -1522,6 +1542,7 @@ AlterPublicationTables(AlterPublicationStmt *stmt, HeapTuple tup,
oldrel = palloc(sizeof(PublicationRelInfo));
oldrel->whereClause = NULL;
oldrel->columns = NIL;
+ oldrel->except = false;
oldrel->relation = table_open(oldrelid,
ShareUpdateExclusiveLock);
delrels = lappend(delrels, oldrel);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index d7fe95a840f..64827dc1b09 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -446,6 +446,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
TriggerTransitions TriggerReferencing
vacuum_relation_list opt_vacuum_relation_list
drop_option_list pub_obj_list except_pub_obj_list
+ opt_exclude_column_list
%type <retclause> returning_clause
%type <node> returning_option
@@ -4413,6 +4414,10 @@ opt_column_list:
| /*EMPTY*/ { $$ = NIL; }
;
+opt_exclude_column_list:
+ '(' columnList ')' { $$ = $2; }
+ ;
+
columnList:
columnElem { $$ = list_make1($1); }
| columnList ',' columnElem { $$ = lappend($1, $3); }
@@ -10679,6 +10684,17 @@ PublicationObjSpec:
$$->pubtable->whereClause = $4;
$$->location = @1;
}
+ | TABLE relation_expr EXCEPT opt_exclude_column_list OptWhereClause
+ {
+ $$ = makeNode(PublicationObjSpec);
+ $$->pubobjtype = PUBLICATIONOBJ_TABLE;
+ $$->pubtable = makeNode(PublicationTable);
+ $$->pubtable->relation = $2;
+ $$->pubtable->columns = $4;
+ $$->pubtable->whereClause = $5;
+ $$->pubtable->except = true;
+ $$->location = @1;
+ }
| TABLES IN_P SCHEMA ColId
{
$$ = makeNode(PublicationObjSpec);
@@ -10719,6 +10735,34 @@ PublicationObjSpec:
}
$$->location = @1;
}
+ | ColId EXCEPT opt_exclude_column_list OptWhereClause
+ {
+ $$ = makeNode(PublicationObjSpec);
+ $$->pubobjtype = PUBLICATIONOBJ_CONTINUATION;
+ /*
+ * If either a row filter or exclude column list is
+ * specified, create a PublicationTable object.
+ */
+ if ($3 || $4)
+ {
+ /*
+ * The OptWhereClause must be stored here but it is
+ * valid only for tables. For non-table objects, an
+ * error will be thrown later via
+ * preprocess_pubobj_list().
+ */
+ $$->pubtable = makeNode(PublicationTable);
+ $$->pubtable->relation = makeRangeVar(NULL, $1, @1);
+ $$->pubtable->columns = $3;
+ $$->pubtable->whereClause = $4;
+ $$->pubtable->except = true;
+ }
+ else
+ {
+ $$->name = $1;
+ }
+ $$->location = @1;
+ }
| ColId indirection opt_column_list OptWhereClause
{
$$ = makeNode(PublicationObjSpec);
@@ -10729,6 +10773,17 @@ PublicationObjSpec:
$$->pubtable->whereClause = $4;
$$->location = @1;
}
+ | ColId indirection EXCEPT opt_exclude_column_list OptWhereClause
+ {
+ $$ = makeNode(PublicationObjSpec);
+ $$->pubobjtype = PUBLICATIONOBJ_CONTINUATION;
+ $$->pubtable = makeNode(PublicationTable);
+ $$->pubtable->relation = makeRangeVarFromQualifiedName($1, $2, @1, yyscanner);
+ $$->pubtable->columns = $4;
+ $$->pubtable->whereClause = $5;
+ $$->pubtable->except = true;
+ $$->location = @1;
+ }
/* grammar like tablename * , ONLY tablename, ONLY ( tablename ) */
| extended_relation_expr opt_column_list OptWhereClause
{
@@ -10739,6 +10794,16 @@ PublicationObjSpec:
$$->pubtable->columns = $2;
$$->pubtable->whereClause = $3;
}
+ | extended_relation_expr EXCEPT opt_exclude_column_list OptWhereClause
+ {
+ $$ = makeNode(PublicationObjSpec);
+ $$->pubobjtype = PUBLICATIONOBJ_CONTINUATION;
+ $$->pubtable = makeNode(PublicationTable);
+ $$->pubtable->relation = $1;
+ $$->pubtable->columns = $3;
+ $$->pubtable->whereClause = $4;
+ $$->pubtable->except = true;
+ }
| CURRENT_SCHEMA
{
$$ = makeNode(PublicationObjSpec);
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 5512b4cba7f..be9f548007d 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -185,6 +185,9 @@ typedef struct RelationSyncEntry
* row filter expressions, column list, etc.
*/
MemoryContext entry_cxt;
+
+ /* Indicate if no column is included in the publication */
+ bool no_cols_published;
} RelationSyncEntry;
/*
@@ -1066,12 +1069,19 @@ check_and_init_gencol(PGOutputData *data, List *publications,
*/
foreach_ptr(Publication, pub, publications)
{
+ bool found = false;
+ bool except_columns = false;
+
+ found = check_and_fetch_column_list(pub, entry->publish_as_relid, NULL,
+ NULL, &except_columns);
+
/*
* The column list takes precedence over the
* 'publish_generated_columns' parameter. Those will be checked later,
- * see pgoutput_column_list_init.
+ * see pgoutput_column_list_init. But when a column list is specified
+ * with EXCEPT, it should be checked.
*/
- if (check_and_fetch_column_list(pub, entry->publish_as_relid, NULL, NULL))
+ if (found && !except_columns)
continue;
if (first)
@@ -1120,11 +1130,30 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
{
Publication *pub = lfirst(lc);
Bitmapset *cols = NULL;
+ bool except_columns = false;
+ bool no_col_published = false;
/* Retrieve the bitmap of columns for a column list publication. */
found_pub_collist |= check_and_fetch_column_list(pub,
entry->publish_as_relid,
- entry->entry_cxt, &cols);
+ entry->entry_cxt, &cols,
+ &except_columns);
+
+ /*
+ * If column list is specified with EXCEPT retrieve bitmap of columns
+ * which are not part of this column list.
+ */
+ if (except_columns)
+ {
+ MemoryContext oldcxt = MemoryContextSwitchTo(entry->entry_cxt);
+
+ cols = pub_form_cols_map(relation,
+ entry->include_gencols_type, cols);
+ MemoryContextSwitchTo(oldcxt);
+
+ if (!cols)
+ no_col_published = true;
+ }
/*
* For non-column list publications — e.g. TABLE (without a column
@@ -1132,7 +1161,7 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
* of the table (including generated columns when
* 'publish_generated_columns' parameter is true).
*/
- if (!cols)
+ if (!no_col_published && !cols)
{
/*
* Cache the table columns for the first publication with no
@@ -1144,7 +1173,7 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
MemoryContext oldcxt = MemoryContextSwitchTo(entry->entry_cxt);
relcols = pub_form_cols_map(relation,
- entry->include_gencols_type);
+ entry->include_gencols_type, NULL);
MemoryContextSwitchTo(oldcxt);
}
@@ -1154,9 +1183,11 @@ pgoutput_column_list_init(PGOutputData *data, List *publications,
if (first)
{
entry->columns = cols;
+ entry->no_cols_published = no_col_published;
first = false;
}
- else if (!bms_equal(entry->columns, cols))
+ else if ((entry->no_cols_published != no_col_published) ||
+ !bms_equal(entry->columns, cols))
ereport(ERROR,
errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot use different column lists for table \"%s.%s\" in different publications",
@@ -1480,6 +1511,13 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
relentry = get_rel_sync_entry(data, relation);
+ /*
+ * If all columns of a table is present in column list specified with
+ * EXCEPT, skip publishing the changes.
+ */
+ if (relentry->no_cols_published)
+ return;
+
/* First check the table filter */
switch (action)
{
@@ -2057,6 +2095,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->publish_as_relid = InvalidOid;
entry->columns = NULL;
entry->attrmap = NULL;
+ entry->no_cols_published = false;
}
/* Validate the entry */
@@ -2106,6 +2145,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate = false;
entry->pubactions.pubdelete = false;
entry->pubactions.pubtruncate = false;
+ entry->no_cols_published = false;
/*
* Tuple slots cleanups. (Will be rebuilt later if needed).
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 6cc55afd498..25f9161f35e 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -4780,10 +4780,10 @@ getPublicationTables(Archive *fout, TableInfo tblinfo[], int numTables)
continue;
/* OK, make a DumpableObject for this relationship */
- if (strcmp(prexcept, "f") == 0)
- pubrinfo[j].dobj.objType = DO_PUBLICATION_REL;
- else
+ if (strcmp(prexcept, "t") == 0 && PQgetisnull(res, i, i_prattrs))
pubrinfo[j].dobj.objType = DO_PUBLICATION_EXCEPT_REL;
+ else
+ pubrinfo[j].dobj.objType = DO_PUBLICATION_REL;
pubrinfo[j].dobj.catId.tableoid =
atooid(PQgetvalue(res, i, i_tableoid));
@@ -4797,6 +4797,7 @@ getPublicationTables(Archive *fout, TableInfo tblinfo[], int numTables)
pubrinfo[j].pubrelqual = NULL;
else
pubrinfo[j].pubrelqual = pg_strdup(PQgetvalue(res, i, i_prrelqual));
+ pubrinfo[j].pubexcept = (strcmp(prexcept, "t") == 0);
if (!PQgetisnull(res, i, i_prattrs))
{
@@ -4825,7 +4826,7 @@ getPublicationTables(Archive *fout, TableInfo tblinfo[], int numTables)
/* Decide whether we want to dump it */
selectDumpablePublicationObject(&(pubrinfo[j].dobj), fout);
- if (strcmp(prexcept, "t") == 0)
+ if (strcmp(prexcept, "t") == 0 && PQgetisnull(res, i, i_prattrs))
simple_ptr_list_append(&exceptinfo, &pubrinfo[j]);
j++;
@@ -4905,7 +4906,12 @@ dumpPublicationTable(Archive *fout, const PublicationRelInfo *pubrinfo)
fmtQualifiedDumpable(tbinfo));
if (pubrinfo->pubrattrs)
- appendPQExpBuffer(query, " (%s)", pubrinfo->pubrattrs);
+ {
+ if (pubrinfo->pubexcept)
+ appendPQExpBuffer(query, " EXCEPT (%s)", pubrinfo->pubrattrs);
+ else
+ appendPQExpBuffer(query, " (%s)", pubrinfo->pubrattrs);
+ }
if (pubrinfo->pubrelqual)
{
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 096f29346d8..daf6fb51a3f 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -681,6 +681,7 @@ typedef struct _PublicationRelInfo
TableInfo *pubtable;
char *pubrelqual;
char *pubrattrs;
+ bool pubexcept;
} PublicationRelInfo;
/*
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 10b5f7f29cb..a55c4c6505d 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -3019,12 +3019,14 @@ describeOneTableDetails(const char *schemaname,
/* print any publications */
if (pset.sversion >= 100000)
{
- if (pset.sversion >= 150000)
+ /* FIXME: 180000 should be changed to 190000 later for PG19. */
+ if (pset.sversion >= 180000)
{
printfPQExpBuffer(&buf,
"SELECT pubname\n"
" , NULL\n"
" , NULL\n"
+ " , NULL\n"
"FROM pg_catalog.pg_publication p\n"
" JOIN pg_catalog.pg_publication_namespace pn ON p.oid = pn.pnpubid\n"
" JOIN pg_catalog.pg_class pc ON pc.relnamespace = pn.pnnspid\n"
@@ -3038,37 +3040,61 @@ describeOneTableDetails(const char *schemaname,
" pg_catalog.pg_attribute\n"
" WHERE attrelid = pr.prrelid AND attnum = prattrs[s])\n"
" ELSE NULL END) "
+ " , prexcept "
"FROM pg_catalog.pg_publication p\n"
" JOIN pg_catalog.pg_publication_rel pr ON p.oid = pr.prpubid\n"
" JOIN pg_catalog.pg_class c ON c.oid = pr.prrelid\n"
- "WHERE pr.prrelid = '%s'\n",
- oid, oid, oid);
-
- /* FIXME: 180000 should be changed to 190000 later for PG19. */
- if (pset.sversion >= 180000)
- appendPQExpBufferStr(&buf, " AND NOT pr.prexcept\n");
-
- appendPQExpBuffer(&buf,
+ "WHERE pr.prrelid = '%s' "
+ "AND c.relnamespace NOT IN (\n "
+ " SELECT pnnspid FROM\n"
+ " pg_catalog.pg_publication_namespace)\n"
"UNION\n"
"SELECT pubname\n"
" , NULL\n"
" , NULL\n"
+ " , NULL\n"
+ "FROM pg_catalog.pg_publication p\n"
+ "WHERE p.puballtables AND pg_catalog.pg_relation_is_publishable('%s')\n"
+ " AND NOT EXISTS (\n"
+ " SELECT 1\n"
+ " FROM pg_catalog.pg_publication_rel pr\n"
+ " JOIN pg_catalog.pg_class pc\n"
+ " ON pr.prrelid = pc.oid\n"
+ " WHERE pr.prrelid = '%s' AND pr.prpubid = p.oid)\n"
+ "ORDER BY 1;",
+ oid, oid, oid, oid, oid);
+ }
+ else if (pset.sversion >= 150000)
+ {
+ printfPQExpBuffer(&buf,
+ "SELECT pubname\n"
+ " , NULL\n"
+ " , NULL\n"
+ "FROM pg_catalog.pg_publication p\n"
+ " JOIN pg_catalog.pg_publication_namespace pn ON p.oid = pn.pnpubid\n"
+ " JOIN pg_catalog.pg_class pc ON pc.relnamespace = pn.pnnspid\n"
+ "WHERE pc.oid ='%s' and pg_catalog.pg_relation_is_publishable('%s')\n"
+ "UNION\n"
+ "SELECT pubname\n"
+ " , pg_get_expr(pr.prqual, c.oid)\n"
+ " , (CASE WHEN pr.prattrs IS NOT NULL THEN\n"
+ " (SELECT string_agg(attname, ', ')\n"
+ " FROM pg_catalog.generate_series(0, pg_catalog.array_upper(pr.prattrs::pg_catalog.int2[], 1)) s,\n"
+ " pg_catalog.pg_attribute\n"
+ " WHERE attrelid = pr.prrelid AND attnum = prattrs[s])\n"
+ " ELSE NULL END) "
"FROM pg_catalog.pg_publication p\n"
- "WHERE p.puballtables AND pg_catalog.pg_relation_is_publishable('%s')\n",
- oid);
-
- /* FIXME: 180000 should be changed to 190000 later for PG19. */
- if (pset.sversion >= 180000)
- appendPQExpBuffer(&buf,
- " AND NOT EXISTS (\n"
- " SELECT 1\n"
- " FROM pg_catalog.pg_publication_rel pr\n"
- " JOIN pg_catalog.pg_class pc\n"
- " ON pr.prrelid = pc.oid\n"
- " WHERE pr.prrelid = '%s' AND pr.prpubid = p.oid)\n",
- oid);
-
- appendPQExpBufferStr(&buf, "ORDER BY 1;");
+ " JOIN pg_catalog.pg_publication_rel pr ON p.oid = pr.prpubid\n"
+ " JOIN pg_catalog.pg_class c ON c.oid = pr.prrelid\n"
+ "WHERE pr.prrelid = '%s'\n"
+ "UNION\n"
+ "SELECT pubname\n"
+ " , NULL\n"
+ " , NULL\n"
+ "FROM pg_catalog.pg_publication p\n"
+ "WHERE p.puballtables AND pg_catalog.pg_relation_is_publishable('%s')\n"
+ "ORDER BY 1;",
+ oid, oid, oid, oid);
}
else
{
@@ -3106,8 +3132,15 @@ describeOneTableDetails(const char *schemaname,
/* column list (if any) */
if (!PQgetisnull(result, i, 2))
- appendPQExpBuffer(&buf, " (%s)",
- PQgetvalue(result, i, 2));
+ {
+ if (!PQgetisnull(result, i, 3) &&
+ strcmp(PQgetvalue(result, i, 3), "t") == 0)
+ appendPQExpBuffer(&buf, " EXCEPT (%s)",
+ PQgetvalue(result, i, 2));
+ else
+ appendPQExpBuffer(&buf, " (%s)",
+ PQgetvalue(result, i, 2));
+ }
/* row filter (if any) */
if (!PQgetisnull(result, i, 1))
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index 33b771990bd..9ac94159c13 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -180,16 +180,20 @@ extern Oid GetTopMostAncestorInPublication(Oid puboid, List *ancestors,
extern bool is_publishable_relation(Relation rel);
extern bool is_schema_publication(Oid pubid);
extern bool check_and_fetch_column_list(Publication *pub, Oid relid,
- MemoryContext mcxt, Bitmapset **cols);
+ MemoryContext mcxt, Bitmapset **cols,
+ bool *except_columns);
extern ObjectAddress publication_add_relation(Oid pubid, PublicationRelInfo *pri,
bool if_not_exists);
-extern Bitmapset *pub_collist_validate(Relation targetrel, List *columns);
+extern Bitmapset *pub_collist_validate(Relation targetrel, List *columns,
+ bool except_columns,
+ PublishGencolsType pubgencols_type);
extern ObjectAddress publication_add_schema(Oid pubid, Oid schemaid,
bool if_not_exists);
extern Bitmapset *pub_collist_to_bitmapset(Bitmapset *columns, Datum pubcols,
MemoryContext mcxt);
extern Bitmapset *pub_form_cols_map(Relation relation,
- PublishGencolsType include_gencols_type);
+ PublishGencolsType include_gencols_type,
+ Bitmapset *excludecols);
#endif /* PG_PUBLICATION_H */
diff --git a/src/include/catalog/pg_publication_rel.h b/src/include/catalog/pg_publication_rel.h
index e7d7f3ba85c..6a2168fc32c 100644
--- a/src/include/catalog/pg_publication_rel.h
+++ b/src/include/catalog/pg_publication_rel.h
@@ -31,11 +31,12 @@ CATALOG(pg_publication_rel,6106,PublicationRelRelationId)
Oid oid; /* oid */
Oid prpubid BKI_LOOKUP(pg_publication); /* Oid of the publication */
Oid prrelid BKI_LOOKUP(pg_class); /* Oid of the relation */
- bool prexcept BKI_DEFAULT(f); /* exclude the relation */
+ bool prexcept BKI_DEFAULT(f); /* exclude the relation or columns */
#ifdef CATALOG_VARLEN /* variable-length fields start here */
pg_node_tree prqual; /* qualifications */
- int2vector prattrs; /* columns to replicate */
+ int2vector prattrs; /* columns to replicate or exclude to
+ * replicate */
#endif
} FormData_pg_publication_rel;
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 5d025328704..833c19305a9 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -2125,6 +2125,73 @@ SET ROLE regress_publication_user;
DROP PUBLICATION testpub_reset;
DROP TABLE pub_sch1.tbl1;
DROP TABLE pub_sch1.tbl2;
+-- ======================================================
+-- Test EXCEPT columns for CREATE PUBLICATION
+SET client_min_messages = 'ERROR';
+CREATE TABLE pub_test_except1 (a int NOT NULL, b int, c int NOT NULL, d int);
+CREATE TABLE pub_sch1.pub_test_except2 (a int, b int, c int, d int);
+CREATE TABLE pub_test_except3 (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED, gen2 int GENERATED ALWAYS AS (a * 2) VIRTUAL);
+-- Verify that publication is created with EXCEPT
+CREATE PUBLICATION testpub_except FOR TABLE pub_test_except1, pub_sch1.pub_test_except2 EXCEPT (b, c);
+SELECT * FROM pg_publication_tables WHERE pubname = 'testpub_except';
+ pubname | schemaname | tablename | attnames | rowfilter
+----------------+------------+------------------+-----------+-----------
+ testpub_except | public | pub_test_except1 | {a,b,c,d} |
+ testpub_except | pub_sch1 | pub_test_except2 | {a,d} |
+(2 rows)
+
+-- Check for invalid cases
+CREATE PUBLICATION testpub_except2 FOR TABLES IN SCHEMA pub_sch1, TABLE pub_test_except1 EXCEPT (b, c);
+ERROR: cannot use column list for relation "public.pub_test_except1" in publication "testpub_except2"
+DETAIL: Column lists cannot be specified in publications containing FOR TABLES IN SCHEMA elements.
+CREATE PUBLICATION testpub_except2 FOR TABLE pub_test_except1 EXCEPT;
+ERROR: syntax error at or near ";"
+LINE 1: ...BLICATION testpub_except2 FOR TABLE pub_test_except1 EXCEPT;
+ ^
+CREATE PUBLICATION testpub_except2 FOR TABLE pub_test_except3 EXCEPT (gen1);
+ERROR: cannot use stored generated column "gen1" in publication column list specified with EXCEPT when "publish_generated_columns" set to "stored"
+CREATE PUBLICATION testpub_except2 FOR TABLE pub_test_except3 EXCEPT (gen2) with (publish_generated_columns);
+ERROR: cannot use virtual generated column "gen2" in publication column list
+-- Verify that publication can be altered with EXCEPT
+ALTER PUBLICATION testpub_except SET TABLE pub_test_except1 EXCEPT (a, b), pub_sch1.pub_test_except2;
+SELECT * FROM pg_publication_tables WHERE pubname = 'testpub_except';
+ pubname | schemaname | tablename | attnames | rowfilter
+----------------+------------+------------------+-----------+-----------
+ testpub_except | public | pub_test_except1 | {c,d} |
+ testpub_except | pub_sch1 | pub_test_except2 | {a,b,c,d} |
+(2 rows)
+
+-- Verify ALTER PUBLICATION ... DROP
+ALTER PUBLICATION testpub_except DROP TABLE pub_test_except1 EXCEPT (a, b);
+ERROR: column list must not be specified in ALTER PUBLICATION ... DROP
+ALTER PUBLICATION testpub_except DROP TABLE pub_test_except1;
+ALTER PUBLICATION testpub_except ADD TABLE pub_test_except1 EXCEPT (c, d);
+SELECT * FROM pg_publication_tables WHERE pubname = 'testpub_except';
+ pubname | schemaname | tablename | attnames | rowfilter
+----------------+------------+------------------+-----------+-----------
+ testpub_except | public | pub_test_except1 | {a,b} |
+ testpub_except | pub_sch1 | pub_test_except2 | {a,b,c,d} |
+(2 rows)
+
+-- Verify excluded columns cannot be part of REPLICA IDENTITY
+ALTER TABLE pub_test_except1 REPLICA IDENTITY FULL;
+UPDATE pub_test_except1 SET a = 3 WHERE a = 1;
+ERROR: cannot update table "pub_test_except1"
+DETAIL: Column list used by the publication does not cover the replica identity.
+CREATE UNIQUE INDEX pub_test_except1_a_idx ON pub_test_except1 (a, c);
+ALTER TABLE pub_test_except1 REPLICA IDENTITY USING INDEX pub_test_except1_a_idx;
+UPDATE pub_test_except1 SET a = 3 WHERE a = 1;
+ERROR: cannot update table "pub_test_except1"
+DETAIL: Column list used by the publication does not cover the replica identity.
+DROP INDEX pub_test_except1_a_idx;
+CREATE UNIQUE INDEX pub_test_except1_a_idx ON pub_test_except1 (a);
+ALTER TABLE pub_test_except1 REPLICA IDENTITY USING INDEX pub_test_except1_a_idx;
+UPDATE pub_test_except1 SET a = 3 WHERE a = 1;
+DROP INDEX pub_test_except1_a_idx;
+DROP PUBLICATION testpub_except;
+DROP TABLE pub_test_except1;
+DROP TABLE pub_sch1.pub_test_except2;
+DROP TABLE pub_test_except3;
DROP SCHEMA pub_sch1;
RESET client_min_messages;
RESET SESSION AUTHORIZATION;
diff --git a/src/test/regress/sql/publication.sql b/src/test/regress/sql/publication.sql
index af31a2214ca..3dbbddb1950 100644
--- a/src/test/regress/sql/publication.sql
+++ b/src/test/regress/sql/publication.sql
@@ -1318,6 +1318,52 @@ SET ROLE regress_publication_user;
DROP PUBLICATION testpub_reset;
DROP TABLE pub_sch1.tbl1;
DROP TABLE pub_sch1.tbl2;
+
+-- ======================================================
+-- Test EXCEPT columns for CREATE PUBLICATION
+
+SET client_min_messages = 'ERROR';
+CREATE TABLE pub_test_except1 (a int NOT NULL, b int, c int NOT NULL, d int);
+CREATE TABLE pub_sch1.pub_test_except2 (a int, b int, c int, d int);
+CREATE TABLE pub_test_except3 (a int, gen1 int GENERATED ALWAYS AS (a * 2) STORED, gen2 int GENERATED ALWAYS AS (a * 2) VIRTUAL);
+
+-- Verify that publication is created with EXCEPT
+CREATE PUBLICATION testpub_except FOR TABLE pub_test_except1, pub_sch1.pub_test_except2 EXCEPT (b, c);
+SELECT * FROM pg_publication_tables WHERE pubname = 'testpub_except';
+
+-- Check for invalid cases
+CREATE PUBLICATION testpub_except2 FOR TABLES IN SCHEMA pub_sch1, TABLE pub_test_except1 EXCEPT (b, c);
+CREATE PUBLICATION testpub_except2 FOR TABLE pub_test_except1 EXCEPT;
+CREATE PUBLICATION testpub_except2 FOR TABLE pub_test_except3 EXCEPT (gen1);
+CREATE PUBLICATION testpub_except2 FOR TABLE pub_test_except3 EXCEPT (gen2) with (publish_generated_columns);
+
+-- Verify that publication can be altered with EXCEPT
+ALTER PUBLICATION testpub_except SET TABLE pub_test_except1 EXCEPT (a, b), pub_sch1.pub_test_except2;
+SELECT * FROM pg_publication_tables WHERE pubname = 'testpub_except';
+
+-- Verify ALTER PUBLICATION ... DROP
+ALTER PUBLICATION testpub_except DROP TABLE pub_test_except1 EXCEPT (a, b);
+ALTER PUBLICATION testpub_except DROP TABLE pub_test_except1;
+
+ALTER PUBLICATION testpub_except ADD TABLE pub_test_except1 EXCEPT (c, d);
+SELECT * FROM pg_publication_tables WHERE pubname = 'testpub_except';
+
+-- Verify excluded columns cannot be part of REPLICA IDENTITY
+ALTER TABLE pub_test_except1 REPLICA IDENTITY FULL;
+UPDATE pub_test_except1 SET a = 3 WHERE a = 1;
+CREATE UNIQUE INDEX pub_test_except1_a_idx ON pub_test_except1 (a, c);
+ALTER TABLE pub_test_except1 REPLICA IDENTITY USING INDEX pub_test_except1_a_idx;
+UPDATE pub_test_except1 SET a = 3 WHERE a = 1;
+DROP INDEX pub_test_except1_a_idx;
+CREATE UNIQUE INDEX pub_test_except1_a_idx ON pub_test_except1 (a);
+ALTER TABLE pub_test_except1 REPLICA IDENTITY USING INDEX pub_test_except1_a_idx;
+UPDATE pub_test_except1 SET a = 3 WHERE a = 1;
+
+DROP INDEX pub_test_except1_a_idx;
+DROP PUBLICATION testpub_except;
+DROP TABLE pub_test_except1;
+DROP TABLE pub_sch1.pub_test_except2;
+DROP TABLE pub_test_except3;
DROP SCHEMA pub_sch1;
RESET client_min_messages;
diff --git a/src/test/subscription/t/036_rep_changes_except_table.pl b/src/test/subscription/t/036_rep_changes_except_table.pl
index 1d115283809..aefa67e6145 100644
--- a/src/test/subscription/t/036_rep_changes_except_table.pl
+++ b/src/test/subscription/t/036_rep_changes_except_table.pl
@@ -1,7 +1,7 @@
# Copyright (c) 2021-2022, PostgreSQL Global Development Group
-# Logical replication tests for except table publications
+# Logical replication tests for except table and except column publications
use strict;
use warnings;
use PostgreSQL::Test::Cluster;
@@ -77,6 +77,98 @@ $result = $node_subscriber->safe_psql('postgres',
"SELECT count(*), min(a), max(a) FROM public.tab1");
is($result, qq(0||), 'check rows on subscriber catchup');
+# Test for except column publications
+# Initial setup
+$node_publisher->safe_psql('postgres',
+ "CREATE TABLE tab2 (a int, b int NOT NULL, c int)");
+$node_publisher->safe_psql('postgres',
+ "CREATE TABLE sch1.tab2 (a int, b int, c int)");
+$node_publisher->safe_psql('postgres',
+ "CREATE TABLE tab3 (a int, b int, c int)");
+$node_publisher->safe_psql('postgres',
+ "CREATE TABLE tab4 (a int, b int GENERATED ALWAYS AS (a * 2) STORED, c int GENERATED ALWAYS AS (a * 3) STORED)"
+);
+$node_publisher->safe_psql('postgres', "INSERT INTO tab2 VALUES (1, 2, 3)");
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO sch1.tab2 VALUES (1, 2, 3)");
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION tap_pub_col FOR TABLE tab2 EXCEPT (a), sch1.tab2 EXCEPT (b, c)"
+);
+$node_subscriber->safe_psql('postgres',
+ "CREATE TABLE tab2 (a int, b int NOT NULL, c int)");
+$node_subscriber->safe_psql('postgres',
+ "CREATE TABLE sch1.tab2 (a int, b int, c int)");
+$node_subscriber->safe_psql('postgres',
+ "CREATE TABLE tab3 (a int, b int, c int)");
+$node_subscriber->safe_psql('postgres',
+ "CREATE TABLE tab4 (a int, b int, c int)");
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION tap_sub_col CONNECTION '$publisher_connstr' PUBLICATION tap_pub_col"
+);
+$node_subscriber->wait_for_subscription_sync($node_publisher, 'tap_sub_col');
+
+# Test initial sync
+$result = $node_subscriber->safe_psql('postgres', "SELECT * FROM tab2");
+is($result, qq(|2|3),
+ 'check that initial sync for except column publication');
+$result = $node_subscriber->safe_psql('postgres', "SELECT * FROM sch1.tab2");
+is($result, qq(1||), 'check that initial sync for except column publication');
+$node_publisher->safe_psql('postgres', "INSERT INTO tab2 VALUES (4, 5, 6)");
+$node_publisher->safe_psql('postgres',
+ "INSERT INTO sch1.tab2 VALUES (4, 5, 6)");
+$node_publisher->wait_for_catchup('tap_sub_col');
+
+# Test incremental changes
+$result = $node_subscriber->safe_psql('postgres', "SELECT * FROM tab2");
+is( $result, qq(|2|3
+|5|6),
+ 'check incremental insert for except column publication');
+$result = $node_subscriber->safe_psql('postgres', "SELECT * FROM sch1.tab2");
+is( $result, qq(1||
+4||), 'check incremental insert for except column publication');
+
+# Test for update
+$node_publisher->safe_psql('postgres',
+ "CREATE UNIQUE INDEX b_idx ON tab2 (b)");
+$node_publisher->safe_psql('postgres',
+ "ALTER TABLE tab2 REPLICA IDENTITY USING INDEX b_idx");
+$node_subscriber->safe_psql('postgres',
+ "CREATE UNIQUE INDEX b_idx ON tab2 (b)");
+$node_subscriber->safe_psql('postgres',
+ "ALTER TABLE tab2 REPLICA IDENTITY USING INDEX b_idx");
+$node_publisher->safe_psql('postgres',
+ "UPDATE tab2 SET a = 3, b = 4, c = 5 WHERE a = 1");
+$node_publisher->wait_for_catchup('tap_sub_col');
+$result = $node_subscriber->safe_psql('postgres', "SELECT * FROM tab2");
+is( $result, qq(|5|6
+|4|5),
+ 'check update for except column publication');
+
+# Test ALTER PUBLICATION for EXCEPT (col_list)
+$node_publisher->safe_psql('postgres',
+ "ALTER PUBLICATION tap_pub_col ADD TABLE tab3 EXCEPT(b)");
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub_col REFRESH PUBLICATION");
+$node_publisher->safe_psql('postgres', "INSERT INTO tab3 VALUES (1, 2, 3)");
+$node_publisher->wait_for_catchup('tap_sub_col');
+$result = $node_subscriber->safe_psql('postgres', "SELECT * FROM tab3");
+is($result, qq(1||3), 'check alter publication with EXCEPT');
+
+# Test for publication created on table with generated columns and column list
+# specified with EXCEPT
+$node_publisher->safe_psql('postgres', "INSERT INTO tab4 VALUES (1)");
+$node_publisher->safe_psql('postgres',
+ "ALTER PUBLICATION tap_pub_col SET (publish_generated_columns)");
+$node_publisher->safe_psql('postgres',
+ "ALTER PUBLICATION tap_pub_col SET TABLE tab4 EXCEPT(b)");
+$node_subscriber->safe_psql('postgres',
+ "ALTER SUBSCRIPTION tap_sub_col REFRESH PUBLICATION");
+$node_publisher->safe_psql('postgres', "INSERT INTO tab4 VALUES (2)");
+$node_publisher->wait_for_catchup('tap_sub_col');
+$result = $node_subscriber->safe_psql('postgres', "SELECT * FROM tab4");
+is( $result, qq(1||3
+2||6), 'check publication with generated columns and EXCEPT');
+
$node_subscriber->stop('fast');
$node_publisher->stop('fast');
--
2.34.1