Horton
Menu

Re: Column Filtering in Logical Replication

Alvaro Herrera <alvherre@alvh.no-ip.org>

View thread
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
To: vignesh C <vignesh21@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Rahila Syed <rahilasyed90@gmail.com>, Tomas Vondra <tomas.vondra@enterprisedb.com>, Peter Smith <smithpb2250@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2021-09-15T12:36:51Z
Lists: pgsql-hackers 
On 2021-Sep-15, vignesh C wrote:

> I have extracted the parser code and attached it here, so that it will
> be easy to go through. We wanted to support the following syntax as in
> [1]:
> CREATE PUBLICATION pub1 FOR
> TABLE t1,t2,t3, ALL TABLES IN SCHEMA s1,s2,
> SEQUENCE seq1,seq2, ALL SEQUENCES IN SCHEMA s3,s4;

Oh, thanks, it looks like this can be useful.  We can get the common
grammar done and then rebase all the other patches (I was also just told
about support for sequences in [1]) on top.

[1] https://postgr.es/m/3d6df331-5532-6848-eb45-344b265e0238@enterprisedb.com

> Columns can be added to PublicationObjSpec data structure.

Right.  (As a List of String, I imagine.)

> The patch
> Generic_object_type_parser_002_table_schema_publication.patch has the
> changes that were used to handle the parsing. Schema and Relation both
> are different objects, schema is of string type and relation is of
> RangeVar type. While parsing, schema name is parsed in string format
> and relation is parsed and converted to rangevar type, these objects
> will be then handled accordingly during post processing.

Yeah, I think it'd be cleaner if the node type has two members, something like
this

typedef struct PublicationObjSpec
{
	NodeTag		type;
	PublicationObjSpecType pubobjtype;	/* type of this publication object */
	RangeVar	*rv;		/* if a table */
	String		*objname;	/* if a schema */
	int		location;		/* token location, or -1 if unknown */
} PublicationObjSpec;

and only one of them is set, the other is NULL, depending on the object type.

-- 
Álvaro Herrera           39°49'30"S 73°17'W  —  https://www.EnterpriseDB.com/

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Doc: Explain about Column List feature.

  2. Doc: fix column list vs. replica identity rules.

  3. Prohibit combining publications with different column lists.

  4. Fix the check to limit sync workers.

  5. Wait for subscription to sync in t/031_column_list.sql

  6. Move prattrs to the pg_publication_rel section in docs

  7. Allow specifying column lists for logical replication

  8. Fix row filters with multiple publications

  9. Fix publish_as_relid with multiple publications

  10. Add some additional tests for row filters in logical replication.

  11. Add index on pg_publication_rel.prpubid

  12. Avoid using DefElemAction in AlterPublicationStmt

  13. Small cleanups related to PUBLICATION framework code

  14. Add PublicationTable and PublicationRelInfo structs

  15. Fix various concurrency issues in logical replication worker launching