Horton
Menu

Re: Column Filtering in Logical Replication

Amit Kapila <amit.kapila16@gmail.com>

View thread
From: Amit Kapila <amit.kapila16@gmail.com>
To: Alvaro Herrera <alvherre@alvh.no-ip.org>
Cc: vignesh C <vignesh21@gmail.com>, Rahila Syed <rahilasyed90@gmail.com>, Tomas Vondra <tomas.vondra@enterprisedb.com>, Peter Smith <smithpb2250@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2021-09-16T03:15:15Z
Lists: pgsql-hackers 
On Wed, Sep 15, 2021 at 6:06 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> On 2021-Sep-15, vignesh C wrote:
> > The patch
> > Generic_object_type_parser_002_table_schema_publication.patch has the
> > changes that were used to handle the parsing. Schema and Relation both
> > are different objects, schema is of string type and relation is of
> > RangeVar type. While parsing, schema name is parsed in string format
> > and relation is parsed and converted to rangevar type, these objects
> > will be then handled accordingly during post processing.
>
> Yeah, I think it'd be cleaner if the node type has two members, something like
> this
>
> typedef struct PublicationObjSpec
> {
>         NodeTag         type;
>         PublicationObjSpecType pubobjtype;      /* type of this publication object */
>         RangeVar        *rv;            /* if a table */
>         String          *objname;       /* if a schema */
>         int             location;               /* token location, or -1 if unknown */
> } PublicationObjSpec;
>
> and only one of them is set, the other is NULL, depending on the object type.
>

I think the problem here is that with the proposed grammar we won't be
always able to distinguish names at the gram.y stage. Some post
parsing analysis is required to attribute the right type to name as is
done in the patch. The same seems to be indicated by Tom in his email
as well where he has proposed this syntax [1]. Also, something similar
is done for privilege_target (GRANT syntax) where we have a list of
objects but here the story is slightly more advanced because we are
planning to allow specifying multiple objects in one command. One
might think that we can identify each type of objects lists separately
but that gives grammar conflicts as it is not able to identify whether
the comma ',' is used for the same type object or for the next type.
Due to which we need to come up with a generic object for names to
which we attribute the right type in post parse analysis. Now, I think
instead of void *, it might be better to use Node * for generic
objects unless we have some problem.

[1] - https://www.postgresql.org/message-id/877603.1629120678%40sss.pgh.pa.us

-- 
With Regards,
Amit Kapila.

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Doc: Explain about Column List feature.

  2. Doc: fix column list vs. replica identity rules.

  3. Prohibit combining publications with different column lists.

  4. Fix the check to limit sync workers.

  5. Wait for subscription to sync in t/031_column_list.sql

  6. Move prattrs to the pg_publication_rel section in docs

  7. Allow specifying column lists for logical replication

  8. Fix row filters with multiple publications

  9. Fix publish_as_relid with multiple publications

  10. Add some additional tests for row filters in logical replication.

  11. Add index on pg_publication_rel.prpubid

  12. Avoid using DefElemAction in AlterPublicationStmt

  13. Small cleanups related to PUBLICATION framework code

  14. Add PublicationTable and PublicationRelInfo structs

  15. Fix various concurrency issues in logical replication worker launching