Horton
Menu

Re: Column Filtering in Logical Replication

vignesh C <vignesh21@gmail.com>

View thread
From: vignesh C <vignesh21@gmail.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>, Rahila Syed <rahilasyed90@gmail.com>, Tomas Vondra <tomas.vondra@enterprisedb.com>, Peter Smith <smithpb2250@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2021-09-16T05:05:21Z
Lists: pgsql-hackers

Attachments

On Thu, Sep 16, 2021 at 8:45 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Wed, Sep 15, 2021 at 6:06 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> >
> > On 2021-Sep-15, vignesh C wrote:
> > > The patch
> > > Generic_object_type_parser_002_table_schema_publication.patch has the
> > > changes that were used to handle the parsing. Schema and Relation both
> > > are different objects, schema is of string type and relation is of
> > > RangeVar type. While parsing, schema name is parsed in string format
> > > and relation is parsed and converted to rangevar type, these objects
> > > will be then handled accordingly during post processing.
> >
> > Yeah, I think it'd be cleaner if the node type has two members, something like
> > this
> >
> > typedef struct PublicationObjSpec
> > {
> >         NodeTag         type;
> >         PublicationObjSpecType pubobjtype;      /* type of this publication object */
> >         RangeVar        *rv;            /* if a table */
> >         String          *objname;       /* if a schema */
> >         int             location;               /* token location, or -1 if unknown */
> > } PublicationObjSpec;
> >
> > and only one of them is set, the other is NULL, depending on the object type.
> >
>
> I think the problem here is that with the proposed grammar we won't be
> always able to distinguish names at the gram.y stage.

This is the issue that Amit was talking about:
gram.y: error: shift/reduce conflicts: 2 found, 0 expected
gram.y: warning: shift/reduce conflict on token ',' [-Wcounterexamples]
  First example: CREATE PUBLICATION name FOR TABLE relation_expr_list
• ',' relation_expr ',' PublicationObjSpec opt_definition $end
  Shift derivation
    $accept
    ↳ parse_toplevel
                                                              $end
      ↳ stmtmulti
        ↳ toplevel_stmt
          ↳ stmt
            ↳ CreatePublicationStmt
              ↳ CREATE PUBLICATION name FOR pub_obj_list
                                               opt_definition
                                            ↳ PublicationObjSpec
                        ',' PublicationObjSpec
                                              ↳ TABLE relation_expr_list
                                                      ↳
relation_expr_list • ',' relation_expr
  Second example: CREATE PUBLICATION name FOR TABLE relation_expr_list
• ',' PublicationObjSpec opt_definition $end
  Reduce derivation
    $accept
    ↳ parse_toplevel
                                            $end
      ↳ stmtmulti
        ↳ toplevel_stmt
          ↳ stmt
            ↳ CreatePublicationStmt
              ↳ CREATE PUBLICATION name FOR pub_obj_list
                             opt_definition
                                            ↳ pub_obj_list
      ',' PublicationObjSpec
                                              ↳ PublicationObjSpec
                                                ↳ TABLE relation_expr_list •
Here it is not able to distinguish if ',' is used for the next table
name or the next object.
I was able to reproduce this issue with the attached patch.

Regards,
Vignesh

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Doc: Explain about Column List feature.

  2. Doc: fix column list vs. replica identity rules.

  3. Prohibit combining publications with different column lists.

  4. Fix the check to limit sync workers.

  5. Wait for subscription to sync in t/031_column_list.sql

  6. Move prattrs to the pg_publication_rel section in docs

  7. Allow specifying column lists for logical replication

  8. Fix row filters with multiple publications

  9. Fix publish_as_relid with multiple publications

  10. Add some additional tests for row filters in logical replication.

  11. Add index on pg_publication_rel.prpubid

  12. Avoid using DefElemAction in AlterPublicationStmt

  13. Small cleanups related to PUBLICATION framework code

  14. Add PublicationTable and PublicationRelInfo structs

  15. Fix various concurrency issues in logical replication worker launching