Re: Skipping logical replication transactions on subscriber side

Masahiko Sawada <sawada.mshk@gmail.com>

From: Masahiko Sawada <sawada.mshk@gmail.com>
To: vignesh C <vignesh21@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>, Alexey Lesovsky <lesovsky@gmail.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2021-08-04T11:43:33Z
Lists: pgsql-hackers
On Tue, Aug 3, 2021 at 7:54 PM vignesh C <vignesh21@gmail.com> wrote:
>
> On Tue, Aug 3, 2021 at 12:20 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> >
> > On Mon, Aug 2, 2021 at 12:21 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > >
> > > On Mon, Aug 2, 2021 at 7:45 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> > > >
> > > > On Fri, Jul 30, 2021 at 12:52 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > > > >
> > > > > On Thu, Jul 29, 2021 at 11:18 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> > > > >
> > > > > Setting up logical rep error context in a generic function looks a bit
> > > > > odd to me. Do we really need to set up error context here? I
> > > > > understand we can't do this in caller but anyway I think we are not
> > > > > sending this to logical replication view as well, so not sure we need
> > > > > to do it here.
> > > >
> > > > Yeah, I'm not convinced of this part yet. I wanted to show relid also
> > > > in truncate cases but I came up with only this idea.
> > > >
> > > > If an error happens during truncating the table (in
> > > > ExecuteTruncateGuts()), relid set by
> > > > set_logicalrep_error_context_rel() is actually sent to the view. If we
> > > > don’t have it, the view always shows relid as NULL in truncate cases.
> > > > On the other hand, it doesn’t cover all cases. For example, it doesn’t
> > > > cover an error that the target table doesn’t exist on the subscriber,
> > > > which happens when opening the target table. Anyway, in most cases,
> > > > even if relid is NULL, the error message in the view helps users to
> > > > know which relation the error happened on. What do you think?
> > > >
> > >
> > > Yeah, I also think at this stage error message is sufficient in such cases.
> >
> > I've attached new patches that incorporate all comments I got so far.
> > Please review them.
>
> I had a look at the first patch, couple of minor comments:
> 1) Should we include this in typedefs.lst
> +/* Struct for saving and restoring apply information */
> +typedef struct ApplyErrCallbackArg
> +{
> +       LogicalRepMsgType command;      /* 0 if invalid */
> +
> +       /* Local relation information */
> +       char       *nspname;
>
> 2)  We can keep the case statement in the same order as in the
> LogicalRepMsgType enum, this will help in easily identifying if any
> enum gets missed.
> +               case LOGICAL_REP_MSG_RELATION:
> +                       return "RELATION";
> +               case LOGICAL_REP_MSG_TYPE:
> +                       return "TYPE";
> +               case LOGICAL_REP_MSG_ORIGIN:
> +                       return "ORIGIN";
> +               case LOGICAL_REP_MSG_MESSAGE:
> +                       return "MESSAGE";
> +               case LOGICAL_REP_MSG_STREAM_START:
> +                       return "STREAM START";
>

Thank you for reviewing the patch!

I agreed with all comments and will fix them in the next version patch.

Regards,

-- 
Masahiko Sawada
EDB:  https://www.enterprisedb.com/



Commits

  1. Test ALIGNOF_DOUBLE==4 compatibility under ALIGNOF_DOUBLE==8.

  2. Reorder subskiplsn in pg_subscription to avoid alignment issues.

  3. Add ALTER SUBSCRIPTION ... SKIP.

  4. Optionally disable subscriptions on error.

  5. Update docs of logical replication for commit 8d74fc96db.

  6. Respect permissions within logical replication.

  7. Fix regression test failure caused by commit 8d74fc96db.

  8. Add a view to show the stats of subscription workers.

  9. Add logical change details to logical replication worker errcontext.

  10. Rename LOGICAL_REP_MSG_STREAM_END to LOGICAL_REP_MSG_STREAM_STOP.

  11. Fix typo in protocol.sgml.

  12. Remove unused argument in apply_handle_commit_internal().

  13. Fix replication of in-progress transactions in tablesync worker.

  14. Reorder pg_sequence columns to avoid alignment issue