Horton

Re: Skipping logical replication transactions on subscriber side

Masahiko Sawada <sawada.mshk@gmail.com>

View thread View on postgr.es
From: Masahiko Sawada <sawada.mshk@gmail.com>
To: "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>
Cc: "osumi.takamichi@fujitsu.com" <osumi.takamichi@fujitsu.com>, Alexey Lesovsky <lesovsky@gmail.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>, Amit Kapila <amit.kapila16@gmail.com>
Date: 2021-08-19T01:47:30Z
Lists: pgsql-hackers

Attachments

On Wed, Aug 18, 2021 at 5:39 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
>
> On Wed, Aug 18, 2021 at 3:33 PM houzj.fnst@fujitsu.com
> <houzj.fnst@fujitsu.com> wrote:
> >
> > On Tues, Aug 17, 2021 1:01 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> > > On Mon, Aug 16, 2021 at 3:59 PM houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com> wrote:
> > > > 3)
> > > > Do we need to invoke set_apply_error_context_xact() in the function
> > > > apply_handle_stream_prepare() to save the xid and timestamp ?
> > >
> > > Yes. I think that v8-0001 patch already set xid and timestamp just after parsing
> > > stream_prepare message. You meant it's not necessary?
> >
> > Sorry, I thought of something wrong, please ignore the above comment.
> >
> > >
> > > I'll submit the updated patches soon.
> >
> > I was thinking about the place to set the errcallback.callback.
> >
> > apply_dispatch(StringInfo s)
> >  {
> >         LogicalRepMsgType action = pq_getmsgbyte(s);
> > +       ErrorContextCallback errcallback;
> > +       bool            set_callback = false;
> > +
> > +       /*
> > +        * Push apply error context callback if not yet. Other fields will be
> > +        * filled during applying the change.  Since this function can be called
> > +        * recursively when applying spooled changes, we set the callback only
> > +        * once.
> > +        */
> > +       if (apply_error_callback_arg.command == 0)
> > +       {
> > +               errcallback.callback = apply_error_callback;
> > +               errcallback.previous = error_context_stack;
> > +               error_context_stack = &errcallback;
> > +               set_callback = true;
> > +       }
> > ...
> > +       /* Pop the error context stack */
> > +       if (set_callback)
> > +               error_context_stack = errcallback.previous;
> >
> > It seems we can put the above code in the function LogicalRepApplyLoop()
> > around invoking apply_dispatch(), and in that approach we don't need to worry
> > about the recursively case. What do you think ?
>
> Thank you for the comment!
>
> I think you're right. Maybe we can set the callback before entering to
> the main loop and pop it after breaking from it. It would also fix the
> problem reported by Tang[1]. But one thing we need to note that since
> we want to reset apply_error_callback_arg.command at the end of
> apply_dispatch() (otherwise we could end up setting the apply error
> context to an irrelevant error such as network error), when
> apply_dispatch() is called recursively probably we need to save the
> apply_error_callback_arg.command before setting the new command and
> then revert back to the saved command. Is that right?

I've attached the updated version patches that incorporated all
comments I got so far unless I'm missing something. Please review
them.

Regards,

--
Masahiko Sawada
EDB:  https://www.enterprisedb.com/

Commits

  1. Test ALIGNOF_DOUBLE==4 compatibility under ALIGNOF_DOUBLE==8.

  2. Reorder subskiplsn in pg_subscription to avoid alignment issues.

  3. Add ALTER SUBSCRIPTION ... SKIP.

  4. Optionally disable subscriptions on error.

  5. Update docs of logical replication for commit 8d74fc96db.

  6. Respect permissions within logical replication.

  7. Fix regression test failure caused by commit 8d74fc96db.

  8. Add a view to show the stats of subscription workers.

  9. Add logical change details to logical replication worker errcontext.

  10. Rename LOGICAL_REP_MSG_STREAM_END to LOGICAL_REP_MSG_STREAM_STOP.

  11. Fix typo in protocol.sgml.

  12. Remove unused argument in apply_handle_commit_internal().

  13. Fix replication of in-progress transactions in tablesync worker.

  14. Reorder pg_sequence columns to avoid alignment issue