Horton

Re: Skipping logical replication transactions on subscriber side

Masahiko Sawada <sawada.mshk@gmail.com>

View thread View on postgr.es
From: Masahiko Sawada <sawada.mshk@gmail.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: "osumi.takamichi@fujitsu.com" <osumi.takamichi@fujitsu.com>, "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>, Alexey Lesovsky <lesovsky@gmail.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2021-08-12T05:53:15Z
Lists: pgsql-hackers

Attachments

On Wed, Aug 11, 2021 at 5:19 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Wed, Aug 11, 2021 at 11:19 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> >
> > On Tue, Aug 10, 2021 at 7:18 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > > ==============================================================
> > >
> > > 1. While applying DML operations, we are setting up the error context
> > > multiple times due to which the context information is not
> > > appropriate. The first is set in apply_dispatch and then during
> > > processing, we set another error callback slot_store_error_callback in
> > > slot_store_data and slot_modify_data. When I forced one of the errors
> > > in slot_store_data(), it displays the below information in CONTEXT
> > > which doesn't make much sense.
> > >
> > > 2021-08-10 15:16:39.887 IST [6784] ERROR:  incorrect binary data
> > > format in logical replication column 1
> > > 2021-08-10 15:16:39.887 IST [6784] CONTEXT:  processing remote data
> > > for replication target relation "public.test1" column "id"
> > >         during apply of "INSERT" for relation "public.test1" in
> > > transaction with xid 740 committs 2021-08-10 14:44:38.058174+05:30
> >
> > Yes, but we cannot change the error context message depending on other
> > error context messages. So it seems hard to construct a complete
> > sentence in the context message that is okay in terms of English
> > grammar. Is the following message better?
> >
> > CONTEXT:  processing remote data for replication target relation
> > "public.test1" column “id"
> >          applying "INSERT" for relation "public.test1” in transaction
> > with xid 740 committs 2021-08-10 14:44:38.058174+05:30
> >
>
> I don't like the proposed text. How about if we combine both and have
> something like: "processing remote data during "UPDATE" for
> replication target relation "public.test1" column "id" in transaction
> id 740 with commit timestamp 2021-08-10 14:44:38.058174+05:30"? For
> this, I think we need to remove slot_store_error_callback and
> add/change the ApplyErrCallbackArg to include the additional required
> information in that callback.

Oh, I've never thought about that. That's a good idea.

I've attached the updated patches. FYI I've included the patch
(v8-0005) that fixes the assertion failure during shared fileset
cleanup to make cfbot tests happy.

Regards,

--
Masahiko Sawada
EDB:  https://www.enterprisedb.com/

Commits

  1. Test ALIGNOF_DOUBLE==4 compatibility under ALIGNOF_DOUBLE==8.

  2. Reorder subskiplsn in pg_subscription to avoid alignment issues.

  3. Add ALTER SUBSCRIPTION ... SKIP.

  4. Optionally disable subscriptions on error.

  5. Update docs of logical replication for commit 8d74fc96db.

  6. Respect permissions within logical replication.

  7. Fix regression test failure caused by commit 8d74fc96db.

  8. Add a view to show the stats of subscription workers.

  9. Add logical change details to logical replication worker errcontext.

  10. Rename LOGICAL_REP_MSG_STREAM_END to LOGICAL_REP_MSG_STREAM_STOP.

  11. Fix typo in protocol.sgml.

  12. Remove unused argument in apply_handle_commit_internal().

  13. Fix replication of in-progress transactions in tablesync worker.

  14. Reorder pg_sequence columns to avoid alignment issue