Horton

Re: Skipping logical replication transactions on subscriber side

Amit Kapila <amit.kapila16@gmail.com>

View thread View on postgr.es
From: Amit Kapila <amit.kapila16@gmail.com>
To: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: "osumi.takamichi@fujitsu.com" <osumi.takamichi@fujitsu.com>, "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>, Alexey Lesovsky <lesovsky@gmail.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2021-08-11T08:18:52Z
Lists: pgsql-hackers 
On Wed, Aug 11, 2021 at 11:19 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
>
> On Tue, Aug 10, 2021 at 7:18 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > ==============================================================
> >
> > 1. While applying DML operations, we are setting up the error context
> > multiple times due to which the context information is not
> > appropriate. The first is set in apply_dispatch and then during
> > processing, we set another error callback slot_store_error_callback in
> > slot_store_data and slot_modify_data. When I forced one of the errors
> > in slot_store_data(), it displays the below information in CONTEXT
> > which doesn't make much sense.
> >
> > 2021-08-10 15:16:39.887 IST [6784] ERROR:  incorrect binary data
> > format in logical replication column 1
> > 2021-08-10 15:16:39.887 IST [6784] CONTEXT:  processing remote data
> > for replication target relation "public.test1" column "id"
> >         during apply of "INSERT" for relation "public.test1" in
> > transaction with xid 740 committs 2021-08-10 14:44:38.058174+05:30
>
> Yes, but we cannot change the error context message depending on other
> error context messages. So it seems hard to construct a complete
> sentence in the context message that is okay in terms of English
> grammar. Is the following message better?
>
> CONTEXT:  processing remote data for replication target relation
> "public.test1" column “id"
>          applying "INSERT" for relation "public.test1” in transaction
> with xid 740 committs 2021-08-10 14:44:38.058174+05:30
>

I don't like the proposed text. How about if we combine both and have
something like: "processing remote data during "UPDATE" for
replication target relation "public.test1" column "id" in transaction
id 740 with commit timestamp 2021-08-10 14:44:38.058174+05:30"? For
this, I think we need to remove slot_store_error_callback and
add/change the ApplyErrCallbackArg to include the additional required
information in that callback.

-- 
With Regards,
Amit Kapila.

Commits

  1. Test ALIGNOF_DOUBLE==4 compatibility under ALIGNOF_DOUBLE==8.

  2. Reorder subskiplsn in pg_subscription to avoid alignment issues.

  3. Add ALTER SUBSCRIPTION ... SKIP.

  4. Optionally disable subscriptions on error.

  5. Update docs of logical replication for commit 8d74fc96db.

  6. Respect permissions within logical replication.

  7. Fix regression test failure caused by commit 8d74fc96db.

  8. Add a view to show the stats of subscription workers.

  9. Add logical change details to logical replication worker errcontext.

  10. Rename LOGICAL_REP_MSG_STREAM_END to LOGICAL_REP_MSG_STREAM_STOP.

  11. Fix typo in protocol.sgml.

  12. Remove unused argument in apply_handle_commit_internal().

  13. Fix replication of in-progress transactions in tablesync worker.

  14. Reorder pg_sequence columns to avoid alignment issue