Horton

Re: Skipping logical replication transactions on subscriber side

Masahiko Sawada <sawada.mshk@gmail.com>

View thread View on postgr.es
From: Masahiko Sawada <sawada.mshk@gmail.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: "tanghy.fnst@fujitsu.com" <tanghy.fnst@fujitsu.com>, Greg Nancarrow <gregn4422@gmail.com>, "osumi.takamichi@fujitsu.com" <osumi.takamichi@fujitsu.com>, "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>, Alexey Lesovsky <lesovsky@gmail.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2021-08-25T04:22:11Z
Lists: pgsql-hackers

Attachments

On Tue, Aug 24, 2021 at 10:05 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Tue, Aug 24, 2021 at 11:44 AM tanghy.fnst@fujitsu.com
> <tanghy.fnst@fujitsu.com> wrote:
> >
> > On Monday, August 23, 2021 11:09 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> > >
> > > I've attached updated patches. Please review them.
> > >
> >
> > I tested v10-0001 patch in both streaming and no-streaming more. All tests works well.
> >
> > I also tried two-phase commit feature, the error context was set as expected,
> > but please allow me to propose a fix suggestion on the error description:

Thank you for the suggestion!

> >
> > CONTEXT:  processing remote data during "INSERT" for replication target relation
> > "public.test" in transaction 714 with commit timestamp 2021-08-24
> > 13:20:22.480532+08
> >
> > It said "commit timestamp", but for 2pc feature, the timestamp could be "prepare timestamp" or "rollback timestamp", too.
> > Could we make some change to make the error log more comprehensive?
> >
>
> I think we can write something like: (processing remote data during
> "INSERT" for replication target relation "public.test" in transaction
> 714 at 2021-08-24 13:20:22.480532+08). Basically replacing "with
> commit timestamp" with "at". This is similar to what we do
> test_decoding module for transaction timestamp.

+1

> The other idea could
> be we print the exact operation like commit/prepare/rollback which is
> also possible because we have that information while setting context
> info but that might add a bit more complexity which I don't think is
> worth it.

Agreed.

I replaced "with commit timestamp" with "at" and rename 'commit_ts'
field name to 'ts'.

>
> One more point about the  v10-0001* patch: From the commit message
> "Add logical changes details to errcontext of apply worker errors.",
> it appears that the context will be added only for the apply worker
> but won't it get added for tablesync worker as well during its sync
> phase (when it tries to catch up with apply worker)?

Right. I've updated the message.

Attached updated version patches. Please review them.

Regards,

--
Masahiko Sawada
EDB:  https://www.enterprisedb.com/

Commits

  1. Test ALIGNOF_DOUBLE==4 compatibility under ALIGNOF_DOUBLE==8.

  2. Reorder subskiplsn in pg_subscription to avoid alignment issues.

  3. Add ALTER SUBSCRIPTION ... SKIP.

  4. Optionally disable subscriptions on error.

  5. Update docs of logical replication for commit 8d74fc96db.

  6. Respect permissions within logical replication.

  7. Fix regression test failure caused by commit 8d74fc96db.

  8. Add a view to show the stats of subscription workers.

  9. Add logical change details to logical replication worker errcontext.

  10. Rename LOGICAL_REP_MSG_STREAM_END to LOGICAL_REP_MSG_STREAM_STOP.

  11. Fix typo in protocol.sgml.

  12. Remove unused argument in apply_handle_commit_internal().

  13. Fix replication of in-progress transactions in tablesync worker.

  14. Reorder pg_sequence columns to avoid alignment issue