Horton

Re: Skipping logical replication transactions on subscriber side

Amit Kapila <amit.kapila16@gmail.com>

View thread View on postgr.es
From: Amit Kapila <amit.kapila16@gmail.com>
To: "tanghy.fnst@fujitsu.com" <tanghy.fnst@fujitsu.com>
Cc: Masahiko Sawada <sawada.mshk@gmail.com>, Greg Nancarrow <gregn4422@gmail.com>, "osumi.takamichi@fujitsu.com" <osumi.takamichi@fujitsu.com>, "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>, Alexey Lesovsky <lesovsky@gmail.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2021-08-24T13:05:01Z
Lists: pgsql-hackers 
On Tue, Aug 24, 2021 at 11:44 AM tanghy.fnst@fujitsu.com
<tanghy.fnst@fujitsu.com> wrote:
>
> On Monday, August 23, 2021 11:09 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> >
> > I've attached updated patches. Please review them.
> >
>
> I tested v10-0001 patch in both streaming and no-streaming more. All tests works well.
>
> I also tried two-phase commit feature, the error context was set as expected,
> but please allow me to propose a fix suggestion on the error description:
>
> CONTEXT:  processing remote data during "INSERT" for replication target relation
> "public.test" in transaction 714 with commit timestamp 2021-08-24
> 13:20:22.480532+08
>
> It said "commit timestamp", but for 2pc feature, the timestamp could be "prepare timestamp" or "rollback timestamp", too.
> Could we make some change to make the error log more comprehensive?
>

I think we can write something like: (processing remote data during
"INSERT" for replication target relation "public.test" in transaction
714 at 2021-08-24 13:20:22.480532+08). Basically replacing "with
commit timestamp" with "at". This is similar to what we do
test_decoding module for transaction timestamp. The other idea could
be we print the exact operation like commit/prepare/rollback which is
also possible because we have that information while setting context
info but that might add a bit more complexity which I don't think is
worth it.

One more point about the  v10-0001* patch: From the commit message
"Add logical changes details to errcontext of apply worker errors.",
it appears that the context will be added only for the apply worker
but won't it get added for tablesync worker as well during its sync
phase (when it tries to catch up with apply worker)?

-- 
With Regards,
Amit Kapila.

Commits

  1. Test ALIGNOF_DOUBLE==4 compatibility under ALIGNOF_DOUBLE==8.

  2. Reorder subskiplsn in pg_subscription to avoid alignment issues.

  3. Add ALTER SUBSCRIPTION ... SKIP.

  4. Optionally disable subscriptions on error.

  5. Update docs of logical replication for commit 8d74fc96db.

  6. Respect permissions within logical replication.

  7. Fix regression test failure caused by commit 8d74fc96db.

  8. Add a view to show the stats of subscription workers.

  9. Add logical change details to logical replication worker errcontext.

  10. Rename LOGICAL_REP_MSG_STREAM_END to LOGICAL_REP_MSG_STREAM_STOP.

  11. Fix typo in protocol.sgml.

  12. Remove unused argument in apply_handle_commit_internal().

  13. Fix replication of in-progress transactions in tablesync worker.

  14. Reorder pg_sequence columns to avoid alignment issue