Horton

Re: Skipping logical replication transactions on subscriber side

Masahiko Sawada <sawada.mshk@gmail.com>

View thread View on postgr.es
From: Masahiko Sawada <sawada.mshk@gmail.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: "osumi.takamichi@fujitsu.com" <osumi.takamichi@fujitsu.com>, vignesh C <vignesh21@gmail.com>, Greg Nancarrow <gregn4422@gmail.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>, "tanghy.fnst@fujitsu.com" <tanghy.fnst@fujitsu.com>, Alexey Lesovsky <lesovsky@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2022-01-18T02:32:14Z
Lists: pgsql-hackers 
On Mon, Jan 17, 2022 at 10:15 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Mon, Jan 17, 2022 at 6:22 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> >
> > >
> > > (5)
> > >
> > > I can miss something here but, in one of
> > > the past discussions, there seems a consensus that
> > > if the user specifies XID of a subtransaction,
> > > it would be better to skip only the subtransaction.
> > >
> > > This time, is it out of the range of the patch ?
> > > If so, I suggest you include some description about it
> > > either in the commit message or around codes related to it.
> >
> > How can the user know subtransaction XID? I suppose you refer to
> > streaming protocol cases but while applying spooled changes we don't
> > report subtransaction XID neither in server log nor
> > pg_stat_subscription_workers.
> >
>
> I also think in the current system users won't be aware of
> subtransaction's XID but I feel Osumi-San's point is valid that we
> should at least add it in docs that we allow to skip only top-level
> xacts. Also, in the future, it won't be impossible to imagine that we
> can have subtransaction's XID info also available to users as we have
> that in the case of streaming xacts (See subxact_data).

Fair point and more accurate, but I'm a bit concerned that using these
words could confuse the user. There are some places in the doc where
we use the words “top-level transaction” and "sub transactions” but
these are not commonly used in the doc. The user normally would not be
aware that sub transactions are used to implement SAVEPOINTs. Also,
the publisher's subtransaction ID doesn’t appear anywhere on the
subscriber. So if we want to mention it, I think we should use other
words instead of them but I don’t have a good idea for that. Do you
have any ideas?

>
> Few minor points:
> ===============
> 1.
> + * the subscription if hte user has specified skip_xid.
>
> Typo. /hte/the

Will fix.

>
> 2.
> + * PREPARED won’t be resent but subskipxid is left.
>
> In diffmerge tool, won't is showing some funny chars. When I manually
> removed 't and added it again, everything is fine. I am not sure why
> it is so? I think Osumi-San has also raised this complaint.

Oh I didn't realize that. I'll check it again by using diffmerge tool.

>
> 3.
> + /*
> + * We don't expect that the user set the XID of the transaction that is
> + * rolled back but if the skip XID is set, clear it.
> + */
>
> /user set/user to set/

Will fix.

Regards,

-- 
Masahiko Sawada
EDB:  https://www.enterprisedb.com/

Commits

  1. Test ALIGNOF_DOUBLE==4 compatibility under ALIGNOF_DOUBLE==8.

  2. Reorder subskiplsn in pg_subscription to avoid alignment issues.

  3. Add ALTER SUBSCRIPTION ... SKIP.

  4. Optionally disable subscriptions on error.

  5. Update docs of logical replication for commit 8d74fc96db.

  6. Respect permissions within logical replication.

  7. Fix regression test failure caused by commit 8d74fc96db.

  8. Add a view to show the stats of subscription workers.

  9. Add logical change details to logical replication worker errcontext.

  10. Rename LOGICAL_REP_MSG_STREAM_END to LOGICAL_REP_MSG_STREAM_STOP.

  11. Fix typo in protocol.sgml.

  12. Remove unused argument in apply_handle_commit_internal().

  13. Fix replication of in-progress transactions in tablesync worker.

  14. Reorder pg_sequence columns to avoid alignment issue