Horton

Re: Skipping logical replication transactions on subscriber side

Amit Kapila <amit.kapila16@gmail.com>

View thread View on postgr.es
From: Amit Kapila <amit.kapila16@gmail.com>
To: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: "osumi.takamichi@fujitsu.com" <osumi.takamichi@fujitsu.com>, vignesh C <vignesh21@gmail.com>, Greg Nancarrow <gregn4422@gmail.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>, "tanghy.fnst@fujitsu.com" <tanghy.fnst@fujitsu.com>, Alexey Lesovsky <lesovsky@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2022-01-18T02:52:30Z
Lists: pgsql-hackers 
On Tue, Jan 18, 2022 at 8:02 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
>
> On Mon, Jan 17, 2022 at 10:15 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >
> > On Mon, Jan 17, 2022 at 6:22 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> > >
> > > >
> > > > (5)
> > > >
> > > > I can miss something here but, in one of
> > > > the past discussions, there seems a consensus that
> > > > if the user specifies XID of a subtransaction,
> > > > it would be better to skip only the subtransaction.
> > > >
> > > > This time, is it out of the range of the patch ?
> > > > If so, I suggest you include some description about it
> > > > either in the commit message or around codes related to it.
> > >
> > > How can the user know subtransaction XID? I suppose you refer to
> > > streaming protocol cases but while applying spooled changes we don't
> > > report subtransaction XID neither in server log nor
> > > pg_stat_subscription_workers.
> > >
> >
> > I also think in the current system users won't be aware of
> > subtransaction's XID but I feel Osumi-San's point is valid that we
> > should at least add it in docs that we allow to skip only top-level
> > xacts. Also, in the future, it won't be impossible to imagine that we
> > can have subtransaction's XID info also available to users as we have
> > that in the case of streaming xacts (See subxact_data).
>
> Fair point and more accurate, but I'm a bit concerned that using these
> words could confuse the user. There are some places in the doc where
> we use the words “top-level transaction” and "sub transactions” but
> these are not commonly used in the doc. The user normally would not be
> aware that sub transactions are used to implement SAVEPOINTs. Also,
> the publisher's subtransaction ID doesn’t appear anywhere on the
> subscriber. So if we want to mention it, I think we should use other
> words instead of them but I don’t have a good idea for that. Do you
> have any ideas?
>

How about changing existing text:
+          Specifies the ID of the transaction whose changes are to be skipped
+          by the logical replication worker.  Setting <literal>NONE</literal>
+          resets the transaction ID.

to

Specifies the top-level transaction identifier whose changes are to be
skipped by the logical replication worker.  We don't support skipping
individual subtransactions.  Setting <literal>NONE</literal> resets
the transaction ID.

-- 
With Regards,
Amit Kapila.

Commits

  1. Test ALIGNOF_DOUBLE==4 compatibility under ALIGNOF_DOUBLE==8.

  2. Reorder subskiplsn in pg_subscription to avoid alignment issues.

  3. Add ALTER SUBSCRIPTION ... SKIP.

  4. Optionally disable subscriptions on error.

  5. Update docs of logical replication for commit 8d74fc96db.

  6. Respect permissions within logical replication.

  7. Fix regression test failure caused by commit 8d74fc96db.

  8. Add a view to show the stats of subscription workers.

  9. Add logical change details to logical replication worker errcontext.

  10. Rename LOGICAL_REP_MSG_STREAM_END to LOGICAL_REP_MSG_STREAM_STOP.

  11. Fix typo in protocol.sgml.

  12. Remove unused argument in apply_handle_commit_internal().

  13. Fix replication of in-progress transactions in tablesync worker.

  14. Reorder pg_sequence columns to avoid alignment issue