Horton

Re: Skipping logical replication transactions on subscriber side

Masahiko Sawada <sawada.mshk@gmail.com>

View thread View on postgr.es
From: Masahiko Sawada <sawada.mshk@gmail.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Peter Eisentraut <peter.eisentraut@enterprisedb.com>, "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>, "tanghy.fnst@fujitsu.com" <tanghy.fnst@fujitsu.com>, "osumi.takamichi@fujitsu.com" <osumi.takamichi@fujitsu.com>, Alexey Lesovsky <lesovsky@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>, Greg Nancarrow <gregn4422@gmail.com>
Date: 2021-10-19T02:52:44Z
Lists: pgsql-hackers 
On Mon, Oct 18, 2021 at 6:07 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Mon, Oct 11, 2021 at 1:00 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> >
> > On Sun, Oct 10, 2021 at 11:04 PM Peter Eisentraut
> > <peter.eisentraut@enterprisedb.com> wrote:
> > >
> > > On 04.10.21 02:31, Masahiko Sawada wrote:
> > > > I guess disabling subscriptions on error/conflict and skipping the
> > > > particular transactions are somewhat different types of functions.
> > > > Disabling subscriptions on error/conflict seems likes a setting
> > > > parameter of subscriptions. The users might want to specify this
> > > > option at creation time. Whereas, skipping the particular transaction
> > > > is a repair function that the user might want to use on the spot in
> > > > case of a failure. I’m concerned a bit that combining these functions
> > > > to one syntax could confuse the users.
> > >
> > > Also, would the skip option be dumped and restored using pg_dump?  Maybe
> > > there is an argument for yes, but if not, then we probably need a
> > > different path of handling it separate from the more permanent options.
> >
> > Good point. I don’t think the skip option should be dumped and
> > restored using pg_dump since the utilization of transaction ids in
> > another installation is different.
> >
>
> This is a xid of publisher which subscriber wants to skip. So, even if
> one restores the subscriber data in a different installation why would
> it matter till it points to the same publisher?
>
> Either way, can't we handle this in pg_dump?

Because of backups (dumps), I think we cannot expect that the user
restore it somewhere soon. If the dump is restored several months
later, the publisher could be a different installation (by rebuilding
from scratch) or XID of the publisher could already be wrapped around.
It might be useful to dump the skip_xid by pg_dump in some cases, but
I think it should be optional if we want to do that.

Regards,

-- 
Masahiko Sawada
EDB:  https://www.enterprisedb.com/

Commits

  1. Test ALIGNOF_DOUBLE==4 compatibility under ALIGNOF_DOUBLE==8.

  2. Reorder subskiplsn in pg_subscription to avoid alignment issues.

  3. Add ALTER SUBSCRIPTION ... SKIP.

  4. Optionally disable subscriptions on error.

  5. Update docs of logical replication for commit 8d74fc96db.

  6. Respect permissions within logical replication.

  7. Fix regression test failure caused by commit 8d74fc96db.

  8. Add a view to show the stats of subscription workers.

  9. Add logical change details to logical replication worker errcontext.

  10. Rename LOGICAL_REP_MSG_STREAM_END to LOGICAL_REP_MSG_STREAM_STOP.

  11. Fix typo in protocol.sgml.

  12. Remove unused argument in apply_handle_commit_internal().

  13. Fix replication of in-progress transactions in tablesync worker.

  14. Reorder pg_sequence columns to avoid alignment issue