Horton

Re: Skipping logical replication transactions on subscriber side

Masahiko Sawada <sawada.mshk@gmail.com>

View thread View on postgr.es
From: Masahiko Sawada <sawada.mshk@gmail.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: vignesh C <vignesh21@gmail.com>, Greg Nancarrow <gregn4422@gmail.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>, "tanghy.fnst@fujitsu.com" <tanghy.fnst@fujitsu.com>, "osumi.takamichi@fujitsu.com" <osumi.takamichi@fujitsu.com>, Alexey Lesovsky <lesovsky@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2022-01-11T06:01:23Z
Lists: pgsql-hackers 
On Tue, Jan 11, 2022 at 11:22 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Mon, Jan 10, 2022 at 2:57 PM vignesh C <vignesh21@gmail.com> wrote:
> >
> > 2) Can we have an option to specify last_error_xid of
> > pg_stat_subscription_workers. Something like:
> > alter subscription sub1 skip ( XID = 'last_subscription_error');
> >
> > When the user specified last_subscription_error, it should pick
> > last_error_xid from pg_stat_subscription_workers.
> > As this operation is a critical operation, if there is an option which
> > could automatically pick and set from pg_stat_subscription_workers, it
> > would be useful.
> >
>
> I think having some automatic functionality around this would be good
> but I am not so sure about this idea because it is possible that the
> error has not reached the stats collector and the user might be
> referring to server logs to set the skip xid. In such cases, even
> though an error would have occurred but we won't be able to set the
> required xid. Now, one can imagine that if we don't get the required
> value from pg_stat_subscription_workers then we can return an error to
> the user indicating that she can cross-verify the server logs and set
> the appropriate xid value but IMO it could be confusing. I feel even
> if we want some automatic functionality like you are proposing or
> something else, it could be done as a separate patch but let's wait
> and see what Sawada-San or others think about this?

Agreed. The automatically setting XID would be a good idea but we can
do that in a separate patch so we can keep the first patch simple.

Regards,

--
Masahiko Sawada
EDB:  https://www.enterprisedb.com/

Commits

  1. Test ALIGNOF_DOUBLE==4 compatibility under ALIGNOF_DOUBLE==8.

  2. Reorder subskiplsn in pg_subscription to avoid alignment issues.

  3. Add ALTER SUBSCRIPTION ... SKIP.

  4. Optionally disable subscriptions on error.

  5. Update docs of logical replication for commit 8d74fc96db.

  6. Respect permissions within logical replication.

  7. Fix regression test failure caused by commit 8d74fc96db.

  8. Add a view to show the stats of subscription workers.

  9. Add logical change details to logical replication worker errcontext.

  10. Rename LOGICAL_REP_MSG_STREAM_END to LOGICAL_REP_MSG_STREAM_STOP.

  11. Fix typo in protocol.sgml.

  12. Remove unused argument in apply_handle_commit_internal().

  13. Fix replication of in-progress transactions in tablesync worker.

  14. Reorder pg_sequence columns to avoid alignment issue