Horton

Re: Skipping logical replication transactions on subscriber side

Amit Kapila <amit.kapila16@gmail.com>

View thread View on postgr.es
From: Amit Kapila <amit.kapila16@gmail.com>
To: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: Peter Eisentraut <peter.eisentraut@enterprisedb.com>, "David G. Johnston" <david.g.johnston@gmail.com>, "tanghy.fnst@fujitsu.com" <tanghy.fnst@fujitsu.com>, vignesh C <vignesh21@gmail.com>, Greg Nancarrow <gregn4422@gmail.com>, "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>, "osumi.takamichi@fujitsu.com" <osumi.takamichi@fujitsu.com>, Alexey Lesovsky <lesovsky@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2022-02-14T09:16:29Z
Lists: pgsql-hackers 
On Fri, Feb 11, 2022 at 7:40 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
>
> On Thu, Jan 27, 2022 at 10:42 PM Peter Eisentraut
> <peter.eisentraut@enterprisedb.com> wrote:
> >
> > On 26.01.22 05:05, Masahiko Sawada wrote:
> > >> I think it is okay to clear after the first successful application of
> > >> any transaction. What I was not sure was about the idea of giving
> > >> WARNING/ERROR if the first xact to be applied is not the same as
> > >> skip_xid.
> > > Do you prefer not to do anything in this case?
> >
> > I think a warning would be sensible.  If the user specifies to skip a
> > certain transaction and then that doesn't happen, we should at least say
> > something.
>
> Meanwhile waiting for comments on the discussion about the designs of
> both pg_stat_subscription_workers and ALTER SUBSCRIPTION SKIP feature,
> I’ve incorporated some (minor) comments on the current design patch,
> which includes:
>
> * Use LSN instead of XID.
>

I think exposing LSN is a better approach as it doesn't have the
dangers of wraparound. And, I think users can use it with the existing
function pg_replication_origin_advance() which will save us from
adding additional code for this feature. We can explain/expand in docs
how users can use the error information from view/error_logs and use
the existing function to skip conflicting transactions. We might want
to even expose error_origin to make it a bit easier for users but not
sure. I feel the need for the new syntax (and then added code
complexity due to that) isn't warranted if we expose error_LSN and let
users use it with the existing functions.

Do you see any problem with the same?

-- 
With Regards,
Amit Kapila.

Commits

  1. Test ALIGNOF_DOUBLE==4 compatibility under ALIGNOF_DOUBLE==8.

  2. Reorder subskiplsn in pg_subscription to avoid alignment issues.

  3. Add ALTER SUBSCRIPTION ... SKIP.

  4. Optionally disable subscriptions on error.

  5. Update docs of logical replication for commit 8d74fc96db.

  6. Respect permissions within logical replication.

  7. Fix regression test failure caused by commit 8d74fc96db.

  8. Add a view to show the stats of subscription workers.

  9. Add logical change details to logical replication worker errcontext.

  10. Rename LOGICAL_REP_MSG_STREAM_END to LOGICAL_REP_MSG_STREAM_STOP.

  11. Fix typo in protocol.sgml.

  12. Remove unused argument in apply_handle_commit_internal().

  13. Fix replication of in-progress transactions in tablesync worker.

  14. Reorder pg_sequence columns to avoid alignment issue