Horton

Re: Skipping logical replication transactions on subscriber side

Dilip Kumar <dilipbalaut@gmail.com>

View thread View on postgr.es
From: Dilip Kumar <dilipbalaut@gmail.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Masahiko Sawada <sawada.mshk@gmail.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>, vignesh C <vignesh21@gmail.com>, Greg Nancarrow <gregn4422@gmail.com>, "tanghy.fnst@fujitsu.com" <tanghy.fnst@fujitsu.com>, "osumi.takamichi@fujitsu.com" <osumi.takamichi@fujitsu.com>, Alexey Lesovsky <lesovsky@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2021-12-14T10:11:27Z
Lists: pgsql-hackers 
On Tue, Dec 14, 2021 at 2:36 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> > >
> > > I agree with this theory. Can we reflect this in comments so that in
> > > the future we know why we didn't pursue this direction?
> >
> > I might be missing something here, but for streaming, transaction
> > users can decide whether they wants to skip or not only once we start
> > applying no?  I mean only once we start applying the changes we can
> > get some errors and by that time we must be having all the changes for
> > the transaction.
> >
>
> That is right and as per my understanding, the patch is trying to
> accomplish the same.
>
> >  So I do not understand the point we are trying to
> > discuss here?
> >
>
> The point is that whether we can skip the changes while streaming
> itself like when we get the changes and write to a stream file. Now,
> it is possible that streams from multiple transactions can be
> interleaved and users can change the skip_xid in between. It is not
> that we can't handle this but that would require a more complex design
> and it doesn't seem worth it because we can anyway skip the changes
> while applying as you mentioned in the previous paragraph.

Actually, I was trying to understand the use case for skipping while
streaming.  Actually, during streaming we are not doing any database
operation that means this will not generate any error.  So IIUC, there
is no use case for skipping while streaming itself? Is there any use
case which I am not aware of?


-- 
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com

Commits

  1. Test ALIGNOF_DOUBLE==4 compatibility under ALIGNOF_DOUBLE==8.

  2. Reorder subskiplsn in pg_subscription to avoid alignment issues.

  3. Add ALTER SUBSCRIPTION ... SKIP.

  4. Optionally disable subscriptions on error.

  5. Update docs of logical replication for commit 8d74fc96db.

  6. Respect permissions within logical replication.

  7. Fix regression test failure caused by commit 8d74fc96db.

  8. Add a view to show the stats of subscription workers.

  9. Add logical change details to logical replication worker errcontext.

  10. Rename LOGICAL_REP_MSG_STREAM_END to LOGICAL_REP_MSG_STREAM_STOP.

  11. Fix typo in protocol.sgml.

  12. Remove unused argument in apply_handle_commit_internal().

  13. Fix replication of in-progress transactions in tablesync worker.

  14. Reorder pg_sequence columns to avoid alignment issue