Horton

Re: Skipping logical replication transactions on subscriber side

Amit Kapila <amit.kapila16@gmail.com>

View thread View on postgr.es
From: Amit Kapila <amit.kapila16@gmail.com>
To: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: Peter Eisentraut <peter.eisentraut@enterprisedb.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2021-07-01T03:56:45Z
Lists: pgsql-hackers 
On Wed, Jun 30, 2021 at 4:35 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Mon, Jun 28, 2021 at 10:12 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> >
> >
> > 0003 patch adds pg_stat_logical_replication_error statistics view
> > discussed on another thread[1]. The apply worker sends the error
> > information to the stats collector if an error happens during applying
> > changes. We can check those errors as follow:
> >
> > postgres(1:25250)=# select * from pg_stat_logical_replication_error;
> >  subname  | relid | action | xid |         last_failure
> > ----------+-------+--------+-----+-------------------------------
> >  test_sub | 16384 | INSERT | 736 | 2021-06-27 12:12:45.142675+09
> > (1 row)
> >
> > I added only columns required for the skipping transaction feature to
> > the view for now.
> >
>
> Isn't it better to add an error message if possible?
>

Don't we want to clear stats at drop subscription as well? We do drop
database stats in dropdb via pgstat_drop_database, so I think we need
to clear subscription stats at the time of drop subscription.

In the 0003 patch, if I am reading it correctly then the patch is not
doing anything for tablesync worker. It is not clear to me at this
stage what exactly we want to do about it? Do we want to just ignore
errors from tablesync worker and let the system behave as it is
without this feature? If we want to do anything then I think the way
to skip the initial table sync would be to behave like the user has
given 'copy_data' option as false.

-- 
With Regards,
Amit Kapila.

Commits

  1. Test ALIGNOF_DOUBLE==4 compatibility under ALIGNOF_DOUBLE==8.

  2. Reorder subskiplsn in pg_subscription to avoid alignment issues.

  3. Add ALTER SUBSCRIPTION ... SKIP.

  4. Optionally disable subscriptions on error.

  5. Update docs of logical replication for commit 8d74fc96db.

  6. Respect permissions within logical replication.

  7. Fix regression test failure caused by commit 8d74fc96db.

  8. Add a view to show the stats of subscription workers.

  9. Add logical change details to logical replication worker errcontext.

  10. Rename LOGICAL_REP_MSG_STREAM_END to LOGICAL_REP_MSG_STREAM_STOP.

  11. Fix typo in protocol.sgml.

  12. Remove unused argument in apply_handle_commit_internal().

  13. Fix replication of in-progress transactions in tablesync worker.

  14. Reorder pg_sequence columns to avoid alignment issue