Horton

RE: Skipping logical replication transactions on subscriber side

Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>

View thread View on postgr.es
From: "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>
To: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: Alexey Lesovsky <lesovsky@gmail.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>, Amit Kapila <amit.kapila16@gmail.com>
Date: 2021-07-19T11:38:54Z
Lists: pgsql-hackers 
On July 19, 2021 2:40 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> I've attached the updated version patch that incorporated all comments
> I got so far except for the clearing error details part I mentioned
> above. After getting a consensus on those parts, I'll incorporate the
> idea into the patches.

Hi Sawada-san,

I am interested in this feature.
After having a look at the patch, I have a few questions about it.
(Sorry in advance if I missed something)

1) In 0002 patch, it introduces a new view called pg_stat_subscription_errors.
   Since it won't be cleaned automatically after we resolve the conflict, do we
   need a reset function to clean the statistics in it ? Maybe something
   similar to pg_stat_reset_replication_slot which clean the
   pg_stat_replication_slots.

2) For 0003 patch, When I am faced with a conflict, I set skip_xid = xxx, and
   then I resolve the conflict. If I reset skip_xid after resolving the
   conflict, will the change(which cause the conflict before) be applied again ?

3) For 0003 patch, if user set skip_xid to a wrong xid which have not been
   assigned, and then will the change be skipped when the xid is assigned in
   the future even if it doesn't cause any conflicts ?

Besides, It might be better to add some description of patch in each patch's
commit message which will make it easier for new reviewers to follow.


Best regards,
Houzj

Commits

  1. Test ALIGNOF_DOUBLE==4 compatibility under ALIGNOF_DOUBLE==8.

  2. Reorder subskiplsn in pg_subscription to avoid alignment issues.

  3. Add ALTER SUBSCRIPTION ... SKIP.

  4. Optionally disable subscriptions on error.

  5. Update docs of logical replication for commit 8d74fc96db.

  6. Respect permissions within logical replication.

  7. Fix regression test failure caused by commit 8d74fc96db.

  8. Add a view to show the stats of subscription workers.

  9. Add logical change details to logical replication worker errcontext.

  10. Rename LOGICAL_REP_MSG_STREAM_END to LOGICAL_REP_MSG_STREAM_STOP.

  11. Fix typo in protocol.sgml.

  12. Remove unused argument in apply_handle_commit_internal().

  13. Fix replication of in-progress transactions in tablesync worker.

  14. Reorder pg_sequence columns to avoid alignment issue