Horton

Re: Skipping logical replication transactions on subscriber side

Masahiko Sawada <sawada.mshk@gmail.com>

View thread View on postgr.es
From: Masahiko Sawada <sawada.mshk@gmail.com>
To: "tanghy.fnst@fujitsu.com" <tanghy.fnst@fujitsu.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, "osumi.takamichi@fujitsu.com" <osumi.takamichi@fujitsu.com>, "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>, Alexey Lesovsky <lesovsky@gmail.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2021-08-19T01:53:01Z
Lists: pgsql-hackers 
On Tue, Aug 17, 2021 at 5:21 PM tanghy.fnst@fujitsu.com
<tanghy.fnst@fujitsu.com> wrote:
>
> On Thursday, August 12, 2021 1:53 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> >
> > I've attached the updated patches. FYI I've included the patch
> > (v8-0005) that fixes the assertion failure during shared fileset
> > cleanup to make cfbot tests happy.
>
>
> Hi
>
> Thanks for your patch. I met a problem when using it. The log is not what I expected in some cases, but in streaming mode, they work well.
>
> For example:
> ------publisher------
> create table test (a int primary key, b varchar);
> create publication pub for table test;
>
> ------subscriber------
> create table test (a int primary key, b varchar);
> insert into test values (10000);
> create subscription sub connection 'dbname=postgres port=5432' publication pub with(streaming=on);
>
> ------publisher------
> insert into test values (10000);
>
> Subscriber log:
> 2021-08-17 14:24:43.415 CST [3630341] ERROR:  duplicate key value violates unique constraint "test_pkey"
> 2021-08-17 14:24:43.415 CST [3630341] DETAIL:  Key (a)=(10000) already exists.
>
> It didn't give more context info generated by apply_error_callback function.

Thank you for reporting the issue! This issue must be fixed in the
latest (v9) patches I've just submitted[1].

Regards,

[1] https://www.postgresql.org/message-id/CAD21AoCH4Jwn_NkJhvS6W5bZJKSaAYnC9inXqMJc6dLLvhvTQg%40mail.gmail.com

-- 
Masahiko Sawada
EDB:  https://www.enterprisedb.com/

Commits

  1. Test ALIGNOF_DOUBLE==4 compatibility under ALIGNOF_DOUBLE==8.

  2. Reorder subskiplsn in pg_subscription to avoid alignment issues.

  3. Add ALTER SUBSCRIPTION ... SKIP.

  4. Optionally disable subscriptions on error.

  5. Update docs of logical replication for commit 8d74fc96db.

  6. Respect permissions within logical replication.

  7. Fix regression test failure caused by commit 8d74fc96db.

  8. Add a view to show the stats of subscription workers.

  9. Add logical change details to logical replication worker errcontext.

  10. Rename LOGICAL_REP_MSG_STREAM_END to LOGICAL_REP_MSG_STREAM_STOP.

  11. Fix typo in protocol.sgml.

  12. Remove unused argument in apply_handle_commit_internal().

  13. Fix replication of in-progress transactions in tablesync worker.

  14. Reorder pg_sequence columns to avoid alignment issue