Horton

Re: Skipping logical replication transactions on subscriber side

Noah Misch <noah@leadboat.com>

View thread View on postgr.es
From: Noah Misch <noah@leadboat.com>
To: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Euler Taveira <euler@eulerto.com>, "osumi.takamichi@fujitsu.com" <osumi.takamichi@fujitsu.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, "David G. Johnston" <david.g.johnston@gmail.com>, "tanghy.fnst@fujitsu.com" <tanghy.fnst@fujitsu.com>, vignesh C <vignesh21@gmail.com>, Greg Nancarrow <gregn4422@gmail.com>, "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>, Alexey Lesovsky <lesovsky@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2022-04-02T00:11:53Z
Lists: pgsql-hackers

Attachments

On Fri, Apr 01, 2022 at 09:25:52PM +0900, Masahiko Sawada wrote:
> > On Fri, Apr 1, 2022 at 4:44 PM Noah Misch <noah@leadboat.com> wrote:
> > > src/test/subscription/t/029_on_error.pl has been failing reliably on the five
> > > AIX buildfarm members:
> > >
> > > # poll_query_until timed out executing this query:
> > > # SELECT subskiplsn = '0/0' FROM pg_subscription WHERE subname = 'sub'
> > > # expecting this output:
> > > # t
> > > # last actual query output:
> > > # f
> > > # with stderr:
> > > timed out waiting for match: (?^:LOG:  done skipping logical replication transaction finished at 0/1D30788) at t/029_on_error.pl line 50.
> > >
> > > I've posted five sets of logs (2.7 MiB compressed) here:
> > > https://drive.google.com/file/d/16NkyNIV07o0o8WM7GwcaAYFQDPTkULkR/view?usp=sharing

> Given that "SELECT subskiplsn = '0/0'
> FROM pg_subscription WHERE subname = 'sub’” didn't return true, some
> value was set to subskiplsn even after the unique key error.
> 
> So I'm guessing that the apply worker could not get the updated value
> of the subskiplsn or its MySubscription->skiplsn could not match with
> the transaction's finish LSN. Also, given that the test is failing on
> all AIX buildfarm members, there might be something specific to AIX.
> 
> Noah, to investigate this issue further, is it possible for you to
> apply the attached patch and run the 029_on_error.pl test? The patch
> adds some logs to get additional information.

Logs attached.  I ran this outside the buildfarm script environment.  Most
notably, I didn't override PG_TEST_TIMEOUT_DEFAULT like my buildfarm
configuration does, so the total log size is smaller.

Commits

  1. Test ALIGNOF_DOUBLE==4 compatibility under ALIGNOF_DOUBLE==8.

  2. Reorder subskiplsn in pg_subscription to avoid alignment issues.

  3. Add ALTER SUBSCRIPTION ... SKIP.

  4. Optionally disable subscriptions on error.

  5. Update docs of logical replication for commit 8d74fc96db.

  6. Respect permissions within logical replication.

  7. Fix regression test failure caused by commit 8d74fc96db.

  8. Add a view to show the stats of subscription workers.

  9. Add logical change details to logical replication worker errcontext.

  10. Rename LOGICAL_REP_MSG_STREAM_END to LOGICAL_REP_MSG_STREAM_STOP.

  11. Fix typo in protocol.sgml.

  12. Remove unused argument in apply_handle_commit_internal().

  13. Fix replication of in-progress transactions in tablesync worker.

  14. Reorder pg_sequence columns to avoid alignment issue