Re: logical replication seems broken

Amit Kapila <amit.kapila16@gmail.com>

From: Amit Kapila <amit.kapila16@gmail.com>
To: Erik Rijkers <er@xs4all.nl>
Cc: "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2021-02-13T10:49:43Z
Lists: pgsql-hackers

Attachments

On Fri, Feb 12, 2021 at 10:00 PM <er@xs4all.nl> wrote:
>
> > On 02/12/2021 1:51 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >
> >
> > On Fri, Feb 12, 2021 at 6:04 PM Erik Rijkers <er@xs4all.nl> wrote:
> > >
> > > Hello,
> > >
> > > I am seeing errors in replication in a test program that I've been running for years with very little change (since 2017, really [1]).
>
> Hi,
>
> Here is a test program.  Careful, it deletes stuff.  And it will need some changes:
>

Thanks for sharing the test. I think I have found the problem.
Actually, it was an existing code problem exposed by the commit
ce0fdbfe97. In pgoutput_begin_txn(), we were sometimes sending the
prepare_write ('w') message but then the actual message was not being
sent. This was the case when we didn't found the origin of a txn. This
can happen after that commit because we have now started using origins
for tablesync workers as well and those origins are removed once the
tablesync workers are finished. We might want to change the behavior
related to the origin messages as indicated in the comments but for
now, fixing the existing code.

Can you please test if the attached fixes the problem at your end as well?

-- 
With Regards,
Amit Kapila.

Commits

  1. Remove the unnecessary PrepareWrite in pgoutput.

  2. Allow multiple xacts during table sync in logical replication.

  3. Stop demanding that top xact must be seen before subxact in decoding.