Re: BUG #16643: PG13 - Logical replication - initial startup never finishes and gets stuck in startup loop

Amit Kapila <amit.kapila16@gmail.com>

From: Amit Kapila <amit.kapila16@gmail.com>
To: Peter Smith <smithpb2250@gmail.com>
Cc: Dilip Kumar <dilipbalaut@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Petr Jelinek <petr@2ndquadrant.com>, Tom Lane <tgl@sss.pgh.pa.us>, Henry Hinze <henry.hinze@gmail.com>, PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>, Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Date: 2020-11-17T08:58:53Z
Lists: pgsql-bugs
On Tue, Nov 17, 2020 at 7:44 AM Peter Smith <smithpb2250@gmail.com> wrote:
>
> On Tue, Nov 17, 2020 at 1:07 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> > Yeah, this seems to be possible and this is the reason I mentioned
> > above to dig more into this case. Did you try it via some test case? I
> > think we can generate a test via debugger where after the tablesync
> > worker reaches CATCHUP state stop it via debugger, then we can perform
> > some large transaction on the same table which apply worker will skip
> > and tablesync worker will try to apply changes and should fail.
>
> Hello Amit.
>
> FYI - This email is just to confirm that your above idea for debugging
> the tablesync worker does work.
>

Thanks for trying this out.

> ---
>
> I have so far only been trying above with the non-streaming
> subscription, and TBH stepping through this startup state "dance" of
> the tablesync/apply workers is already causing more questions than
> answers for me. Anyway, I will raise any questions as separate emails
> to this one.
>
> BTW, do you think these tablesync discussions should be moved to
> hackers list?
>

Sure. I think it is better to start a new thread for the streaming
issue we have suspected here and possible ways to fix the same. I
guess you have some other observations as well which you might want to
discuss separately.



-- 
With Regards,
Amit Kapila.



Commits

  1. Fix replication of in-progress transactions in tablesync worker.

  2. Review logical replication tablesync code

  3. Restore replication protocol's duplicate command tags