Re: Excessive number of replication slots for 12->14 logical replication

Bowen Shi <zxwsbg12138@gmail.com>

From: Bowen Shi <zxwsbg12138@gmail.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Masahiko Sawada <sawada.mshk@gmail.com>, "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>, Ajin Cherian <itsajin@gmail.com>, Peter Smith <smithpb2250@gmail.com>, Hubert Lubaczewski <depesz@depesz.com>, PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>
Date: 2024-01-18T07:29:52Z
Lists: pgsql-bugs
Dears,

I encountered a similar problem when I used logical replication to
replicate databases from pg 16 to pg 16.

I started 3 subscription in parallel, and  subscriber's postgresql.conf is
following:
max_replication_slots = 10
max_sync_workers_per_subscription = 2

However, after 3 minutes, I found three COPY errors in subscriber:
"error while shutting down streaming COPY: ERROR:  could not find record
while sending logically-decoded data: missing contrecord at xxxx/xxxxxxxxx""
Then,  the subscriber began to print a large number of errors: "could not
find free replication state slot for replication origin with ID 11,
Increase max_replication_slots and try again."

And the publisher was full of pg_xxx_sync_xxxxxxx slots, printing lots of
"all replication slots are in use, Free one or increase
max_replication_slots."

This question is very similar to
https://www.postgresql.org/message-id/flat/20220714115155.GA5439%40depesz.com
. When the table sync worker encounters an error and exits while copying a
table, the replication origin will not be deleted. And new table sync
workers would create sync slot in the publisher and then exit without
dropping them.

Bowen Shi

Amit Kapila <amit.kapila16@gmail.com> 于2024年1月18日周四 15:02写道:

> On Mon, Sep 12, 2022 at 12:25 PM Masahiko Sawada <sawada.mshk@gmail.com>
> wrote:
> >
> > On Mon, Sep 12, 2022 at 3:15 PM Amit Kapila <amit.kapila16@gmail.com>
> wrote:
> > >
> > > Changed this and a few other comments in the patch. Kindly let me know
> > > what you think of the attached.
> >
> > Thank you for updating the patch. It looks good to me.
> >
>
> Thanks for the verification. Pushed!
>
> --
> With Regards,
> Amit Kapila.
>
>
>
>
>

Commits

  1. Make the tablesync worker's replication origin drop logic robust.

  2. Drop replication origin slots before tablesync worker exits.

  3. Allow multiple xacts during table sync in logical replication.