Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

Amit Kapila <amit.kapila16@gmail.com>

From: Amit Kapila <amit.kapila16@gmail.com>
To: Melih Mutlu <m.melihmutlu@gmail.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2022-09-07T04:35:59Z
Lists: pgsql-hackers
On Mon, Aug 15, 2022 at 4:56 PM Melih Mutlu <m.melihmutlu@gmail.com> wrote:
>
> Hi Amit,
>
> Amit Kapila <amit.kapila16@gmail.com>, 6 Ağu 2022 Cmt, 16:01 tarihinde şunu yazdı:
>>
>> I think there is some basic flaw in slot reuse design. Currently, we
>> copy the table by starting a repeatable read transaction (BEGIN READ
>> ONLY ISOLATION LEVEL REPEATABLE READ) and create a slot that
>> establishes a snapshot which is first used for copy and then LSN
>> returned by it is used in the catchup phase after the copy is done.
>> The patch won't establish such a snapshot before a table copy as it
>> won't create a slot each time. If this understanding is correct, I
>> think we need to use ExportSnapshot/ImportSnapshot functionality to
>> achieve it or do something else to avoid the problem mentioned.
>
>
> I did not really think about the snapshot created by replication slot while making this change. Thanks for pointing it out.
> I've been thinking about how to fix this issue. There are some points I'm still not sure about.
> If the worker will not create a new replication slot, which snapshot should we actually export and then import?
>

Can we (export/import) use the snapshot we used the first time when a
slot is created for future transactions that copy other tables?
Because if we can do that then I think we can use the same LSN as
returned for the slot when it was created for all other table syncs.

-- 
With Regards,
Amit Kapila.



Commits

  1. Refactor to split Apply and Tablesync Workers code.

  2. Fix possible logical replication crash.