RE: [PATCH] Reuse Workers and Replication Slots during Logical Replication
Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
From: "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>
To: 'Melih Mutlu' <m.melihmutlu@gmail.com>
Cc: Peter Smith <smithpb2250@gmail.com>, Melanie Plageman <melanieplageman@gmail.com>, "Wei Wang (Fujitsu)" <wangw.fnst@fujitsu.com>, "Yu Shi (Fujitsu)" <shiy.fnst@fujitsu.com>, Amit Kapila <amit.kapila16@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, shveta malik <shveta.malik@gmail.com>
Date: 2023-06-27T07:42:49Z
Lists: pgsql-hackers
Attachments
- 0004-Add-replication-protocol-cmd-to-create-a-snapshot.patch (application/octet-stream) patch 0004
- 0005-Reuse-Replication-Slot-and-Origin-in-Tablesync.patch (application/octet-stream) patch 0005
- 0001-Refactor-to-split-Apply-and-Tablesync-Workers.patch (application/octet-stream) patch 0001
- 0002-Reuse-Tablesync-Workers.patch (application/octet-stream) patch 0002
- 0003-reuse-connection-when-tablesync-workers-change-the-t.patch (application/octet-stream) patch 0003
Dear Melih,
Thank you for updating the patch! I have not reviewed yet, but I wanted
to reply your comments.
> This actually makes sense. I quickly try to do that without adding any
> new replication message. As you would expect, it did not work.
> I don't really know what's needed to make a connection to last for
> more than one iteration. Need to look into this. Happy to hear any
> suggestions and thoughts.
I have analyzed how we handle this. Please see attached the patch (0003) which
allows reusing connection. The patchset passed tests on my CI.
To make cfbot happy I reassigned the patch number.
In this patch, the tablesync worker does not call clean_sync_worker() at the end
of iterations, and the establishment of the connection is done only once.
The creation of memory context is also suppressed.
Regarding the walsender, streamingDone{Sending|Receiving} is now initialized
before executing StartLogicalReplication(). These flags have been used to decide
when the process exits copy mode. The default value is false, and they are set
to true when the copy mode is finished.
I think there was no use-case that the same walsender executes START_REPLICATION
replication twice so there were no codes for restoring flags. Please tell me if any other
reasons.
Best Regards,
Hayato Kuroda
FUJITSU LIMITED
Commits
-
Refactor to split Apply and Tablesync Workers code.
- 02c1b64fb15c 17.0 landed
-
Fix possible logical replication crash.
- e7e7da2f8d57 16.0 cited