Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication
Melih Mutlu <m.melihmutlu@gmail.com>
From: Melih Mutlu <m.melihmutlu@gmail.com>
To: pgsql-hackers <pgsql-hackers@postgresql.org>
Cc: Melanie Plageman <melanieplageman@gmail.com>, "wangw.fnst@fujitsu.com" <wangw.fnst@fujitsu.com>,
"shiy.fnst@fujitsu.com" <shiy.fnst@fujitsu.com>, Amit Kapila <amit.kapila16@gmail.com>, shveta malik <shveta.malik@gmail.com>, Peter Smith <smithpb2250@gmail.com>, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>
Date: 2023-06-23T13:32:47Z
Lists: pgsql-hackers
Attachments
- v14-0004-Reuse-Replication-Slot-and-Origin-in-Tablesync.patch (application/octet-stream) patch v14-0004
- 0001-Refactor-to-split-Apply-and-Tablesync-Workers.patch (application/octet-stream) patch 0001
- v3-0002-Reuse-Tablesync-Workers.patch (application/octet-stream) patch v3-0002
- v11-0003-Add-replication-protocol-cmd-to-create-a-snapshot.patch (application/octet-stream) patch v11-0003
Hi hackers, You can find the updated patchset attached. I worked to address the reviews and made some additional changes. Let me first explain the new patchset. 0001: Refactors the logical replication code, mostly worker.c and tablesync.c. Although this patch makes it easier to reuse workers, I believe that it's useful even by itself without other patches. It does not improve performance or anything but aims to increase readability and such. 0002: This is only to reuse worker processes, everything else stays the same (replication slots/origins etc.). 0003: Adds a new command for streaming replication protocol to create a snapshot by an existing replication slot. 0004: Reuses replication slots/origins together with workers. Even only 0001 and 0002 are enough to improve table sync performance at the rates previously shared on this thread. This also means that currently 0004 (reusing replication slots/origins) does not improve as much as I would expect, even though it does not harm either. I just wanted to share what I did so far, while I'm continuing to investigate it more to see what I'm missing in patch 0004. Thanks, -- Melih Mutlu Microsoft
Commits
-
Refactor to split Apply and Tablesync Workers code.
- 02c1b64fb15c 17.0 landed
-
Fix possible logical replication crash.
- e7e7da2f8d57 16.0 cited