RE: [PATCH] Reuse Workers and Replication Slots during Logical Replication
Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
From: "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>
To: 'Amit Kapila' <amit.kapila16@gmail.com>, Melih Mutlu <m.melihmutlu@gmail.com>
Cc: Peter Smith <smithpb2250@gmail.com>, Melanie Plageman <melanieplageman@gmail.com>, "Wei Wang (Fujitsu)" <wangw.fnst@fujitsu.com>, "Yu Shi (Fujitsu)" <shiy.fnst@fujitsu.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, shveta malik <shveta.malik@gmail.com>
Date: 2023-06-28T06:31:54Z
Lists: pgsql-hackers
Attachments
- 0005-Reuse-Replication-Slot-and-Origin-in-Tablesync.patch (application/octet-stream) patch 0005
- 0006-Use-slot-name-as-application_name-again.patch (application/octet-stream) patch 0006
- 0001-Refactor-to-split-Apply-and-Tablesync-Workers.patch (application/octet-stream) patch 0001
- 0002-Reuse-Tablesync-Workers.patch (application/octet-stream) patch 0002
- 0003-reuse-connection-when-tablesync-workers-change-the-t.patch (application/octet-stream) patch 0003
- 0004-Add-replication-protocol-cmd-to-create-a-snapshot.patch (application/octet-stream) patch 0004
Dear Amit, > > > This actually makes sense. I quickly try to do that without adding any > > > new replication message. As you would expect, it did not work. > > > I don't really know what's needed to make a connection to last for > > > more than one iteration. Need to look into this. Happy to hear any > > > suggestions and thoughts. > > > > It is not clear to me what exactly you tried here which didn't work. > Can you please explain a bit more? Just to confirm, this is not my part. Melih can answer this... > > I have analyzed how we handle this. Please see attached the patch (0003) which > > allows reusing connection. > > > > Why did you change the application name during the connection? It was because the lifetime of tablesync worker is longer than slots's one and tablesync worker creates temporary replication slots many times, per the target relation. The name of each slots has relid, so I thought that it was not suitable. But in the later patch the tablesync worker tries to reuse the slot during the synchronization, so in this case the application_name should be same as slotname. I added comment in 0003, and new file 0006 file to use slot name as application_name again. Note again that the separation was just for specifying changes, Melih can include them to one part of files if needed. Best Regards, Hayato Kuroda FUJITSU LIMITED
Commits
-
Refactor to split Apply and Tablesync Workers code.
- 02c1b64fb15c 17.0 landed
-
Fix possible logical replication crash.
- e7e7da2f8d57 16.0 cited