RE: [PATCH] Reuse Workers and Replication Slots during Logical Replication
Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
From: "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>
To: 'Amit Kapila' <amit.kapila16@gmail.com>
Cc: Melih Mutlu <m.melihmutlu@gmail.com>, Peter Smith <smithpb2250@gmail.com>, Melanie Plageman <melanieplageman@gmail.com>, "Wei Wang (Fujitsu)" <wangw.fnst@fujitsu.com>, "Yu Shi (Fujitsu)" <shiy.fnst@fujitsu.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, shveta malik <shveta.malik@gmail.com>
Date: 2023-07-04T05:42:48Z
Lists: pgsql-hackers
Attachments
- v2-0001-Refactor-to-split-Apply-and-Tablesync-Workers.patch (application/octet-stream) patch v2-0001
- v2-0002-Reuse-Tablesync-Workers.patch (application/octet-stream) patch v2-0002
- v2-0003-reuse-connection-when-tablesync-workers-change-th.patch (application/octet-stream) patch v2-0003
- v2-0004-Add-replication-protocol-cmd-to-create-a-snapshot.patch (application/octet-stream) patch v2-0004
- v2-0005-Reuse-Replication-Slot-and-Origin-in-Tablesync.patch (application/octet-stream) patch v2-0005
- v2-0006-Use-slot-name-as-application_name-again.patch (application/octet-stream) patch v2-0006
Dear Amit, > > > > I have analyzed how we handle this. Please see attached the patch (0003) > which > > > > allows reusing connection. > > > > > > > > > > Why did you change the application name during the connection? > > > > It was because the lifetime of tablesync worker is longer than slots's one and > > tablesync worker creates temporary replication slots many times, per the target > > relation. The name of each slots has relid, so I thought that it was not suitable. > > > > Okay, but let's try to give a unique application name to each > tablesync worker for the purpose of pg_stat_activity and synchronous > replication (as mentioned in existing comments as well). One idea is > to generate a name like pg_<sub_id>_sync_<worker_slot> but feel free > to suggest if you have any better ideas. Good point. The slot id is passed as an argument of TablesyncWorkerMain(), so I passed it to LogicalRepSyncTableStart(). PSA new set. > > But in the later patch the tablesync worker tries to reuse the slot during the > > synchronization, so in this case the application_name should be same as > slotname. > > > > Fair enough. I am slightly afraid that if we can't show the benefits > with later patches then we may need to drop them but at this stage I > feel we need to investigate why those are not helping? Agreed. Now I'm planning to do performance testing independently. We can discuss based on that or Melih's one. Best Regards, Hayato Kuroda FUJITSU LIMITED
Commits
-
Refactor to split Apply and Tablesync Workers code.
- 02c1b64fb15c 17.0 landed
-
Fix possible logical replication crash.
- e7e7da2f8d57 16.0 cited