RE: [PATCH] Reuse Workers and Replication Slots during Logical Replication

Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>

From: "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>
To: 'Melih Mutlu' <m.melihmutlu@gmail.com>
Cc: Peter Smith <smithpb2250@gmail.com>, Melanie Plageman <melanieplageman@gmail.com>, "Wei Wang (Fujitsu)" <wangw.fnst@fujitsu.com>, "Yu Shi (Fujitsu)" <shiy.fnst@fujitsu.com>, Amit Kapila <amit.kapila16@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, shveta malik <shveta.malik@gmail.com>
Date: 2023-07-13T04:09:12Z
Lists: pgsql-hackers
Dear Melih,

> > > Thanks for the 0003 patch. But it did not work for me. Can you create
> > > a subscription successfully with patch 0003 applied?
> > > I get the following error: " ERROR:  table copy could not start
> > > transaction on publisher: another command is already in progress".
> >
> > You got the ERROR when all the patches (0001-0005) were applied, right?
> > I have focused on 0001 and 0002 only, so I missed something.
> > If it was not correct, please attach the logfile and test script what you did.
> 
> Yes, I did get an error with all patches applied. But with only 0001
> and 0002, your version seems like working and mine does not.

Hmm, really? IIUC I did not modify 0001 and 0002 patches, I just re-assigned the
version number. I compared between yours and mine, but no meaningful differences
were found. E.g., following command compared v4-0002 and v16-0002:

```
diff --git a/../reuse_workers/v4-0002-Reuse-Tablesync-Workers.patch b/../reuse_workers/hayato/v16-0002-Reuse-Tablesync-Workers.patch
index 5350216e98..7785a573e4 100644
--- a/../reuse_workers/v4-0002-Reuse-Tablesync-Workers.patch
+++ b/../reuse_workers/hayato/v16-0002-Reuse-Tablesync-Workers.patch
@@ -1,7 +1,7 @@
-From d482022b40e0a5ce1b74fd0e320cb5b45da2f671 Mon Sep 17 00:00:00 2001
+From db3e8e2d7aadea79126c5816bce8b06dc82f33c2 Mon Sep 17 00:00:00 2001
 From: Melih Mutlu <m.melihmutlu@gmail.com>
 Date: Tue, 4 Jul 2023 22:04:46 +0300
-Subject: [PATCH 2/5] Reuse Tablesync Workers
+Subject: [PATCH v16 2/5] Reuse Tablesync Workers
 
 This commit allows reusing tablesync workers for syncing more than one
 table sequentially during their lifetime, instead of exiting after
@@ -324,5 +324,5 @@ index 7aba034774..1e9f8e6e72 100644
  static inline bool
  am_tablesync_worker(void)
 -- 
-2.25.1
+2.27.0
```

For confirmation, please attach the logfile and test script what you did
if you could reproduce?

> What do you think about combining 0002 and 0003? Or should those stay
> separate?

I have no strong opinion, but it may be useful to keep them pluggable.

Best Regards,
Hayato Kuroda
FUJITSU LIMITED

Commits

  1. Refactor to split Apply and Tablesync Workers code.

  2. Fix possible logical replication crash.