Horton

Re: Logical Replication of sequences

Dilip Kumar <dilipbalaut@gmail.com>

View thread
From: Dilip Kumar <dilipbalaut@gmail.com>
To: vignesh C <vignesh21@gmail.com>
Cc: shveta malik <shveta.malik@gmail.com>, Nisha Moond <nisha.moond412@gmail.com>, Amit Kapila <amit.kapila16@gmail.com>, Peter Smith <smithpb2250@gmail.com>, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, Masahiko Sawada <sawada.mshk@gmail.com>, Shlok Kyal <shlok.kyal.oss@gmail.com>, Peter Eisentraut <peter@eisentraut.org>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Euler Taveira <euler@eulerto.com>, Michael Paquier <michael@paquier.xyz>, "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>, "Jonathan S. Katz" <jkatz@postgresql.org>
Date: 2025-07-18T08:40:58Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Doc: Add documentation for sequence synchronization.

  2. Remove unused assignment in CREATE PUBLICATION grammar.

  3. Add seq_sync_error_count to subscription statistics.

  4. Fix few issues in commit 5509055d69.

  5. Add sequence synchronization for logical replication.

  6. Add worker type argument to logical replication worker functions.

  7. Introduce "REFRESH SEQUENCES" for subscriptions.

  8. Refactor logical worker synchronization code into a separate file.

  9. Standardize use of REFRESH PUBLICATION in code and messages.

  10. Add "ALL SEQUENCES" support to publications.

  11. Expose sequence page LSN via pg_get_sequence_data.

  12. Resume conflict-relevant data retention automatically.

  13. Post-commit review fixes for 228c370868.

  14. Generate GUC tables from .dat file

On Fri, Jul 18, 2025 at 10:44 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
>
> On Thu, Jul 17, 2025 at 4:52 PM vignesh C <vignesh21@gmail.com> wrote:
> >

I was looking at the high level idea of sequence sync worker patch
i.e. 0005, so far I haven't found anything problematic there, but I
haven't completed the review and testing yet.  Here are some comments
I have while reading through the patch.  I will try to do more
thorough review and testing next week.

1.
+ /*
+ * Count running sync workers for this subscription, while we have the
+ * lock.
+ */
+ nsyncworkers = logicalrep_sync_worker_count(MyLogicalRepWorker->subid);
+
+ /* Now safe to release the LWLock */
+ LWLockRelease(LogicalRepWorkerLock);
+
+ /*
+ * If there is a free sync worker slot, start a new sequencesync worker,
+ * and break from the loop.
+ */
+ if (nsyncworkers < max_sync_workers_per_subscription)
+ {
+ TimestampTz now = GetCurrentTimestamp();
+
+ /*
+ * To prevent starting the sequencesync worker at a high frequency
+ * after a failure, we store its last failure time. We start the
+ * sequencesync worker again after waiting at least
+ * wal_retrieve_retry_interval.
+ */
+ if (!MyLogicalRepWorker->sequencesync_failure_time ||
+ TimestampDifferenceExceeds(MyLogicalRepWorker->sequencesync_failure_time,
+    now, wal_retrieve_retry_interval))
+ {
+ MyLogicalRepWorker->sequencesync_failure_time = 0;
+
+ if (!logicalrep_worker_launch(WORKERTYPE_SEQUENCESYNC,
+   MyLogicalRepWorker->dbid,
+   MySubscription->oid,
+   MySubscription->name,
+   MyLogicalRepWorker->userid,
+   InvalidOid,
+   DSM_HANDLE_INVALID))
+ MyLogicalRepWorker->sequencesync_failure_time = now;
+ }

This code seems to duplicate much of the logic found in
ProcessSyncingTablesForApply() within its final else block, with only
minor differences (perhaps 1-2 lines).

To improve code maintainability and avoid redundancy, consider
extracting the common logic into a static function. This function
could then be called from both places.

2.
+/*
+ * Common function to setup the leader apply, tablesync worker and sequencesync
+ * worker.
+ */

Change to "Common function to setup the leader apply, tablesync and
sequencesync worker"

3.
+ /*
+ * To prevent starting the sequencesync worker at a high frequency
+ * after a failure, we store its last failure time. We start the
+ * sequencesync worker again after waiting at least
+ * wal_retrieve_retry_interval.
+ */

We haven't explained what's the rationale behind comparing with the
last failure time for sequence sync worker whereas for table sync
worker we compare with last start time.

-- 
Regards,
Dilip Kumar
Google