Re: Logical Replication of sequences

shveta malik <shveta.malik@gmail.com>

From: shveta malik <shveta.malik@gmail.com>
To: vignesh C <vignesh21@gmail.com>
Cc: Peter Smith <smithpb2250@gmail.com>, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, Masahiko Sawada <sawada.mshk@gmail.com>, Shlok Kyal <shlok.kyal.oss@gmail.com>, Amit Kapila <amit.kapila16@gmail.com>, Peter Eisentraut <peter@eisentraut.org>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Euler Taveira <euler@eulerto.com>, Michael Paquier <michael@paquier.xyz>, "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>, "Jonathan S. Katz" <jkatz@postgresql.org>, shveta malik <shveta.malik@gmail.com>
Date: 2025-05-09T08:58:15Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →
  1. Doc: Add documentation for sequence synchronization.

  2. Remove unused assignment in CREATE PUBLICATION grammar.

  3. Add seq_sync_error_count to subscription statistics.

  4. Fix few issues in commit 5509055d69.

  5. Add sequence synchronization for logical replication.

  6. Add worker type argument to logical replication worker functions.

  7. Introduce "REFRESH SEQUENCES" for subscriptions.

  8. Refactor logical worker synchronization code into a separate file.

  9. Standardize use of REFRESH PUBLICATION in code and messages.

  10. Add "ALL SEQUENCES" support to publications.

  11. Expose sequence page LSN via pg_get_sequence_data.

  12. Resume conflict-relevant data retention automatically.

  13. Post-commit review fixes for 228c370868.

  14. Generate GUC tables from .dat file

On Sat, May 3, 2025 at 7:27 PM vignesh C <vignesh21@gmail.com> wrote:
>
> >
> > Thanks for the comments, the updated patch has the changes for the same.
>

Thanks for the patches. Please find few comments:


1)
patch004 commit msg:
- Drop published sequences are removed from pg_subscription_rel.

Drop -->Dropped


2)
copy_sequences:

LOG:  Executing query :SELECT s.schname, s.seqname, ps.*, seq.seqtypid,
       seq.seqstart, seq.seqincrement, seq.seqmin,
       seq.seqmax, seq.seqcycle
FROM ( VALUES ('public', 'myseq1'), ('public', 'myseq3') ) AS s
(schname, seqname)
JOIN LATERAL pg_sequence_state(s.schname, s.seqname) ps ON true
....

Do we really need to log this query? If so, shall it be DEBUG1/DEBUG2?


3)
In log, we get:

------------------
LOG:  logical replication synchronized 9 of 9 sequences for subscription "sub1"
WARNING:  parameters differ for the remote and local sequences
("public.myseq1", "public.myseq3") for subscription "sub1"

LOG:  logical replication synchronized 2 of 2 sequences for subscription "sub1"
WARNING:  parameters differ for the remote and local sequences
("public.myseq1", "public.myseq3") for subscription "sub1"
------------------

This is confusing. I have 9 sequences, out of which 2 are mismatched.
So on REFRESH I get the first message as 'synchronized 9 of 9' and
later when it attempts to resynchronize pending ones automatically, it
keeps on displaying  'synchronized 2 of 2'.

Can we mention something like below:
-----------------
Unsynchronized sequences: 9, attempted in this batch: 9, succedded: 7,
mismatched/failed:2

So that if it is more than 100, say 120, it will say:
Unsynchronized sequences: 120, attempted in this batch: 100,
succedded: 98, mismatched/failed:2
And in next batch:
Unsynchronized sequences: 120, attempted in this batch: 20, succedded:
20, mismatched:0

And while attempting to synchronize failed ones, it will say:
Unsynchronized sequences: 2, attempted in this batch: 2, succedded: 0,
mismatched:2
-----------------

Please feel free to change the words. The intent is to get a clear
picture on what is happening.


4)
Why in patch001, we have 'pg_sequence_state' with one argument while
in 4ht patch it is changed to 2 args? Is it intentional to have it the
current way in patch001?


5)
Section1:
  <para>
   A new <firstterm>sequence synchronization worker</firstterm> will be started
   after executing any of the above subscriber commands, and will exit once the
   sequences are synchronized.
  </para>

Section2:
  <sect2 id="sequence-definition-mismatches">
   <title>Sequence Definition Mismatches</title>
   <warning>
    <para>
     During sequence synchronization, the sequence definitions of the publisher
     and the subscriber are compared. A WARNING is logged if any differences
     are detected.
    </para>
   </warning>


None of the section mentions that the synchronization worker will keep
on attempting to synchronize the failed/mismtached sequences until the
differences are resolved (provided disable_on_error is not enabled).
I think we can mention such a thing briefly in
'sequence-definition-mismatches' section.


thanks
Shveta