Horton

Re: logical decoding and replication of sequences, take 2

Masahiko Sawada <sawada.mshk@gmail.com>

View thread
From: Masahiko Sawada <sawada.mshk@gmail.com>
To: Tomas Vondra <tomas.vondra@enterprisedb.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, John Naylor <john.naylor@enterprisedb.com>, vignesh C <vignesh21@gmail.com>, Andres Freund <andres@anarazel.de>, Robert Haas <robertmhaas@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: 2023-03-28T16:34:39Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Migrate logical slots to the new node during an upgrade.

  2. Make test_decoding ddl.out shorter

  3. Fix snapshot handling in logicalmsg_decode

  4. doc: Adjust a few more references to "postmaster"

  5. Revert "Logical decoding of sequences"

On Mon, Mar 27, 2023 at 11:46 PM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:
>
>
>
> On 3/27/23 03:32, Masahiko Sawada wrote:
> > Hi,
> >
> > On Fri, Mar 24, 2023 at 7:26 AM Tomas Vondra
> > <tomas.vondra@enterprisedb.com> wrote:
> >>
> >> I merged the earlier "fixup" patches into the relevant parts, and left
> >> two patches with new tweaks (deducing the corrent "WAL" state from the
> >> current state read by copy_sequence), and the interlock discussed here.
> >>
> >
> > Apart from that, how does the publication having sequences work with
> > subscribers who are not able to handle sequence changes, e.g. in a
> > case where PostgreSQL version of publication is newer than the
> > subscriber? As far as I tested the latest patches, the subscriber
> > (v15)  errors out with the error 'invalid logical replication message
> > type "Q"' when receiving a sequence change. I'm not sure it's sensible
> > behavior. I think we should instead either (1) deny starting the
> > replication if the subscriber isn't able to handle sequence changes
> > and the publication includes that, or (2) not send sequence changes to
> > such subscribers.
> >
>
> I agree the "invalid message" error is not great, but it's not clear to
> me how to do either (1). The trouble is we don't really know if the
> publication contains (or will contain) sequences. I mean, what would
> happen if the replication starts and then someone adds a sequence?
>
> For (2), I think that's not something we should do - silently discarding
> some messages seems error-prone. If the publication includes sequences,
> presumably the user wanted to replicate those. If they want to replicate
> to an older subscriber, create a publication without sequences.
>
> Perhaps the right solution would be to check if the subscriber supports
> replication of sequences in the output plugin, while attempting to write
> the "Q" message. And error-out if the subscriber does not support it.

It might be related to this topic; do we need to bump the protocol
version? The commit 64824323e57d introduced new streaming callbacks
and bumped the protocol version. I think the same seems to be true for
this change as it adds sequence_cb callback.

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com