Horton

Re: logical decoding and replication of sequences, take 2

Amit Kapila <amit.kapila16@gmail.com>

View thread
From: Amit Kapila <amit.kapila16@gmail.com>
To: Dilip Kumar <dilipbalaut@gmail.com>
Cc: Tomas Vondra <tomas.vondra@enterprisedb.com>, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>, Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Masahiko Sawada <sawada.mshk@gmail.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Date: 2023-12-06T10:05:55Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Migrate logical slots to the new node during an upgrade.

  2. Make test_decoding ddl.out shorter

  3. Fix snapshot handling in logicalmsg_decode

  4. doc: Adjust a few more references to "postmaster"

  5. Revert "Logical decoding of sequences"

On Wed, Dec 6, 2023 at 11:12 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
>
> On Sun, Dec 3, 2023 at 11:22 PM Tomas Vondra
> <tomas.vondra@enterprisedb.com> wrote:
> >
>
> > Some time ago I floated the idea of maybe "queuing" the sequence changes
> > and only replay them on the next commit, somehow. But we did ran into
> > problems with which snapshot to use, that I didn't know how to solve.
> > Maybe we should try again. The idea is we'd queue the non-transactional
> > changes somewhere (can't be in the transaction, because we must keep
> > them even if it aborts), and then "inject" them into the next commit.
> > That'd mean we wouldn't do the separate start/abort for each change.
>
> Why can't we use the same concept of
> SnapBuildDistributeNewCatalogSnapshot(), I mean we keep queuing the
> non-transactional changes (have some base snapshot before the first
> change), and whenever there is any catalog change, queue new snapshot
> change also in the queue of the non-transactional sequence change so
> that while sending it to downstream whenever it is necessary we will
> change the historic snapshot?
>

Oh, do you mean maintain different historic snapshots and then switch
based on the change we are processing? I guess the other thing we need
to consider is the order of processing the changes if we maintain
separate queues that need to be processed.

-- 
With Regards,
Amit Kapila.