Horton

Re: logical decoding and replication of sequences, take 2

Dilip Kumar <dilipbalaut@gmail.com>

View thread
From: Dilip Kumar <dilipbalaut@gmail.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Tomas Vondra <tomas.vondra@enterprisedb.com>, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>, Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Masahiko Sawada <sawada.mshk@gmail.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Date: 2023-12-06T11:05:27Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Migrate logical slots to the new node during an upgrade.

  2. Make test_decoding ddl.out shorter

  3. Fix snapshot handling in logicalmsg_decode

  4. doc: Adjust a few more references to "postmaster"

  5. Revert "Logical decoding of sequences"

On Wed, Dec 6, 2023 at 3:36 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> > Why can't we use the same concept of
> > SnapBuildDistributeNewCatalogSnapshot(), I mean we keep queuing the
> > non-transactional changes (have some base snapshot before the first
> > change), and whenever there is any catalog change, queue new snapshot
> > change also in the queue of the non-transactional sequence change so
> > that while sending it to downstream whenever it is necessary we will
> > change the historic snapshot?
> >
>
> Oh, do you mean maintain different historic snapshots and then switch
> based on the change we are processing? I guess the other thing we need
> to consider is the order of processing the changes if we maintain
> separate queues that need to be processed.

I mean we will not specifically maintain the historic changes, but if
there is any catalog change where we are pushing the snapshot to all
the transaction's change queue, at the same time we will push this
snapshot in the non-transactional sequence queue as well.  I am not
sure what is the problem with the ordering? because we will be
queueing all non-transactional sequence changes in a separate queue in
the order they arrive and as soon as we process the next commit we
will process all the non-transactional changes at that time.  Do you
see issue with that?

-- 
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com