Horton

RE: logical decoding and replication of sequences, take 2

Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>

View thread
From: "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>
To: 'Amit Kapila' <amit.kapila16@gmail.com>, Tomas Vondra <tomas.vondra@enterprisedb.com>
Cc: "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>, Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Masahiko Sawada <sawada.mshk@gmail.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, Dilip Kumar <dilipbalaut@gmail.com>
Date: 2023-11-27T12:08:24Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Migrate logical slots to the new node during an upgrade.

  2. Make test_decoding ddl.out shorter

  3. Fix snapshot handling in logicalmsg_decode

  4. doc: Adjust a few more references to "postmaster"

  5. Revert "Logical decoding of sequences"

Attachments

Dear Amit, Tomas,

> > >
> > > I am wondering that instead of building the infrastructure to know
> > > whether a particular change is transactional on the decoding side,
> > > can't we have some flag in the WAL record to note whether the change
> > > is transactional or not? I have discussed this point with my colleague
> > > Kuroda-San and we thought that it may be worth exploring whether we
> > > can use rd_createSubid/rd_newRelfilelocatorSubid in RelationData to
> > > determine if the sequence is created/changed in the current
> > > subtransaction and then record that in WAL record. By this, we need to
> > > have additional information in the WAL record like XLOG_SEQ_LOG but we
> > > can probably do it only with wal_level as logical.
> > >
> >
> > I may not understand the proposal exactly, but it's not enough to know
> > if it was created in the same subxact. It might have been created in
> > some earlier subxact in the same top-level xact.
> >
> 
> We should be able to detect even some earlier subxact or top-level
> xact based on rd_createSubid/rd_newRelfilelocatorSubid.

Here is a small PoC patchset to help your understanding. Please see attached
files.

0001, 0002 were not changed, and 0004 was reassigned to 0003.
(For now, I focused only on test_decoding, because it is only for evaluation purpose.)

0004 is what we really wanted to say. is_transactional is added in WAL record, and it stores
whether the operations is transactional. In order to distinguish the status, rd_createSubid and
rd_newRelfilelocatorSubid are used. According to the comment, they would be a valid value
only when the relation was changed within the transaction
Also, sequences_hash was not needed anymore, so it and related functions were removed.

How do you think?

Best Regards,
Hayato Kuroda
FUJITSU LIMITED