Re: logical decoding and replication of sequences, take 2

Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>

From: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>

To: Tomas Vondra <tomas.vondra@enterprisedb.com>

Cc: Amit Kapila <amit.kapila16@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Masahiko Sawada <sawada.mshk@gmail.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>

Date: 2023-07-24T12:57:13Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Migrate logical slots to the new node during an upgrade.
- 29d0a77fa660 17.0 cited
Make test_decoding ddl.out shorter
- d6677b93c79b 17.0 landed
- c5c5832600e9 14.9 landed
- b1dc946eee3d 16.0 landed
- 3bb8b9342f8a 15.4 landed
Fix snapshot handling in logicalmsg_decode
- 949ac32e1267 15.3 landed
- 8b9cbd42b61f 14.8 landed
- 4df581fa0f4b 13.11 landed
- 497f863f0598 12.15 landed
- 8de91ebf2ac1 11.20 landed
- 7fe1aa991b62 16.0 landed
doc: Adjust a few more references to "postmaster"
- 17e72ec45d31 16.0 cited
Revert "Logical decoding of sequences"
- 2c7ea57e56ca 15.0 cited

On Thu, Jul 20, 2023 at 10:19 PM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:
>
> FWIW there's two questions related to the switch to XLOG_SMGR_CREATE.
>
> 1) Does smgr_decode() need to do the same block as sequence_decode()?
>
>     /* Skip the change if already processed (per the snapshot). */
>     if (transactional &&
>         !SnapBuildProcessChange(builder, xid, buf->origptr))
>         return;
>     else if (!transactional &&
>              (SnapBuildCurrentState(builder) != SNAPBUILD_CONSISTENT ||
>               SnapBuildXactNeedsSkip(builder, buf->origptr)))
>         return;
>
> I don't think it does. Also, we don't have any transactional flag here.
> Or rather, everything is transactional ...

Right.

>
>
> 2) Currently, the sequences hash table is in reorderbuffer, i.e. global.
> I was thinking maybe we should have it in the transaction (because we
> need to do cleanup at the end). It seem a bit inconvenient, because then
> we'd need to either search htabs in all subxacts, or transfer the
> entries to the top-level xact (otoh, we already do that with snapshots),
> and cleanup on abort.
>
> What do you think?

Hash table per transaction seems saner design. Adding it to the top
level transaction should be fine. The entry will contain an XID
anyway. If we add it to every subtransaction we will need to search
hash table in each of the subtransactions when deciding whether a
sequence change is transactional or not. Top transaction is a
reasonable trade off.

-- 
Best Wishes,
Ashutosh Bapat