Horton

Re: logical decoding and replication of sequences, take 2

Tomas Vondra <tomas.vondra@enterprisedb.com>

View thread
From: Tomas Vondra <tomas.vondra@enterprisedb.com>
To: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Masahiko Sawada <sawada.mshk@gmail.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Date: 2023-07-29T13:03:31Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Migrate logical slots to the new node during an upgrade.

  2. Make test_decoding ddl.out shorter

  3. Fix snapshot handling in logicalmsg_decode

  4. doc: Adjust a few more references to "postmaster"

  5. Revert "Logical decoding of sequences"

Attachments

On 7/28/23 14:35, Ashutosh Bapat wrote:
>
> ...
>
> We hold a strong lock on sequence when changing its relfilenode. The
> sequence whose relfilenode is being changed can not be accessed by any
> concurrent transaction. So I am not able to understand what you are
> trying to say.
> 
> I think per (top level) transaction hash table is cleaner design. It
> puts the hash table where it should be. But if that makes code
> difficult, current design works too.
> 

I was thinking about switching to the per-txn hash, so here's a patch
adopting that approach (in part 0006). I can't say it's much simpler,
but maybe it can be simplified a bit. Most of the complexity comes from
assignments maybe happening with a delay, so it's hard to say what's a
top-level xact.

The patch essentially does this:

1) the HTAB is moved to ReorderBufferTXN

2) after decoding SGMR_CREATE, we add an entry to the current TXN and
(for subtransactions) to the parent TXN (even the copy references the
subxact)

3) when processing an assignment, we copy the HTAB entries from the
subxact to the parent

4) after a subxact abort, we remove the HTAB entries from the parent

5) while searching for the relfilenode, we only scan the HTAB in the
top-level xacts (this is possible due to the copying)

This could work without the copy in parent HTAB, but then we'd have to
scan all the transactions for every increment. And there may be many
lookups and many (sub)transactions, but only a small number of new
relfilenodes. So it seems like a good tradeoff.

If we could convince ourselves the subxact has to be already assigned
while decoding the sequence change, then we could simply search only the
current transaction (and the parent). But I've been unable to convince
myself that's guaranteed.


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company