Horton

Re: logical decoding and replication of sequences, take 2

Tomas Vondra <tomas.vondra@enterprisedb.com>

View thread
From: Tomas Vondra <tomas.vondra@enterprisedb.com>
To: Dilip Kumar <dilipbalaut@gmail.com>, Robert Haas <robertmhaas@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Masahiko Sawada <sawada.mshk@gmail.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Date: 2024-03-06T17:34:15Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Migrate logical slots to the new node during an upgrade.

  2. Make test_decoding ddl.out shorter

  3. Fix snapshot handling in logicalmsg_decode

  4. doc: Adjust a few more references to "postmaster"

  5. Revert "Logical decoding of sequences"

Hi,

Let me share a bit of an update regarding this patch and PG17. I have
discussed this patch and how to move it forward with a couple hackers
(both within EDB and outside), and my takeaway is that the patch is not
quite baked yet, not enough to make it into PG17 :-(

There are two main reasons / concerns leading to this conclusion:

* correctness of the decoding part

There are (were) doubts about decoding during startup, before the
snapshot gets consistent, when we can get "temporarily incorrect"
decisions whether a change is transactional. While the behavior is
ultimately correct (we treat all changes as non-transactional and
discard it), it seems "dirty" and it’s unclear to me if it might cause
more serious issues down the line (not necessarily bugs, but perhaps
making it harder to implement future changes).

* handling of sequences in built-in replication

Per the patch, sequences need to be added to the publication explicitly.
But there were suggestions we might (should) add certain sequences
automatically - e.g. sequences backing SERIAL/BIGSERIAL columns, etc.
I’m not sure we really want to do that, and so far I assumed we would
start with the manual approach and move to automatic addition in the
future. But the agreement seems to be it would be a pretty significant
"breaking change", and something we probably don’t want to do.


If someone feels has an opinion on either of the two issues (in either
way), I'd like to hear it.


Obviously, I'm not particularly happy about this outcome. And I'm also
somewhat cautious because this patch was already committed+reverted in
PG16 cycle, and doing the same thing in PG17 is not on my wish list.


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company