Horton

Re: logical decoding and replication of sequences, take 2

Andres Freund <andres@anarazel.de>

View thread
From: Andres Freund <andres@anarazel.de>
To: Tomas Vondra <tomas.vondra@enterprisedb.com>
Cc: Robert Haas <robertmhaas@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2023-01-11T21:53:10Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Migrate logical slots to the new node during an upgrade.

  2. Make test_decoding ddl.out shorter

  3. Fix snapshot handling in logicalmsg_decode

  4. doc: Adjust a few more references to "postmaster"

  5. Revert "Logical decoding of sequences"

Hi,

On 2023-01-11 22:30:42 +0100, Tomas Vondra wrote:
> On 1/11/23 21:58, Andres Freund wrote:
> > If you're thinking of decoding changes in parallel (rather than streaming out
> > large changes before commit when possible), you'd only be able to do that in
> > cases when transaction haven't performed catalog changes, I think. In which
> > case there'd also be no issue wrt transactional sequence changes.
> > 
> 
> Perhaps, although it's not clear to me how would you know that in
> advance? I mean, you could start decoding changes in parallel, and then
> you find one of the earlier transactions touched a catalog.

You could have a running count of in-progress catalog modifying transactions
and not allow parallelized processing when that's not 0.


> Bu maybe I misunderstand what "decoding" refers to - don't we need the
> snapshot only in reorderbuffer? In which case all the other stuff could
> be parallelized (not sure if that's really expensive).

Calling output functions is pretty expensive, so being able to call those in
parallel has some benefits. But I don't think we're there.


> Anyway, all of this is far out of scope of this patch.

Yea, clearly that's independent work. And I don't think relying on commit
order in one more place, i.e. for sequences, would make it harder.

Greetings,

Andres Freund