Horton

Re: logical decoding and replication of sequences

Amit Kapila <amit.kapila16@gmail.com>

View thread View on postgr.es
From: Amit Kapila <amit.kapila16@gmail.com>
To: Tomas Vondra <tomas.vondra@enterprisedb.com>
Cc: Petr Jelinek <petr.jelinek@enterprisedb.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2022-04-02T10:35:35Z
Lists: pgsql-hackers 
On Fri, Apr 1, 2022 at 8:32 PM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:
>
> On 3/28/22 07:29, Amit Kapila wrote:
> > I thought about changing snapshot dealing of
> > non-transactional sequence changes similar to transactional ones but
> > that also won't work because it is only at commit we decide whether we
> > can send the changes.
> >
> I wonder if there's some earlier LSN (similar to the consistent point)
> which might be useful for this.
>
> Or maybe we should queue even the non-transactional changes, not
> per-transaction but in a global list, and then at each commit either
> discard inspect them (at that point we know the lowest LSN for all
> transactions and the consistent point). Seems complex, though.
>

I couldn't follow '..discard inspect them ..'. Do you mean we inspect
them and discard whichever are not required? It seems here we are
talking about a new global ReorderBufferGlobal instead of
ReorderBufferTXN to collect these changes but we don't need only
consistent point LSN because we do send if the commit of containing
transaction is after consistent point LSN, so we need some transaction
information as well. I think it could bring new challenges.

> > For the transactional case, as we are considering the create sequence
> > operation as transactional, we would unnecessarily queue them even
> > though that is not required. Basically, they don't need to be
> > considered transactional and we can simply ignore such messages like
> > other DDLs. But for that probably we need to distinguish Alter/Create
> > case which may or may not be straightforward. Now, queuing them is
> > probably harmless unless it causes the transaction to spill/stream.
> >
>
> I'm not sure I follow. Why would we queue them unnecessarily?
>
> Also, there's the bug with decoding changes in transactions that create
> the sequence and add it to a publication. I think the agreement was that
> this behavior was incorrect, we should not decode changes until the
> subscription is refreshed. Doesn't that mean can't be any CREATE case,
> just ALTER?
>

Yeah, but how will we distinguish them. Aren't they using the same
kind of WAL record?

-- 
With Regards,
Amit Kapila.

Commits

  1. Fix cache invalidation bug in recovery_prefetch.

  2. Revert "Logical decoding of sequences"

  3. Minor improvements in sequence decoding code and docs

  4. Handle sequences in preprocess_pubobj_list

  5. Update tab-completion for CREATE PUBLICATION with sequences

  6. Add decoding of sequences to built-in replication

  7. Stabilize test_decoding touching with sequences

  8. Call ReorderBufferProcessXid from sequence_decode

  9. Add decoding of sequences to test_decoding

  10. Replace Test::More plans with done_testing

  11. Logical decoding of sequences

  12. Respect permissions within logical replication.