Horton

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

Masahiko Sawada <sawada.mshk@gmail.com>

View thread
From: Masahiko Sawada <sawada.mshk@gmail.com>
To: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Cc: vignesh C <vignesh21@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2025-01-13T21:45:15Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Fix regression test failure when wal_level is set to minimal.

  2. Toggle logical decoding dynamically based on logical slot presence.

  3. Disallow server start with sync_replication_slots = on and wal_level < logical.

On Mon, Jan 13, 2025 at 1:31 AM Ashutosh Bapat
<ashutosh.bapat.oss@gmail.com> wrote:
>
> On Mon, Jan 13, 2025 at 2:52 PM vignesh C <vignesh21@gmail.com> wrote:
> >
> > I felt that the only disadvantage with this approach is that we
> > currently wait for all in-progress transactions to complete before
> > enabling logical decoding. If a long-running transaction exists and
> > the session enabling logical decoding fails—due to factors like
> > statement_timeout, transaction_timeout,
> > idle_in_transaction_session_timeout, idle_session_timeout, or any
> > other failure. This would require restarting the wait. During this
> > time, there's a risk that a new long-running transaction could start,
> > further delaying the process. Probably this could be solved if the
> > waiting is done from any of the background processes through
> > PGC_SIGHUP.  While this type of failure is likely rare, I’m unsure
> > whether we should consider this scenario.
>
> A related question: While the operation is waiting for already running
> transactions to end, the backends whose transactions have finished may
> start new transactions. When we switch WAL from 'replica' to
> 'logical', there may be some transactions running. Will this lead to
> WAL stream with mixes WAL records - some with logical information and
> some without?

Yes. There could be mixed WAL records until all running transactions complete.

> Do we need to adjust logical decoding to tackle this
> situation? Is there a chance that some WAL from same transaction have
> logical information and some do not?

In the current approach, we first enable logical info WAL-logging to
let newly started transactions do logical info WAL-logging, then allow
the logical decoding only after all running transactions complete.
Therefore, at the time when we allow the logical decoding, all WAL
records are written with logical information.

Regards,

-- 
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com