Horton

Re: Switching XLog source from archive to streaming when primary available

Nathan Bossart <nathandbossart@gmail.com>

View thread
From: Nathan Bossart <nathandbossart@gmail.com>
To: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Cc: Kyotaro Horiguchi <horikyota.ntt@gmail.com>, cary.huang@highgo.ca, pgsql-hackers@lists.postgresql.org, satyanarlapuram@gmail.com
Date: 2022-10-09T21:47:25Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add basic TAP tests for the low-level backup method, take two

  2. Add a failover option to subscriptions.

  3. Make all Perl warnings fatal

  4. Add PostgreSQL::Test::Cluster::advance_wal

  5. Rename wal_keep_segments to wal_keep_size.

On Sun, Oct 09, 2022 at 02:39:47PM +0530, Bharath Rupireddy wrote:
> We can give it a chance to restore from pg_wal before switching to
> streaming to not change any behaviour of the state machine. But, not
> definitely by setting currentSource to XLOG_FROM_WAL, we basically
> never explicitly set currentSource to XLOG_FROM_WAL, other than when
> not in archive recovery i.e. InArchiveRecovery is false. Also, see the
> comment [1].
> 
> Instead, the simplest would be to just pass XLOG_FROM_WAL to
> XLogFileReadAnyTLI() when we're about to switch the source to stream
> mode. This doesn't change the existing behaviour.

It might be more consistent with existing behavior, but one thing I hadn't
considered is that it might make your proposed feature ineffective when
users are copying files straight into pg_wal.  IIUC as long as the files
are present in pg_wal, the source-switch logic won't kick in.

> Unrelated to this patch, the fact that the standby polls pg_wal is not
> documented or recommended, is not true, it is actually documented [2].
> Whether or not we change the docs to be something like [3], is a
> separate discussion.

I wonder if it would be better to simply remove this extra polling of
pg_wal as a prerequisite to your patch.  The existing commentary leads me
to think there might not be a strong reason for this behavior, so it could
be a nice way to simplify your patch.

-- 
Nathan Bossart
Amazon Web Services: https://aws.amazon.com