Horton

Re: Switching XLog source from archive to streaming when primary available

Andrey Borodin <x4mmm@yandex-team.ru>

View thread
From: "Andrey M. Borodin" <x4mmm@yandex-team.ru>
To: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Cc: Michael Paquier <michael@paquier.xyz>, Nathan Bossart <nathandbossart@gmail.com>, Japin Li <japinli@hotmail.com>, Ian Lawrence Barwick <barwick@gmail.com>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>, Cary Huang <cary.huang@highgo.ca>, SATYANARAYANA NARLAPURAM <satyanarlapuram@gmail.com>, pgsql-hackers@lists.postgresql.org
Date: 2025-01-02T18:12:37Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add basic TAP tests for the low-level backup method, take two

  2. Add a failover option to subscriptions.

  3. Make all Perl warnings fatal

  4. Add PostgreSQL::Test::Cluster::advance_wal

  5. Rename wal_keep_segments to wal_keep_size.


> On 23 Mar 2024, at 14:22, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> wrote:
> 
> IMHO, it makes sense to have something like replay_source_order if
> there's any use case that arises in future requiring the standby to
> intentionally switch to pg_wal or archive. But not as part of this
> feature.

IMO, it's vital part of a feature.

In my observation restore from archive is many orders of magnitude faster than streaming replication. Advanced archive tools employ compression (x6 to speed), download parallelism (x4), are not constrained be primary's network limits (x3) and disk limits, do not depend on complicated FEBE protocol, etc.
When I have to cope with lagging replica, almost always I kill walreceiver and tweak server readahead.

But there might be cases where you still have to attach replica ASAP. I can think of releasing replication slot, transiently failed archive network or storage.

Finally, one might want to have many primary connections: cascading replica might want to stream from any available host from the group of HA hosts.


Best regards, Andrey Borodin.