Re: Requested WAL segment xxx has already been removed

On Mon, 14 Jul 2025 at 20:33, Fujii Masao <masao.fujii@oss.nttdata.com> wrote:
> On 2025/07/14 17:08, Japin Li wrote:
>> Hi all,
>> I recently hit an error with our streaming replication setup:
>>    2025-07-14 11:52:59.361
>> CST,"replicator","",728458,"10.9.9.74:35724",68747f1b.b1d8a,1,"START_REPLICATION",2025-07-14
>> 11:52:59 CST,3/0,0,ERROR,58P01,"requested WAL segment
>> 00000001000000000000000C has already been
>> removed",,,,,,"START_REPLICATION 0/C000000 TIMELINE
>> 1",,,"standby","walsender",,0
>> It appears the requested WAL segment 00000001000000000000000C had
>> already been
>> archived, and I confirmed its presence in the archive directory. However, when
>> the standby tried to request this file, the primary only searched for it in
>> pg_wal and didn't check the archive directory. I had to manually copy the
>> segment into pg_wal to get streaming replication working again.
>> My question is: Can we make the primary automatically search the
>> archive if
>> restore_command is set?
>> I found that Fujii Masao also requested this feature [1], but it
>> seems there
>> wasn't a consensus.
>
> Yeah, I still like this idea. It's useful, for example, when we want to
> temporarily retain WAL files, such as during planned standby maintenance,
> to avoid "requested WAL segment ... removed." error.
>
> Using a replication slot is one way to retain WAL files in pg_wal,
> but it requires the pg_wal directory to be large enough to hold all
> WAL generated during that time, which isn't always practical.
>

Agreed.  Here is a patch that fixes this.

-- 
Regards,
Japin Li