Horton

Thread

Commits

  1. Don't reset 'latest_page_number' when replaying multixid truncation

  2. Set next multixid's offset when creating a new multixid

  1. Proposal: Prevent Primary/Standby SLRU divergence during MultiXact truncation

    Ayush Tiwari <ayushtiwari.slg01@gmail.com> — 2026-03-16T16:09:30Z

    Hi Hackers,

Looking at the MultiXact truncation behavior and reading through the recent
thread regarding the 17.8 standby crashing during WAL replay (commit
8ba61bc), we noticed an architectural edge case that seems to cause a
silent primary/standby SLRU divergence. I'd like to ask if this is a known
accepted risk or if a patch to reorder this logic is worth exploring.

The Issue:
In TruncateMultiXact(), we write the truncation WAL record
(WriteMTruncateXlogRec) before we actually perform the truncation via
PerformOffsetsTruncation() -> SimpleLruTruncate().

The problem arises from the "apparent wraparound" safety check inside
SimpleLruTruncate(). If SlruScanDirectory() detects an apparent wraparound,
SimpleLruTruncate() safely bails out and skips unlinking the SLRU segments
on the primary, logging: could not truncate directory "%s": apparent
wraparound.

However, the WAL record for the truncation has already been flushed.
Standbys replay this TRUNCATE_ID WAL record and blindly delete their SLRU
segments. At this point, the primary and standby have diverged.

The Impact:
If the standby is subsequently promoted to primary, any attempt to access
rows holding those older MultiXact IDs (which the original primary decided
to keep) will throw a FATAL: could not access status of transaction error,
effectively resulting in data loss / inaccessible rows for the user.

While the recent commits address the immediate standby crash involving
latest_page_number during multixact_redo(), they don't seem to prevent the
primary from emitting a "false" WAL truncation record when it abandons its
own truncation.

Proposed Approach:
It seems safer to only emit the WAL record if we are guaranteed to follow
through with the truncation. We could modify SimpleLruTruncate() to perform
its safety checks first and return a boolean indicating whether the
truncation is safe to proceed. TruncateMultiXact() would then only call
WriteMTruncateXlogRec() and proceed with physical deletion if the check
passes.

I have attached a rough draft patch illustrating this sequence change.

Is this a scenario the community has already considered, or is this
reordering something that should be explored further to harden standby
reliability?

PS. Also, noticed this to be the case in clog.c file

Thanks for your time.

Regards,
Ayush

  2. Re: Proposal: Prevent Primary/Standby SLRU divergence during MultiXact truncation

    Heikki Linnakangas <hlinnaka@iki.fi> — 2026-03-16T22:10:41Z

    On 16/03/2026 18:09, Ayush Tiwari wrote:
> The Issue:
> In TruncateMultiXact(), we write the truncation WAL record 
> (WriteMTruncateXlogRec) before we actually perform the truncation via 
> PerformOffsetsTruncation() -> SimpleLruTruncate().
> 
> The problem arises from the "apparent wraparound" safety check inside 
> SimpleLruTruncate(). If SlruScanDirectory() detects an apparent 
> wraparound, SimpleLruTruncate() safely bails out and skips unlinking the 
> SLRU segments on the primary, logging: could not truncate directory 
> "%s": apparent wraparound.
> 
> However, the WAL record for the truncation has already been flushed. 
> Standbys replay this TRUNCATE_ID WAL record and blindly delete their 
> SLRU segments. At this point, the primary and standby have diverged.

Replaying the record will perform the same sanity checks against 
wraparound as the primary does.

Hmm, although why did I not apply commit 817f74600d to 'master', only 
backbranches? The bug that it fixed was related to minor version 
upgrade, and thus it was not needed on 'master', but the code change 
would nevertheless make a lot of sense on 'master' too.

> The Impact:
> If the standby is subsequently promoted to primary, any attempt to 
> access rows holding those older MultiXact IDs (which the original 
> primary decided to keep) will throw a FATAL: could not access status of 
> transaction error, effectively resulting in data loss / inaccessible 
> rows for the user.

Have you been able to reproduce that?

> While the recent commits address the immediate standby crash involving 
> latest_page_number during multixact_redo(), they don't seem to prevent 
> the primary from emitting a "false" WAL truncation record when it 
> abandons its own truncation.
> 
> Proposed Approach:
> It seems safer to only emit the WAL record if we are guaranteed to 
> follow through with the truncation. We could modify SimpleLruTruncate() 
> to perform its safety checks first and return a boolean indicating 
> whether the truncation is safe to proceed. TruncateMultiXact() would 
> then only call WriteMTruncateXlogRec() and proceed with physical 
> deletion if the check passes.
> 
> I have attached a rough draft patch illustrating this sequence change.

I agree that would probably be better. I'm not sure how straightforward 
it will be to implement though, I wouldn't want to add much extra code 
just for this.

P.S. Thanks for looking into this! This is hairy stuff, more review is 
much appreciated.

- Heikki

  3. Re: Proposal: Prevent Primary/Standby SLRU divergence during MultiXact truncation

    Ayush Tiwari <ayushtiwari.slg01@gmail.com> — 2026-03-17T15:07:26Z

    Hi,

Thank you for the response.

On Tue, 17 Mar 2026 at 03:40, Heikki Linnakangas <hlinnaka@iki.fi> wrote:

>
> Replaying the record will perform the same sanity checks against
> wraparound as the primary does.
>
> Hmm, although why did I not apply commit 817f74600d to 'master', only
> backbranches? The bug that it fixed was related to minor version
> upgrade, and thus it was not needed on 'master', but the code change
> would nevertheless make a lot of sense on 'master' too.
>

Agreed, once 817f74600d is on master the standby would honestly evaluate
the SimpleLruTruncate wraparound backstop instead of bypassing it.

However, the backstop is documented as catching "wraparound bugs elsewhere
in SLRU handling." If such a bug corrupts latest_page_number on the
primary, the standby — which derives its latest_page_number independently
from ZERO_OFF_PAGE replay and StartupMultiXact() — would not share the same
corruption. The primary would skip the truncation, but the standby would
see a healthy latest_page_number and proceed.


> Have you been able to reproduce that?
>

I have reproduced the primary-side condition on an unmodified tree using
gdb in batch mode: attach to the VACUUM backend after
WriteMTruncateXlogRec() returns, corrupt latest_page_number, and resume.
The primary logs "apparent wraparound" and skips the physical deletion,
while pg_waldump confirms the TRUNCATE_ID record is present in the WAL. I
have not yet set up a streaming replica to demonstrate end-to-end
divergence and promotion failure.

>
> I agree that would probably be better. I'm not sure how straightforward
> it will be to implement though, I wouldn't want to add much extra code
> just for this.
>

One approach that might keep the footprint small: we could inline the same
PagePrecedes check that SimpleLruTruncate uses directly in
TruncateMultiXact(), before START_CRIT_SECTION(). Something like:

if (MultiXactOffsetCtl->PagePrecedes(
        pg_atomic_read_u64(&MultiXactOffsetCtl->shared->latest_page_number),
        MultiXactIdToOffsetPage(PreviousMultiXactId(newOldestMulti))) ||
    MultiXactMemberCtl->PagePrecedes(
        pg_atomic_read_u64(&MultiXactMemberCtl->shared->latest_page_number),
        MXOffsetToMemberPage(newOldestOffset)))
{
    ereport(LOG,
            (errmsg("skipping multixact truncation due to apparent
wraparound")));
    LWLockRelease(MultiXactTruncationLock);
    return;
}

No new functions, no changes to slru.c or the replay path — just the same
condition evaluated earlier so we never enter the critical section or write
WAL for a truncation that won't be carried out. Does this seem like a
reasonable direction?

Regards,
Ayush

  4. Re: Proposal: Prevent Primary/Standby SLRU divergence during MultiXact truncation

    Heikki Linnakangas <hlinnaka@iki.fi> — 2026-03-22T12:29:08Z

    On 17/03/2026 17:07, Ayush Tiwari wrote:
> On Tue, 17 Mar 2026 at 03:40, Heikki Linnakangas <hlinnaka@iki.fi 
> <mailto:hlinnaka@iki.fi>> wrote:
> 
>> Hmm, although why did I not apply commit 817f74600d to 'master', only
>> backbranches? The bug that it fixed was related to minor version
>> upgrade, and thus it was not needed on 'master', but the code change
>> would nevertheless make a lot of sense on 'master' too.
> 
> Agreed, once 817f74600d is on master the standby would honestly evaluate 
> the SimpleLruTruncate wraparound backstop instead of bypassing it.

I now committed that change to master too.

- Heikki