Horton
Menu

Thread

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Fix pg_upgrade around multixid and mxoff wraparound

  1. mxid and mxoff wraparound issues in pg_upgrade

    Maxim Orlov <orlovmg@gmail.com> — 2025-06-20T15:45:28Z

    Hi!

I've noticed two bugs reported [0] and [1] which are both related to the
wraparound of mxid and
mxoff. Problems for mxid and mxoff are minor, as they require hitting the
exact overflow limit to
occur. But it's better to correct it.

I included a test to reproduce the problem (see 0001). It is not intended
to be committed, I
guess. I then added a commit with a fix.

There two fixes.
1) pg_upgrade does not consider the mxid to be in a wraparound state. In
this case, I adjust
    the mxid value to the FirstMultiXactId. Alternatively, it might be
conceivable to include the
    ability to set InvalidMultiXactId in pg_resetwal, but this seems odd to
me.
2) pg_resetwall forbids to set mxoff to UINT_MAX32. I'm not sure if this
was done on
    purpose or not, but perhaps this check can be removed.

Here is what I've got with the fix applied.

>>> before upgrade:
Latest checkpoint's NextMultiXactId:  0
Latest checkpoint's NextMultiOffset:  4294967295
Latest checkpoint's oldestMultiXid:   4294967295

>>> after upgrade:
Latest checkpoint's NextMultiXactId:  1
Latest checkpoint's NextMultiOffset:  4294967295
Latest checkpoint's oldestMultiXid:   4294967295

Thoughts?

[0]
https://www.postgresql.org/message-id/18863-72f08858855344a2%40postgresql.org
[1]
https://www.postgresql.org/message-id/18865-d4c66cf35c2a67af%40postgresql.org

-- 
Best regards,
Maxim Orlov.

  2. Re: mxid and mxoff wraparound issues in pg_upgrade

    Heikki Linnakangas <hlinnaka@iki.fi> — 2025-11-05T12:15:38Z

    On 20/06/2025 18:45, Maxim Orlov wrote:
> Hi!
> 
> I've noticed two bugs reported [0] and [1] which are both related to the 
> wraparound of mxid and
> mxoff. Problems for mxid and mxoff are minor, as they require hitting 
> the exact overflow limit to
> occur. But it's better to correct it.
> 
> I included a test to reproduce the problem (see 0001). It is not 
> intended to be committed, I
> guess. I then added a commit with a fix.
> 
> There two fixes.
> 1) pg_upgrade does not consider the mxid to be in a wraparound state. In 
> this case, I adjust
>      the mxid value to the FirstMultiXactId. Alternatively, it might be 
> conceivable to include the
>      ability to set InvalidMultiXactId in pg_resetwal, but this seems 
> odd to me.

I think we should allow InvalidMultiXactId in pg_resetwal. It can appear 
in the control file of existing clusters, so it seems weird if you 
cannot use pg_resetwal to reset to that value.

I also find it weird that we ever store InvalidMultiXactId in the 
control file, and have code in all the places that read it to deal with 
the possibility. It would seem much more straightforward to skip over 
InvalidMultiXactId when we set 'nextMXact'. I think we should change 
that in v19, but that's a separate patch.

> 2) pg_resetwall forbids to set mxoff to UINT_MAX32. I'm not sure if this 
> was done on
>      purpose or not, but perhaps this check can be removed.

With your patch, pg_resetwal will *always* reset nextMultiOffset, even 
if you didn't specify --multixact-offset.

I propose the attached fix. It adds separate booleans to pg_resetwal to 
track whether -m or -O option was given, and allows 0 for the next 
multixid and UINT32_MAX for next multixact offset.

Can oldestMultiXactId ever be 0? I think not, but then again 
ReadMultiXactIdRange() seems to be prepared to deal with that.

- Heikki

  3. Re: mxid and mxoff wraparound issues in pg_upgrade

    Heikki Linnakangas <hlinnaka@iki.fi> — 2025-11-12T11:14:00Z

    On 05/11/2025 14:15, Heikki Linnakangas wrote:
> I propose the attached fix. It adds separate booleans to pg_resetwal to 
> track whether -m or -O option was given, and allows 0 for the next 
> multixid and UINT32_MAX for next multixact offset.

Committed, with one more change:

I replaced strtoul() with strtoi64() in parsing the multixact offset. 
strtoul() silently casts negative values to positive ones, which means 
that even before this patch, we accepted any negative value, except for 
-1 which was checked specifically. That's bogus, and with the previous 
patch I posted, we would start accepting -1 too, which makes it even 
worse.  By using strtoi64() and converting to uint32 explicitly, we now 
check and error out on all negative values.

We have the same issue with all the other flags that use strtoul(), but 
I didn't want to touch those right now, especially in back-branches. But 
I also didn't want to start accepting -1 for -O. On 'master' I think it 
would make sense to tighten up the other flags, too. I don't plan to 
work on that myself but would be happy to review if someone writes the 
patch.

I didn't commit the new pg_upgrade test case, but many thanks for 
providing it. It was very helpful in demonstrating the problem.

- Heikki