Horton

Re: POC: make mxidoff 64 bits

Heikki Linnakangas <hlinnaka@iki.fi>

View thread
From: Heikki Linnakangas <hlinnaka@iki.fi>
To: Maxim Orlov <orlovmg@gmail.com>
Cc: wenhui qiu <qiuwenhuifx@gmail.com>, Alexander Korotkov <aekorotkov@gmail.com>, Postgres hackers <pgsql-hackers@lists.postgresql.org>
Date: 2025-11-14T17:48:39Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Fix partial read handling in pg_upgrade's multixact conversion

  2. Increase timeout in multixid_conversion upgrade test

  3. Improve sanity checks on multixid members length

  4. Clarify comment on multixid offset wraparound check

  5. Never store 0 as the nextMXact

  6. Add runtime checks for bogus multixact offsets

  7. Widen MultiXactOffset to 64 bits

  8. Move pg_multixact SLRU page format definitions to a separate header

  9. Convert confusing macros in multixact.c to static inline functions

  10. Index SLRUs by 64-bit integers rather than by 32-bit integers

  11. Cope with possible failure of the oldest MultiXact to exist.

Attachments

On 14/11/2025 17:40, Maxim Orlov wrote:
> On Wed, 12 Nov 2025 at 16:00, Heikki Linnakangas <hlinnaka@iki.fi> 
>> I kept all the new test cases for now. We need to decide which
>> ones are worth keeping, and polish and speed up the ones we decide
>> to keep.

Attached is a new patch version, with more work on the tests. The 
pg_upgrade patch 
(v26-0004-Add-pg_upgrade-for-64-bit-multixact-offsets.patch) now 
includes a test case. I'm proposing to commit that test along with these 
patches. It's a heavily-modified version of the test cases you wrote.

I tested that test using old installations, all the way down to version 
9.4. That required a bunch of changes to the test perl modules, to make 
them work with such old versions. Without any extra changes, the test 
works down to v11.

Later patches in the patch set add more tests, labelled with the TEST: 
prefix. Those are the tests you posted earlier, with little to no 
modifications. I'm just carrying those around, so that I can easily run 
them now during development. But I don't think they're adding much value 
and I plan to leave them out of the final commit.


> I think of two cases here.
> A) Upgrade from "new cluster":
>      * created cluster with pre 32-bit overflow mxoff
>      * consume around of 2k of mxacts (1k before 32-bit overflow
>        and 1k after)
>      * run pg_upgrade
>      * check upgraded cluster is working
>      * check data invariant
> B)  Same as A), but for an "old cluster" with oldinstall env.

Makes sense.

The 007_multixact_conversion.pl test in the attached patches includes 
two test scenarios:  "basic" and "wraparound" test. In the basic 
scenario there's no overflow or wraparound involved, but it can be run 
without an old installation, i.e. in a "new -> new upgrade". The 
"wraparound" scenario is the same, but the old cluster is reset with 
pg_resetwal so that the mxoff wraps around. The "wraparound" requires a 
pre-19 old installation, because the pg_resetwal logic requires pre-v19 
layout.

If we enhance the reset_mxoff() perl function in the test so that it 
also works with v19, we could run the "wraparound" scenario in new->new 
upgrades too. That would essentially the case A) that you listed above.

I think it's already pretty good as it is though. I don't expect the 
point where we cross offset 2^32 in the new version to be very 
interesting now that we use 64-bit offsets everywhere.

- Heikki