Horton

Re: POC: make mxidoff 64 bits

Maxim Orlov <orlovmg@gmail.com>

View thread
From: Maxim Orlov <orlovmg@gmail.com>
To: Heikki Linnakangas <hlinnaka@iki.fi>
Cc: wenhui qiu <qiuwenhuifx@gmail.com>, Alexander Korotkov <aekorotkov@gmail.com>, Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>, Postgres hackers <pgsql-hackers@lists.postgresql.org>
Date: 2025-11-06T15:50:23Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Fix partial read handling in pg_upgrade's multixact conversion

  2. Increase timeout in multixid_conversion upgrade test

  3. Improve sanity checks on multixid members length

  4. Clarify comment on multixid offset wraparound check

  5. Never store 0 as the nextMXact

  6. Add runtime checks for bogus multixact offsets

  7. Widen MultiXactOffset to 64 bits

  8. Move pg_multixact SLRU page format definitions to a separate header

  9. Convert confusing macros in multixact.c to static inline functions

  10. Index SLRUs by 64-bit integers rather than by 32-bit integers

  11. Cope with possible failure of the oldest MultiXact to exist.

Attachments

On Wed, 5 Nov 2025 at 18:38, Heikki Linnakangas <hlinnaka@iki.fi> wrote:

>
> Remaining issues:
>
> - There's one more refactoring I'd like to do before merging this: Move
> the definitions that are now duplicated between
> src/bin/pg_upgrade/multixact_new.c and
> src/backend/access/transam/multixact.c into a new header file,
> multixact_internal.h. One complication with that is that it needs
> SLRU_PAGES_PER_SEGMENT ...

Done. Also put SLRU_PAGES_PER_SEGMENT in pg_config_manual.h
In my opinion, this constant perfectly aligns the description in the
file header. In any case, feel free to move it anywhere you like.


- Have you done any performance testing of the pg_upgrade code? How long
> does the conversion take if you have e.g. 1 billion multixids?
>
Unfortunately, not yet. I'd like to do this soon. Currently, the
bulk of the testing time is spent generating multi-transactions.


- Is the !oldestOffsetKnown case in the code still reachable? I left one
> FIXME comment about that. Needs a comment update at least.
>
Yep, no longer needed. A separate commit has been added.


- The new pg_upgrade test fails on my system with this error in the log:

Unfortunately, I don't face this issue. I think this can be fixed by
providing an explicit path to the utility.


- The new pg_ugprade test is very slow. I would love to include that
> test permanently in the test suite, but it's too slow for that currently.

Yes, unfortunately. The majority of the time is spent on tests that
produce multiple segments. These are cases numbered 4-th and higher.
If we remove these, the testing should be relatively fast.


I also add commit "Handle wraparound of next new multi in pg_upgrade".
Per BUG #18863 and BUG #18865

The issue is that pg_upgrade neglects to handle the wraparound of
mxact/mxoff.

We'll obviously resolve the issue with mxoff wraparound by moving to
64-bits. And the mxact bug can be easily solved with two lines of code.
Or five if you count indents and comments. Test also provided.

This commit is totally optional. If you think it deserves to be treated
as a different issue, feel free to discard it.

-- 
Best regards,
Maxim Orlov.