Re: POC: make mxidoff 64 bits

Heikki Linnakangas <hlinnaka@iki.fi>

From: Heikki Linnakangas <hlinnaka@iki.fi>

To: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>, Maxim Orlov <orlovmg@gmail.com>

Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>, Alexander Korotkov <aekorotkov@gmail.com>, wenhui qiu <qiuwenhuifx@gmail.com>, Postgres hackers <pgsql-hackers@lists.postgresql.org>

Date: 2025-12-08T13:02:11Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Fix partial read handling in pg_upgrade's multixact conversion
- ac94ce8194e5 19 (unreleased) landed
Increase timeout in multixid_conversion upgrade test
- bd43940b02b2 19 (unreleased) landed
Improve sanity checks on multixid members length
- ecb553ae8211 19 (unreleased) landed
Clarify comment on multixid offset wraparound check
- 170361d7b869 14.21 landed
- b0b52b7123ae 15.16 landed
- 7d42e2367c6b 16.12 landed
- cd1a887fe9bf 17.8 landed
- 3fbad030a24d 18.2 landed
- 366dcdaf5779 19 (unreleased) landed
Never store 0 as the nextMXact
- 87a350e1f284 19 (unreleased) landed
Add runtime checks for bogus multixact offsets
- d4b7bde4183b 19 (unreleased) landed
Widen MultiXactOffset to 64 bits
- bd8d9c9bdfa0 19 (unreleased) landed
Move pg_multixact SLRU page format definitions to a separate header
- bb3b1c4f6462 19 (unreleased) landed
Convert confusing macros in multixact.c to static inline functions
- 0099b9408e8c 17.0 landed
Index SLRUs by 64-bit integers rather than by 32-bit integers
- 4ed8f0913bfd 17.0 cited
Cope with possible failure of the oldest MultiXact to exist.
- b6a3444fa635 9.4.4 cited

Attachments

v29-0001-pg_resetwal-Reject-negative-and-out-of-range-arg.patch (text/x-patch) patch v29-0001
v29-0002-pg_resetwal-Use-separate-flags-for-whether-an-op.patch (text/x-patch) patch v29-0002
v29-0003-Move-pg_multixact-SLRU-page-format-definitions-t.patch (text/x-patch) patch v29-0003
v29-0004-FIXME-bump-catversion.patch (text/x-patch) patch v29-0004
v29-0005-Widen-MultiXactOffset-to-64-bits.patch (text/x-patch) patch v29-0005
v29-0006-Add-runtime-checks-for-bogus-multixact-offsets.patch (text/x-patch) patch v29-0006

On 06/12/2025 01:36, Heikki Linnakangas wrote:
> On 05/12/2025 15:42, Ashutosh Bapat wrote:
>> + $newnode->start;
>> + my $new_dump = get_dump_for_comparison($newnode, "newnode_${tag} 
>> _dump");
>> + $newnode->stop;
>>
>>
>> There is no code which actually looks at the multixact offsets here to
>> make sure that the conversion happened correctly. I guess the test
>> relies on visibility checks for that. Anyway, we need a comment
>> explaining why just comparing the contents of the table is enough to
>> ensure correct conversion. Better if we can add an explicit test that
>> the offsets were converted correctly. I don't have any idea of how to
>> do that right now, though. Maybe use pg_get_multixact_members()
>> somehow in the query to extract data out of the table?
> 
> Agreed, the verification here is quite weak. I didn't realize that 
> pg_get_multixact_members() exists! That might indeed be handy here, but 
> I'm not sure how exactly to construct the test. A direct C function like 
> test_create_multixact() in test_multixact.c would be handy here, but 
> we'd need to compile and do run that in the old cluster, which seems 
> difficult.

I added verification of all the multixids between oldest and next 
multixid, using pg_get_multixact_members(). The test now calls 
pg_get_multixact_members() for all updating multixids in the range, 
before and after the upgrade, and compares the results.

The verification ignores locking-only multixids. Verifying their 
correctness would need a little more code because they're not fully 
preserved by the upgrade.

I also expanded the test to cover multixid wraparound. It only covered 
mxoffset wraparound previously.

New patch set attached. Only test changes compared to patch set v28.

- Heikki