Horton

Re: POC: make mxidoff 64 bits

Maxim Orlov <orlovmg@gmail.com>

View thread
From: Maxim Orlov <orlovmg@gmail.com>
To: Heikki Linnakangas <hlinnaka@iki.fi>
Cc: Alexander Korotkov <aekorotkov@gmail.com>, wenhui qiu <qiuwenhuifx@gmail.com>, Postgres hackers <pgsql-hackers@lists.postgresql.org>
Date: 2024-10-22T16:33:58Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Fix partial read handling in pg_upgrade's multixact conversion

  2. Increase timeout in multixid_conversion upgrade test

  3. Improve sanity checks on multixid members length

  4. Clarify comment on multixid offset wraparound check

  5. Never store 0 as the nextMXact

  6. Add runtime checks for bogus multixact offsets

  7. Widen MultiXactOffset to 64 bits

  8. Move pg_multixact SLRU page format definitions to a separate header

  9. Convert confusing macros in multixact.c to static inline functions

  10. Index SLRUs by 64-bit integers rather than by 32-bit integers

  11. Cope with possible failure of the oldest MultiXact to exist.

Attachments

On Tue, 22 Oct 2024 at 12:43, Heikki Linnakangas <hlinnaka@iki.fi> wrote:

> MultiXactMemberFreezeThreshold looks quite bogus now. Now that
> MaxMultiXactOffset==2^64-1, you cannot get anywhere near the
> MULTIXACT_MEMBER_SAFE_THRESHOLD and MULTIXACT_MEMBER_DANGER_THRESHOLD
> values anymore. Can we just get rid of MultiXactMemberFreezeThreshold? I
> guess it would still be useful to trigger autovacuum if multixacts
> members grows large though, to release the disk space, even if you can't
> run out of members as such anymore. What should the logic for that look
> like?
>
Yep, you're totally correct. The MultiXactMemberFreezeThreshold call is not
necessary any more and can be safely removed.
I made this as a separate commit in v4. But, as you rightly say, it will be
useful to trigger autovacuum in some cases. The obvious
place for this machinery is in the GetNewMultiXactId. I imagine this like
"if nextOff - oldestOff > threshold kick autovac". So, the
question is: what kind of threshold we want here? Is it a hard coded define
or GUC? If it is a GUC (32–bit), what values should it be?

And the other issue I feel a little regretful about. We still must be
holding MultiXactGenLock in order to track oldestOffset to do
"nextOff - oldestOff" calculation.


>
> I'd love to see some tests for the pg_upgrade code. Something like a
> little perl script to generate test clusters with different wraparound
> scenarios etc.

Agree. I'll address this as soon as I can.

-- 
Best regards,
Maxim Orlov.