Horton

Re: Add 64-bit XIDs into PostgreSQL 15

Pavel Borisov <pashkin.elfe@gmail.com>

View thread
From: Pavel Borisov <pashkin.elfe@gmail.com>
To: "Finnerty, Jim" <jfinnert@amazon.com>
Cc: Simon Riggs <simon.riggs@enterprisedb.com>, Maxim Orlov <orlovmg@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2022-01-12T14:03:11Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add SLRU tests for 64-bit page case

  2. Make use FullTransactionId in 2PC filenames

  3. Use larger segment file names for pg_notify

  4. Index SLRUs by 64-bit integers rather than by 32-bit integers

>
> Maxim, maybe it's still a good idea to isolate those two patches and
> submit them separately first, to reduce the size of the rest of the patch?
>


>     Looks to me like the best next actions would be:
>
>     1. Submit a patch that uses XID_FMT everywhere, as a cosmetic change.
>     This looks like it will reduce the main patch size considerably and
>     make it much less scary. That can be cleaned up and committed while we
>     discuss the main approach.
>
>     2. Write up the approach in a detailed README, so people can
>     understand the proposal and assess if there are problems. A few short
>     notes and a link back to old conversations isn't enough to allow wide
>     review and give confidence on such a major patch.
>

Big thanks to all for your ideas!

We intend to do the following work on the patch soon:
1. Write a detailed README
2. Split the patch into several pieces including a separate part for
XID_FMT. But if committers meanwhile choose to commit Jim's XID_FMT patch
we also appreciate this and will rebase our patch accordingly.
     2A. Probably refactor it to store precalculated XMIN/XMAX in memory
tuple representation instead of t_xid_base/t_multi_base
     2B. Split the in-memory part of a patch as a separate
3. Construct some variants for leaving "double xmax" format as a temporary
one just after upgrade for having only one persistent on-disk format
instead of two.
     3A. By using SQL function "vacuum doublexmax;"
OR
     3B. By freeing space on all heap pages for pd_special before
pg-upgrade.
OR
     3C. By automatically repacking all "double xmax" pages after upgrade
(with a priority specified by common vacuum-related GUCs)
4. Intentionally prohibit starting a new transaction with XID difference of
more than 2^32 from the oldest currently running one. This is to enforce
some dba's action for cleaning defunct transaction but not binding one:
he/she can wait if they consider these old transactions not defunct.
5. Investigate and add a solution for archs without 64-bit atomic values.
       5A. Provide XID 8-byte alignment for systems where 64-bit atomics is
provided for 8-byte aligned values.
       5B. Wrap XID reading into PG atomic locks for remaining 32-bit ones
(they are expected to be rare).

--
Best regards,
Pavel Borisov

Postgres Professional: http://postgrespro.com <http://www.postgrespro.com>