Re: Add 64-bit XIDs into PostgreSQL 15

Maxim Orlov <orlovmg@gmail.com>

From: Maxim Orlov <orlovmg@gmail.com>
To: Pavel Borisov <pashkin.elfe@gmail.com>
Cc: Justin Pryzby <pryzby@telsasoft.com>, Aleksander Alekseev <aleksander@timescale.com>, Postgres hackers <pgsql-hackers@lists.postgresql.org>, Stephen Frost <sfrost@snowman.net>, Alexander Korotkov <aekorotkov@gmail.com>, Andres Freund <andres@anarazel.de>, Ilya Anfimov <ilan@tzirechnoy.com>
Date: 2022-07-06T12:55:02Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →
  1. Add SLRU tests for 64-bit page case

  2. Make use FullTransactionId in 2PC filenames

  3. Use larger segment file names for pg_notify

  4. Index SLRUs by 64-bit integers rather than by 32-bit integers

Attachments

Hi!

Here is a new version of the patchset with following changes:
- change unit test of page_conversion to address rare cfbot fails (the
reason of cfbot to time out was the unit test not written accurate enough);
- fix pg_upgarde on 32-bit systems;
- switch to XidList type (introduced by f10a025cfe97c1a34) for logical
replication, abandoning previously used Int64List type;
- use cat version instead of major version as a boundary from 32 to 64 bit
xids in pg_upgrade;
- this cat version boundary temporary set to 999999999 for pg_upgrade
testing purpose;
- also rebased to the actual master branch.

On 32-bit arch we have noticed pg_upgrade from 32 to 64 bit xids fails due
to different TOAST_MAX_CHUNK_SIZE. On a 64-bit xid page we have less
available space due to adding a heap page special. This leads to recalc of
TOAST_MAX_CHUNK_SIZE.

This was not a problem on 64 bit architectures, as padding bytes on 32-bit
xids on TOAST pages were enough to accommodate heap page special of 64-bit
xids with TOAST_MAX_CHUNK_SIZE unchanged. On 32 bits architectures padding
bytes were not enough and this needed TOAST_MAX_CHUNK_SIZE to be of
different size (on 64-bit xids version).

Changes of TOAST_MAX_CHUNK_SIZE lead to being unable to pg_upgrade onto
64-bit xids. This was a real problem, since TOAST of relation requires all
chunks to be the same size. In other words, we can not mix TOAST chunks of
previous  (32-bit xid TOAST pages) with the new one with 64-bit xid TOAST
pages.

The solution was to use different specials for TOAST and heap pages. Since,
TOAST tuples can not have multixacts and does not need pd_multi_base on
page.

Thus, v39 is improved relative to v38 a lot.

As always, feel free to review and share your thoughts on a subject.

-- 
Best regards,
Maxim Orlov.