Re: Add 64-bit XIDs into PostgreSQL 15
Maxim Orlov <orlovmg@gmail.com>
From: Maxim Orlov <orlovmg@gmail.com>
To: Pavel Borisov <pashkin.elfe@gmail.com>
Cc: Justin Pryzby <pryzby@telsasoft.com>,
Aleksander Alekseev <aleksander@timescale.com>, Postgres hackers <pgsql-hackers@lists.postgresql.org>,
Stephen Frost <sfrost@snowman.net>, Alexander Korotkov <aekorotkov@gmail.com>,
Andres Freund <andres@anarazel.de>, Ilya Anfimov <ilan@tzirechnoy.com>
Date: 2022-07-06T12:55:02Z
Lists: pgsql-hackers
Commits
Same data as JSON:
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Add SLRU tests for 64-bit page case
- a60b8a58f435 17.0 landed
-
Make use FullTransactionId in 2PC filenames
- 5a1dfde8334b 17.0 landed
-
Use larger segment file names for pg_notify
- 2cdf131c46e6 17.0 landed
-
Index SLRUs by 64-bit integers rather than by 32-bit integers
- 4ed8f0913bfd 17.0 landed
Attachments
- v39-0001-Use-64-bit-numbering-of-SLRU-pages.patch (application/octet-stream) patch v39-0001
- v39-0005-Add-initdb-option-to-initialize-cluster-with-non.patch (application/octet-stream) patch v39-0005
- v39-0002-Use-64-bit-format-to-output-XIDs.patch (application/octet-stream) patch v39-0002
- v39-0004-Use-64-bit-pages-representation-in-SLRU-callers.patch (application/octet-stream) patch v39-0004
- v39-0007-Use-64-bit-GUCs.patch (application/octet-stream) patch v39-0007
- v39-0006-README.XID64.patch (application/octet-stream) patch v39-0006
- v39-0003-Use-64-bit-FullTransactionId-instead-of-Epoch-xi.patch (application/octet-stream) patch v39-0003
- v39-0008-Use-64-bit-XIDs.patch (application/octet-stream) patch v39-0008
Hi! Here is a new version of the patchset with following changes: - change unit test of page_conversion to address rare cfbot fails (the reason of cfbot to time out was the unit test not written accurate enough); - fix pg_upgarde on 32-bit systems; - switch to XidList type (introduced by f10a025cfe97c1a34) for logical replication, abandoning previously used Int64List type; - use cat version instead of major version as a boundary from 32 to 64 bit xids in pg_upgrade; - this cat version boundary temporary set to 999999999 for pg_upgrade testing purpose; - also rebased to the actual master branch. On 32-bit arch we have noticed pg_upgrade from 32 to 64 bit xids fails due to different TOAST_MAX_CHUNK_SIZE. On a 64-bit xid page we have less available space due to adding a heap page special. This leads to recalc of TOAST_MAX_CHUNK_SIZE. This was not a problem on 64 bit architectures, as padding bytes on 32-bit xids on TOAST pages were enough to accommodate heap page special of 64-bit xids with TOAST_MAX_CHUNK_SIZE unchanged. On 32 bits architectures padding bytes were not enough and this needed TOAST_MAX_CHUNK_SIZE to be of different size (on 64-bit xids version). Changes of TOAST_MAX_CHUNK_SIZE lead to being unable to pg_upgrade onto 64-bit xids. This was a real problem, since TOAST of relation requires all chunks to be the same size. In other words, we can not mix TOAST chunks of previous (32-bit xid TOAST pages) with the new one with 64-bit xid TOAST pages. The solution was to use different specials for TOAST and heap pages. Since, TOAST tuples can not have multixacts and does not need pd_multi_base on page. Thus, v39 is improved relative to v38 a lot. As always, feel free to review and share your thoughts on a subject. -- Best regards, Maxim Orlov.