Re: Support for 8-byte TOAST values (aka the TOAST infinite loop problem)
Michael Paquier <michael@paquier.xyz>
From: Michael Paquier <michael@paquier.xyz>
To: "Burd, Greg" <greg@burd.me>
Cc: Nikita Malakhov <hukutoc@gmail.com>, Hannu Krosing <hannuk@google.com>, Postgres hackers <pgsql-hackers@lists.postgresql.org>
Date: 2025-08-08T07:51:45Z
Lists: pgsql-hackers
Commits
Same data as JSON:
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Add data type oid8, 64-bit unsigned identifier
- b139bd3b6ef0 19 (unreleased) landed
Attachments
- v4-0001-Refactor-some-TOAST-value-ID-code-to-use-uint64-i.patch (text/x-diff) patch v4-0001
- v4-0002-Minimize-footprint-of-TOAST_MAX_CHUNK_SIZE-in-hea.patch (text/x-diff) patch v4-0002
- v4-0003-varatt_external-varatt_external_oid-and-VARTAG_ON.patch (text/x-diff) patch v4-0003
- v4-0004-Refactor-external-TOAST-pointer-code-for-better-p.patch (text/x-diff) patch v4-0004
- v4-0005-Move-static-inline-routines-of-varatt_external_oi.patch (text/x-diff) patch v4-0005
- v4-0006-Introduce-new-callback-to-get-fresh-TOAST-values.patch (text/x-diff) patch v4-0006
- v4-0007-Add-catcache-support-for-INT8OID.patch (text/x-diff) patch v4-0007
- v4-0008-Add-GUC-default_toast_type.patch (text/x-diff) patch v4-0008
- v4-0009-Introduce-global-64-bit-TOAST-ID-counter-in-contr.patch (text/x-diff) patch v4-0009
- v4-0010-Switch-pg_column_toast_chunk_id-return-value-from.patch (text/x-diff) patch v4-0010
- v4-0011-Add-support-for-bigint-TOAST-values.patch (text/x-diff) patch v4-0011
- v4-0012-Add-tests-for-TOAST-relations-with-bigint-as-valu.patch (text/x-diff) patch v4-0012
- v4-0013-Add-support-for-TOAST-table-types-in-pg_dump-and-.patch (text/x-diff) patch v4-0013
- v4-0014-Add-new-vartag_external-for-8-byte-TOAST-values.patch (text/x-diff) patch v4-0014
- v4-0015-amcheck-Add-test-cases-for-8-byte-TOAST-values.patch (text/x-diff) patch v4-0015
On Fri, Aug 01, 2025 at 06:03:11PM +0900, Michael Paquier wrote:
> Please find attached a v3, that I have spent some time polishing to
> fix the value ID problem of this thread. v2 had some conflicts, and
> the CI previously failed with warning job (CI is green here now).
Attached is a v4, due to conflicts mainly caused by the recent changes
in varatt.h done by e035863c9a04. This had an interesting side
benefit when rebasing, where I have been able to isolate most of the
knowledge related to the struct varatt_external (well
varatt_external_oid in the patch set) into toast_external.c, at the
exception of VARTAG_SIZE. That's done in a separate patch, numbered
0006.
The rest of the patch set has a couple of adjustements to document
better the new API expectations for toast_external.{c,h}, comment
adjustments, some more beautification changes, some indentation
applied, etc.
> As things stand, I am getting pretty happy with the patch set up to
> 0005 and how things are getting in shape for the interface, and I am
> planning to begin applying this stuff up to 0005 in the next couple of
> weeks.
As of this patch set, this means a new target of 0006, to get the
TOAST code refactored so as it is able to support more than 1 type of
external on-disk pointer with the 8-byte value problem in scope. Any
comments?
--
Michael