Support allocating memory for large strings

Maxim Zibitsker <max.zibitsker@gmail.com>

From: Maxim Zibitsker <max.zibitsker@gmail.com>
To: pgsql-hackers@postgresql.org
Date: 2025-11-08T02:15:22Z
Lists: pgsql-hackers

Attachments

PostgreSQL's MaxAllocSize limit prevents storing individual variable-length character strings exceeding ~1GB, causing "invalid memory alloc request size" errors during INSERT operations on tables with large text columns. Example reproduction included in artifacts.md.

This limitation also affects pg_dump when exporting a PostgreSQL database with such data. The attached patches demonstrates a proof of concept using palloc_extended with MCXT_ALLOC_HUGE in the write path. For the read path, there are a couple of possible approaches: extending existing functions to handle huge allocations, or implementing a chunked storage mechanism that avoids single large allocations.

Thoughts?

Maxim