Support allocating memory for large strings
Maxim Zibitsker <max.zibitsker@gmail.com>
From: Maxim Zibitsker <max.zibitsker@gmail.com>
To: pgsql-hackers@postgresql.org
Date: 2025-11-08T02:15:22Z
Lists: pgsql-hackers
Attachments
- 0001-Support-allocating-memory-for-large-strings.patch (application/octet-stream) patch 0001
- (unnamed) (text/plain)
- artifacts.md (text/markdown)
PostgreSQL's MaxAllocSize limit prevents storing individual variable-length character strings exceeding ~1GB, causing "invalid memory alloc request size" errors during INSERT operations on tables with large text columns. Example reproduction included in artifacts.md. This limitation also affects pg_dump when exporting a PostgreSQL database with such data. The attached patches demonstrates a proof of concept using palloc_extended with MCXT_ALLOC_HUGE in the write path. For the read path, there are a couple of possible approaches: extending existing functions to handle huge allocations, or implementing a chunked storage mechanism that avoids single large allocations. Thoughts? Maxim