Re: Crash: invalid DSA memory alloc request

On Thu, 12 Dec 2024 at 22:28, Andreas 'ads' Scherbaum <ads@pgug.de> wrote:
>
>
> Hello,
>
> I'm running a couple of large tests, and in this particular test I have
> a few million tables more.
>
> At some point it fails, and I gathered the following trace:
>
>
> 2024-12-12 22:22:55.307 CET [1496210] ERROR: invalid DSA memory alloc
> request size 1073741824
> 2024-12-12 22:22:55.307 CET [1496210] BACKTRACE:
>          postgres: ads tabletest [local] CREATE TABLE(+0x15e570)
> [0x6309c379c570]
>          postgres: ads tabletest [local] CREATE
> TABLE(dshash_find_or_insert+0x1a4) [0x6309c39882d4]
>          postgres: ads tabletest [local] CREATE
> TABLE(pgstat_get_entry_ref+0x440) [0x6309c3b0a530]

It looks like the dshash table used in the pgstats system uses
resize(), which only specifies DSA_ALLOC_ZERO, not DSA_ALLOC_HUGE,
causing issues when the table grows larger than 1 GB.

I expect that error to disappear when you replace the
dsa_allocate0(...) call in dshash.c's resize function with
dsa_allocate_extended(..., DSA_ALLOC_HUGE | DSA_ALLOC_ZERO) as
attached, but haven't tested it due to a lack of database with
millions of relations.

Kind regards,

Matthias van de Meent