Re: BUG #16925: ERROR: invalid DSA memory alloc request size 1073741824 CONTEXT: parallel worker

On Sat, Mar 13, 2021 at 8:12 AM PG Bug reporting form
<noreply@postgresql.org> wrote:
> Using TPC-H benchmark scale-factor 1000.  To recreate dataset, see
> https://github.com/swarm64/s64da-benchmark-toolkit.  Use psql_native
> schema.

I'd like to reproduce this but it may take me some time.  Can you
please show the query plan?

> 2021-03-12 19:45:37.352 CET [316243] ERROR:  XX000: invalid DSA memory alloc
> request size 1073741824

So, this means we have a call to dsa_allocate_extended() without the
DSA_ALLOC_HUGE flag failing the sanity check that surely no one wants
a GB of memory at once.  In hash joins, we deliberately avoid making
our hash table bucket array long enough to hit that, since commit
86a2218e, and all other data is allocated in small chunks.  So the
only way to hit this would be with an individual tuple that takes 1GB
to store.  Other paths that use DSA include bitmap heap scans, but
they use the DSA_ALLOC_HUGE flag at least in one place.

>  max_parallel_workers                              | 1000
>  max_parallel_workers_per_gather                   | 52

>  work_mem                                          | 6291456

6GB  * 52 workers = 321GB.  I can see how we can get up to some
largish quantities of memory here, but I don't yet see how we try to
make an individual allocation of 1GB.