Re: PG18 GIN parallel index build crash - invalid memory alloc request size

On Fri, Oct 24, 2025 at 8:38 AM Tomas Vondra <tomas@vondra.me> wrote:

> Hmmm, I wonder if the m_w_m is high enough to confuse the trimming logic
> in some way. Can you try if using smaller m_w_m (maybe 128MB-256MB)
> makes the issue go away?
>

The index builds at up to 4GB of m_w_m.  5GB and above crashes.

Now that I know roughly where the limits are that de-escalates things a
bit.  The sort of customers deploying a month after release should be OK
with just knowing to be careful about high m_w_m settings on PG18 until a
fix is ready.

Hope everyone is enjoying Latvia.  My obscure music collection includes a
band from there I used to see in the NYC area, The Quags;
https://www.youtube.com/watch?v=Bg3P4736CxM

Can you show the contents of buffer and tup? I'm especially interested
> in these fields:
>   buffer->nitems
>   buffer->maxitems
>   buffer->nfrozen
>   tup->nitems
>

I'll see if I can grab that data at the crash point.

FYI for anyone who wants to replicate this:  if you have a system with
128GB+ of RAM you could probably recreate the test case.   Just have to
download the Planet file and run osm2pgsql with the overly tweaked settings
I use.  I've published all the details of how I run this regression test
now.

Settings:  https://github.com/gregs1104/pgbent/tree/main/conf/18/conf.d
Script setup:  https://github.com/gregs1104/pgbent/blob/main/wl/osm-import
Test runner:
https://github.com/gregs1104/pgbent/blob/main/util/osm-importer
Parse results:
https://github.com/gregs1104/pgbent/blob/main/util/pgbench-init-parse


> If I'm right, I think there are two ways to fix this:
> (1) apply the trimming earlier, i.e. try to freeze + flush before
> actually merging the data (essentially, update nfrozen earlier)
> (2) use repalloc_huge (and palloc_huge) in GinBufferStoreTuple
> Or we probably should do both.


Sounds like (2) is probably mandatory and (1) is good hygiene.

--
Greg Smith, Software engineering
Snowflake - Where Data Does More
gregory.smith@snowflake.com