Re: Fix overflow of nbatch

On 10/14/25 23:13, Tomas Vondra wrote:
> ...
> 
> I'll give this a bit more testing and review tomorrow, and then I'll
> push. I don't want to hold this back through pgconf.eu.
> 

Pushed and backpatched, after some minor tweaks. Thanks for the reviews
and feedback. I consider this is fixed now.


One remaining tweak I've been experimenting with (for master) is fixing
the weird behavior I described at the end of [1]. The root cause is that
we cap nbuckets by max_pointers, which is the max number of pointers we
can fit into work_mem. The consequence is that increasing work_mem also
increases nbatch too, which is counter-intuitive. It's a bit strange, as
it caps the number of batch pointers, while it ignores the buffers that
are ~1000x larger.

I experimented with capping the nbatch by how many pointers fit into
MaxAllocSize (and INT_MAX).

    Min(MaxAllocSize / sizeof(void *), INT_MAX / 2 + 1);

But I think this does not really matter much in practice. First, this
only happens with low work_mem values, while systems doing large joins
tend to have work_mem increased. Second, this means the nbatch is set
too low, and it'll get "fixed" by the memory balaning at runtime.

So I thinks we don't need to do anything about this.


[1]
https://www.postgresql.org/message-id/244dc6c1-3b3d-4de2-b3de-b1511e6a6d10%40vondra.me

-- 
Tomas Vondra