Re: Fix overflow of nbatch

Melanie Plageman <melanieplageman@gmail.com>

From: Melanie Plageman <melanieplageman@gmail.com>
To: Tomas Vondra <tomas@vondra.me>
Cc: Nathan Bossart <nathandbossart@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>, David Rowley <dgrowleyml@gmail.com>, Vaibhav Jain <jainva@google.com>, pgsql-hackers@postgresql.org, Madhukar <madhukarprasad@google.com>, Sangeetha Seshadri <sangsesh@google.com>
Date: 2025-10-08T19:16:06Z
Lists: pgsql-hackers
On Wed, Oct 8, 2025 at 1:37 PM Melanie Plageman
<melanieplageman@gmail.com> wrote:
>
> I have updated my patch to fix the mistakes above. I also noticed then
> that I wasn't doubling space_allowed in the loop but instead setting
> it to hash_table_bytes at the end. This doesn't produce a power of 2
> because we subtract skew_mcvs from the hash_table_bytes. So, we have
> to keep using space_allowed if we want a power of 2 in the end.
>
> I've changed my patch to do this, but this made me wonder if we want
> to be doing this or instead take hash_table_bytes at the end and round
> it up to a power of 2 and set space_allowed to that. If the skew
> hashtable is large, we may be allocating way more space_allowed than
> we need for new hash_table_bytes + skew hashtable buckets.

Oh wait, that doesn't make sense because each batch could have a skew hashtable.

- Melanie



Commits

  1. Fix hashjoin memory balancing logic