Horton

Re: Shared hash table allocations

Heikki Linnakangas <hlinnaka@iki.fi>

View thread View on postgr.es
From: Heikki Linnakangas <hlinnaka@iki.fi>
To: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Cc: Tomas Vondra <tomas@vondra.me>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>, Robert Haas <robertmhaas@gmail.com>, Rahila Syed <rahilasyed90@gmail.com>
Date: 2026-04-02T16:44:56Z
Lists: pgsql-hackers 
On 02/04/2026 19:13, Ashutosh Bapat wrote:
> On Thu, Apr 2, 2026 at 7:44 PM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
>>
>>> I think we should highlight the change in default in the release notes
>>> though. The users which use default configuration will notice an
>>> increase in the memory. If they are using a custom value, they will
>>> think of bumping it up. Can we give them some ballpark % by which they
>>> should increase their max_locks_per_transaction? E.g. double the
>>> number or something?
>>
>> I don't think people who are using the defaults will notice. I'm worried
>> about the people who have set max_locks_per_transactions manually, and
>> now effectively get less lock space for the same setting. Yeah, doubling
>> the previous value is a good rule of thumb.
> 
> Users who have set max_locks_per_transaction to a non-default value
> but have tuned their application to respect those limits are safe even
> after this change, since their lock hash table never used wiggle room.
> Those users who weren't careful to respect those limits will need to
> bump their setting.

That's technically true, but in practice it's very hard for someone to 
carefully tune the setting. It's difficult to know how many locks a 
particular set of queries take. In practice what people do is they bump 
up the setting if the get the "out of shared memory" error, until the 
error goes away. If you do the tuning that way, it's quite possible that 
you are relying the "wiggle room" without realizing it.

> I think the release notes should "nudge" all the
> users who use non-default max_locks_per_transaction to increase it if
> they see "out of memory" errors. I don't think it should provide a
> blanket advise to double their locks

How about:

"If you had previously set max_locks_per_transaction, you might need to 
set it to a higher value in v19 to avoid "out of shared memory" errors. 
If you are unsure what to set it to and don't mind the increased memory 
usage, you can double the value to ensure that you can acquire at least 
as many locks as before"

TODO: do some more calculations and testing of how exactly the doubling 
rule works with different values. Is it guaranteed to be enough in all 
cases?

- Heikki

Commits

  1. Remove HASH_DIRSIZE, always use the default algorithm to select it

  2. Allocate all parts of shmem hash table from a single contiguous area

  3. Prevent shared memory hash tables from growing beyond initial size

  4. Merge init and max size options on shmem hash tables

  5. Change default of max_locks_per_transactions to 128

  6. Make the lock hash tables fixed-sized

  7. Remove 10% safety margin from lock manager hash table estimates

  8. Remove bogus "safety margin" from predicate.c shmem estimates

  9. Make ShmemIndex visible in the pg_shmem_allocations view

  10. Change the signature of dynahash's alloc function

  11. Remove HASH_SEGMENT option

  12. Use ShmemInitStruct to allocate shmem for semaphores