Horton

Estimating HugePages Requirements?

Don Seiler <don@seiler.us>

View thread View on postgr.es
From: Don Seiler <don@seiler.us>
To: pgsql-admin <pgsql-admin@postgresql.org>
Date: 2021-06-09T16:41:52Z
Lists: pgsql-hackers 
Good day,

I'm trying to set up a chef recipe to reserve enough HugePages on a linux
system for our PG servers. A given VM will only host one PG cluster and
that will be the only thing on that host that uses HugePages. Blogs that
I've seen suggest that it would be as simple as taking the shared_buffers
setting and dividing that by 2MB (huge page size), however I found that I
needed some more.

In my test case, shared_buffers is set to 4003MB (calculated by chef) but
PG failed to start until I reserved a few hundred more MB. When I checked
VmPeak, it was 4321MB, so I ended up having to reserve over 2161 huge
pages, over a hundred more than I had originally thought.

I'm told other factors contribute to this additional memory requirement,
such as max_connections, wal_buffers, etc. I'm wondering if anyone has been
able to come up with a reliable method for determining the HugePages
requirements for a PG cluster based on the GUC values (that would be known
at deployment time).

Thanks,
Don.

-- 
Don Seiler
www.seiler.us

Commits

  1. Silence extra logging when using "postgres -C" on runtime-computed GUCs

  2. doc: Improve postgres command for shared_memory_size_in_huge_pages

  3. Introduce GUC shared_memory_size_in_huge_pages

  4. Support "postgres -C" with runtime-computed GUCs

  5. Make shared_memory_size a preset option

  6. Introduce GUC shared_memory_size

  7. Move the shared memory size calculation to its own function

  8. Add new GUC, max_worker_processes, limiting number of bgworkers.