Horton

Re: Estimating HugePages Requirements?

Magnus Hagander <magnus@hagander.net>

View thread View on postgr.es
From: Magnus Hagander <magnus@hagander.net>
To: Julien Rouhaud <rjuju123@gmail.com>
Cc: Don Seiler <don@seiler.us>, pgsql-admin <pgsql-admin@postgresql.org>
Date: 2021-06-09T19:07:09Z
Lists: pgsql-hackers 
On Wed, Jun 9, 2021 at 7:23 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
>
> On Thu, Jun 10, 2021 at 12:42 AM Don Seiler <don@seiler.us> wrote:
> >
> > I'm told other factors contribute to this additional memory requirement, such as max_connections, wal_buffers, etc. I'm wondering if anyone has been able to come up with a reliable method for determining the HugePages requirements for a PG cluster based on the GUC values (that would be known at deployment time).
>
> It also depends on modules like pg_stat_statements and their own
> configuration.  I think that you can find the required size that your
> current configuration will allocate with:
>
> SELECT sum(allocated_size) FROM pg_shmem_allocations ;

I wonder how hard it would be to for example expose that through a
commandline switch or tool.

The point being that in order to run the query you suggest, the server
must already be running. There is no way to use this to estimate the
size that you're going to need after changing the value of
shared_buffers, which is a very common scenario. (You can change it,
restart without using huge pages because it fails, run that query,
change huge pages, and restart again -- but that's not exactly...
convenient)

-- 
 Magnus Hagander
 Me: https://www.hagander.net/
 Work: https://www.redpill-linpro.com/

Commits

  1. Silence extra logging when using "postgres -C" on runtime-computed GUCs

  2. doc: Improve postgres command for shared_memory_size_in_huge_pages

  3. Introduce GUC shared_memory_size_in_huge_pages

  4. Support "postgres -C" with runtime-computed GUCs

  5. Make shared_memory_size a preset option

  6. Introduce GUC shared_memory_size

  7. Move the shared memory size calculation to its own function

  8. Add new GUC, max_worker_processes, limiting number of bgworkers.