Horton

Re: Estimating HugePages Requirements?

Michael Paquier <michael@paquier.xyz>

View thread View on postgr.es
From: Michael Paquier <michael@paquier.xyz>
To: Magnus Hagander <magnus@hagander.net>
Cc: Nathan Bossart <nathandbossart@gmail.com>, "Bossart, Nathan" <bossartn@amazon.com>, Fujii Masao <masao.fujii@oss.nttdata.com>, Justin Pryzby <pryzby@telsasoft.com>, Andres Freund <andres@anarazel.de>, Mark Dilger <mark.dilger@enterprisedb.com>, Don Seiler <don@seiler.us>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2022-04-26T01:34:06Z
Lists: pgsql-hackers 
On Mon, Apr 25, 2022 at 04:55:25PM +0200, Magnus Hagander wrote:
> AIUI that was the original use-case for this feature. It certainly was for
> me :)

Perhaps we'd be fine with relaxing the requirements here knowing that
the control file should never be larger than PG_CONTROL_MAX_SAFE_SIZE
(aka the read should be atomic so it could be made lockless).  At the
end of the day, to be absolutely correct in the shmem size estimation,
I think that we are going to need what's proposed here or the sizing
may not be right depending on how extensions adjust GUCs after they
load their _PG_init():
https://www.postgresql.org/message-id/20220419154658.GA2487941@nathanxps13

That's a bit independent, but not completely unrelated either
depending on how exact you want your number of estimated huge pages to
be.  Just wanted to mention it.

>> Contrary to Linux, we don't need to care about the number of large
>> pages that are necessary because there is no equivalent of
>> vm.nr_hugepages on Windows (see [1]), do we?  If that were the case,
>> we'd have a use case for huge_page_size, additionally.
> 
> Right, for this one in particular -- that's what I meant with my comment
> about there not being a limit. But this feature works for other settings as
> well, not just the huge pages one.  Exactly what the use-cases are can
> vary, but surely they would have the same problems wrt redirects?

Yes, the redirection issue would apply to all the run-time GUCs.
--
Michael

Commits

  1. Silence extra logging when using "postgres -C" on runtime-computed GUCs

  2. doc: Improve postgres command for shared_memory_size_in_huge_pages

  3. Introduce GUC shared_memory_size_in_huge_pages

  4. Support "postgres -C" with runtime-computed GUCs

  5. Make shared_memory_size a preset option

  6. Introduce GUC shared_memory_size

  7. Move the shared memory size calculation to its own function

  8. Add new GUC, max_worker_processes, limiting number of bgworkers.