Horton

Re: Estimating HugePages Requirements?

Michael Paquier <michael@paquier.xyz>

View thread View on postgr.es
From: Michael Paquier <michael@paquier.xyz>
To: "Bossart, Nathan" <bossartn@amazon.com>
Cc: Andres Freund <andres@anarazel.de>, Magnus Hagander <magnus@hagander.net>, Mark Dilger <mark.dilger@enterprisedb.com>, Don Seiler <don@seiler.us>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2021-08-28T02:00:11Z
Lists: pgsql-hackers 
On Fri, Aug 27, 2021 at 08:16:40PM +0000, Bossart, Nathan wrote:
> On 8/27/21, 12:39 PM, "Andres Freund" <andres@anarazel.de> wrote:
>> One thing I wonder is if this wouldn't better be dealt with in a more generic
>> way. While this is the most problematic runtime computed GUC, it's not the
>> only one. What if we introduced a new shared_memory_size GUC, and made
>> --describe-config output it? Perhaps adding --describe-config=guc-name?
>>
>> I also wonder if we should output the number of hugepages needed instead of
>> the "raw" bytes of shared memory. The whole business about figuring out the
>> huge page size, dividing the shared memory size by that and then rounding up
>> could be removed in that case. Due to huge_page_size it's not even immediately
>> obvious which huge page size one should use...
> 
> I like both of these ideas.

That pretty much looks like -C in concept, isn't it?  Except that you
cannot get the actual total shared memory value because we'd do this
operation before loading shared_preload_libraries and miss any amount
asked by extensions.  There is a problem similar when attempting to do
postgres -C data_checksums, for example, which would output an
incorrect value even if the cluster has data checksums enabled.
--
Michael

Commits

  1. Silence extra logging when using "postgres -C" on runtime-computed GUCs

  2. doc: Improve postgres command for shared_memory_size_in_huge_pages

  3. Introduce GUC shared_memory_size_in_huge_pages

  4. Support "postgres -C" with runtime-computed GUCs

  5. Make shared_memory_size a preset option

  6. Introduce GUC shared_memory_size

  7. Move the shared memory size calculation to its own function

  8. Add new GUC, max_worker_processes, limiting number of bgworkers.