Horton

Re: Estimating HugePages Requirements?

Nathan Bossart <bossartn@amazon.com>

View thread View on postgr.es
From: "Bossart, Nathan" <bossartn@amazon.com>
To: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Cc: "michael@paquier.xyz" <michael@paquier.xyz>, "pryzby@telsasoft.com" <pryzby@telsasoft.com>, "andres@anarazel.de" <andres@anarazel.de>, "magnus@hagander.net" <magnus@hagander.net>, "mark.dilger@enterprisedb.com" <mark.dilger@enterprisedb.com>, "don@seiler.us" <don@seiler.us>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2021-09-03T17:46:05Z
Lists: pgsql-hackers 
On 9/2/21, 10:12 PM, "Kyotaro Horiguchi" <horikyota.ntt@gmail.com> wrote:
> By the way I noticed that postgres -C huge_page_size shows 0, which I
> think should have the number used for the calculation if we show
> huge_page_required.

I would agree with this if huge_page_size was a runtime-computed GUC,
but since it's intended for users to explicitly request the huge page
size, it might be slightly confusing.  Perhaps another option would be
to create a new GUC for this.  Or maybe it's enough to note that the
value will be changed from 0 at runtime if huge pages are supported.
In any case, it might be best to handle this separately.

> I noticed that postgres -C shared_memory_size showed 137 (= 144703488)
> whereas the error message above showed 148897792 bytes (142MB). So it
> seems that something is forgotten while calculating
> shared_memory_size.  As the consequence, launching postgres setting
> huge_pages_required (69 pages) as vm.nr_hugepages ended up in the
> "could not map anonymous shared memory" error.

Hm.  I'm not seeing this with the v5 patch set, so maybe I
inadvertently fixed it already.  Can you check this again with v5?

Nathan

Commits

  1. Silence extra logging when using "postgres -C" on runtime-computed GUCs

  2. doc: Improve postgres command for shared_memory_size_in_huge_pages

  3. Introduce GUC shared_memory_size_in_huge_pages

  4. Support "postgres -C" with runtime-computed GUCs

  5. Make shared_memory_size a preset option

  6. Introduce GUC shared_memory_size

  7. Move the shared memory size calculation to its own function

  8. Add new GUC, max_worker_processes, limiting number of bgworkers.