Horton

Re: Estimating HugePages Requirements?

Kyotaro Horiguchi <horikyota.ntt@gmail.com>

View thread View on postgr.es
From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
To: bossartn@amazon.com
Cc: michael@paquier.xyz, pryzby@telsasoft.com, andres@anarazel.de, magnus@hagander.net, mark.dilger@enterprisedb.com, don@seiler.us, pgsql-hackers@postgresql.org
Date: 2021-09-07T06:24:21Z
Lists: pgsql-hackers 
At Fri, 3 Sep 2021 17:46:05 +0000, "Bossart, Nathan" <bossartn@amazon.com> wrote in 
> On 9/2/21, 10:12 PM, "Kyotaro Horiguchi" <horikyota.ntt@gmail.com> wrote:
> > By the way I noticed that postgres -C huge_page_size shows 0, which I
> > think should have the number used for the calculation if we show
> > huge_page_required.
> 
> I would agree with this if huge_page_size was a runtime-computed GUC,
> but since it's intended for users to explicitly request the huge page
> size, it might be slightly confusing.  Perhaps another option would be
> to create a new GUC for this.  Or maybe it's enough to note that the
> value will be changed from 0 at runtime if huge pages are supported.
> In any case, it might be best to handle this separately.

(Sorry, I was confused, but) yeah, agreed.

> > I noticed that postgres -C shared_memory_size showed 137 (= 144703488)
> > whereas the error message above showed 148897792 bytes (142MB). So it
> > seems that something is forgotten while calculating
> > shared_memory_size.  As the consequence, launching postgres setting
> > huge_pages_required (69 pages) as vm.nr_hugepages ended up in the
> > "could not map anonymous shared memory" error.
> 
> Hm.  I'm not seeing this with the v5 patch set, so maybe I
> inadvertently fixed it already.  Can you check this again with v5?

Thanks! I confirmed that the numbers match with v5.

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center

Commits

  1. Silence extra logging when using "postgres -C" on runtime-computed GUCs

  2. doc: Improve postgres command for shared_memory_size_in_huge_pages

  3. Introduce GUC shared_memory_size_in_huge_pages

  4. Support "postgres -C" with runtime-computed GUCs

  5. Make shared_memory_size a preset option

  6. Introduce GUC shared_memory_size

  7. Move the shared memory size calculation to its own function

  8. Add new GUC, max_worker_processes, limiting number of bgworkers.