Re: Estimating HugePages Requirements?
Justin Pryzby <pryzby@telsasoft.com>
From: Justin Pryzby <pryzby@telsasoft.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: "Bossart, Nathan" <bossartn@amazon.com>, Andres Freund <andres@anarazel.de>, Magnus Hagander <magnus@hagander.net>, Mark Dilger <mark.dilger@enterprisedb.com>, Don Seiler <don@seiler.us>, pgsql-hackers@postgresql.org
Date: 2021-08-28T03:57:22Z
Lists: pgsql-hackers
On Sat, Aug 28, 2021 at 11:00:11AM +0900, Michael Paquier wrote: > On Fri, Aug 27, 2021 at 08:16:40PM +0000, Bossart, Nathan wrote: > > On 8/27/21, 12:39 PM, "Andres Freund" <andres@anarazel.de> wrote: > >> One thing I wonder is if this wouldn't better be dealt with in a more generic > >> way. While this is the most problematic runtime computed GUC, it's not the > >> only one. What if we introduced a new shared_memory_size GUC, and made > >> --describe-config output it? Perhaps adding --describe-config=guc-name? > >> > >> I also wonder if we should output the number of hugepages needed instead of > >> the "raw" bytes of shared memory. The whole business about figuring out the > >> huge page size, dividing the shared memory size by that and then rounding up > >> could be removed in that case. Due to huge_page_size it's not even immediately > >> obvious which huge page size one should use... > > > > I like both of these ideas. > > That pretty much looks like -C in concept, isn't it? Except that you > cannot get the actual total shared memory value because we'd do this > operation before loading shared_preload_libraries and miss any amount > asked by extensions. There is a problem similar when attempting to do > postgres -C data_checksums, for example, which would output an > incorrect value even if the cluster has data checksums enabled. Since we don't want to try to allocate the huge pages, and we also don't want to compute based on shared_buffers alone, did anyone consider if pg_controldata is the right place to put this ? It includes a lot of related stuff: max_connections setting: 100 max_worker_processes setting: 8 - (added in 2013: 6bc8ef0b7f1f1df3998745a66e1790e27424aa0c) max_wal_senders setting: 10 max_prepared_xacts setting: 2 max_locks_per_xact setting: 64 I'm not sure if there's any reason these aren't also shown (?) autovacuum_max_workers - added in 2007: e2a186b03 max_predicate_locks_per_xact - added in 2011: dafaa3efb max_logical_replication_workers max_replication_slots -- Justin
Commits
-
Silence extra logging when using "postgres -C" on runtime-computed GUCs
- 8bbf8461a3a2 15.0 landed
-
doc: Improve postgres command for shared_memory_size_in_huge_pages
- bbd4951b73ec 15.0 landed
-
Introduce GUC shared_memory_size_in_huge_pages
- 43c1c4f65eab 15.0 landed
-
Support "postgres -C" with runtime-computed GUCs
- 0c39c292077e 15.0 landed
-
Make shared_memory_size a preset option
- 3b231596ccfc 15.0 landed
-
Introduce GUC shared_memory_size
- bd1788051b02 15.0 landed
-
Move the shared memory size calculation to its own function
- 0bd305ee1d42 15.0 landed
-
Add new GUC, max_worker_processes, limiting number of bgworkers.
- 6bc8ef0b7f1f 9.4.0 cited