Re: Estimating HugePages Requirements?

Magnus Hagander <magnus@hagander.net>

From: Magnus Hagander <magnus@hagander.net>
To: Michael Paquier <michael@paquier.xyz>
Cc: Nathan Bossart <nathandbossart@gmail.com>, "Bossart, Nathan" <bossartn@amazon.com>, Fujii Masao <masao.fujii@oss.nttdata.com>, Justin Pryzby <pryzby@telsasoft.com>, Andres Freund <andres@anarazel.de>, Mark Dilger <mark.dilger@enterprisedb.com>, Don Seiler <don@seiler.us>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2022-03-15T22:02:37Z
Lists: pgsql-hackers
On Tue, Mar 15, 2022 at 3:41 AM Michael Paquier <michael@paquier.xyz> wrote:
>
> On Mon, Mar 14, 2022 at 10:34:17AM -0700, Nathan Bossart wrote:
> > On Mon, Mar 14, 2022 at 04:26:43PM +0100, Magnus Hagander wrote:
> >> And in fact, the command documented on
> >> https://www.postgresql.org/docs/devel/kernel-resources.html doesn't
> >> actually produce the output that the docs show, it also shows the log
> >> line, in the default config? If we can't fix the extra logging we
> >> should at least have our docs represent reality -- maybe by adding a
> >> "2>/dev/null" entry? But it'd be better to have it not output that log
> >> in the first place...
> >
> > I attached a patch to adjust the documentation for now.  This apparently
> > crossed my mind earlier [0], but I didn't follow through with it for some
> > reason.
>
> Another thing that we can add is -c log_min_messages=fatal, but my
> method is more complicated than a simple redirection, of course :)

Either does work, but yours has more characters :)


> >> (Of course what I'd really want is to be able to run it on a cluster
> >> that's running, but that was discussed downthread so I'm not bringing
> >> that one up for changes now)
> >
> > I think it is worth revisiting the extra logging and the ability to view
> > runtime-computed GUCs on a running server.  Should this be an open item for
> > v15, or do you think it's alright to leave it for the v16 development
> > cycle?
>
> Well, this is a completely new problem as it opens the door of
> potential concurrent access issues with the data directory lock file
> while reading values from the control file.  And that's not mandatory
> to be able to get those estimations without having to allocate a large
> chunk of memory, which was the primary goal discussed upthread as far
> as I recall.  So I would leave that as an item to potentially tackle
> in future versions.

I think we're talking about two different things here.

I think the "avoid extra logging" would be worth seeing if we can
address for 15.

The "able to run on a live cluster" seems a lot bigger and more scary
and not 15 material.


-- 
 Magnus Hagander
 Me: https://www.hagander.net/
 Work: https://www.redpill-linpro.com/



Commits

  1. Silence extra logging when using "postgres -C" on runtime-computed GUCs

  2. doc: Improve postgres command for shared_memory_size_in_huge_pages

  3. Introduce GUC shared_memory_size_in_huge_pages

  4. Support "postgres -C" with runtime-computed GUCs

  5. Make shared_memory_size a preset option

  6. Introduce GUC shared_memory_size

  7. Move the shared memory size calculation to its own function

  8. Add new GUC, max_worker_processes, limiting number of bgworkers.