Re: Estimating HugePages Requirements?

Magnus Hagander <magnus@hagander.net>

From: Magnus Hagander <magnus@hagander.net>
To: Michael Paquier <michael@paquier.xyz>
Cc: Nathan Bossart <nathandbossart@gmail.com>, "Bossart, Nathan" <bossartn@amazon.com>, Fujii Masao <masao.fujii@oss.nttdata.com>, Justin Pryzby <pryzby@telsasoft.com>, Andres Freund <andres@anarazel.de>, Mark Dilger <mark.dilger@enterprisedb.com>, Don Seiler <don@seiler.us>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2022-04-22T07:49:34Z
Lists: pgsql-hackers
On Wed, Apr 20, 2022, 00:12 Michael Paquier <michael@paquier.xyz> wrote:

> On Thu, Mar 24, 2022 at 02:07:26PM +0100, Magnus Hagander wrote:
> > But neither would the suggestion of redirecting stderr to /dev/null.
> > In fact, doing the redirect it will *also* throw away any FATAL that
> > happens. In fact, using the 2>/dev/null method, we *also* remove the
> > message that says there's another postmaster running in this
> > directory, which is strictly worse than the override of
> > log_min_messages.
>
> Well, we could also tweak more the command with a redirection of
> stderr to a log file or such, and tell to look at it for errors.
>

That's would be a pretty terrible ux though.



> > That said, the redirect can be removed without recompiling postgres,
> > so it is probably still hte better choice as a temporary workaround.
> > But we should really look into getting a better solution in place once
> > we start on 16.
>
> But do we really need a better or more invasive solution for already
> running servers though?  A SHOW command would be able to do the work
> already in this case.  This would lack consistency compared to the
> offline case, but we are not without option either.  That leaves the
> case where the server is running, has allocated memory but is not
> ready to accept connections, like crash recovery, still this use case
> looks rather thin to me.



I agree that thats a very narrow use case. And I'm nog sure the use case of
a running server is even that important here - it's really the offline one
that's important. Or rather, the really compelling one is when there is a
server running but I want to check the value offline because it will
change. SHOW doesn't help there because it shows the value based on the
currently running configuration, not the new one after a restart.

I don't agree that the redirect is a solution. It's a workaround.


>> My solution for the docs is perhaps too confusing for the end-user,
> >> and we are talking about a Linux-only thing here anyway.  So, at the
> >> end, I am tempted to just add the "2> /dev/null" as suggested upthread
> >> by Nathan and call it a day.  Does that sound fine?
> >
> > What would be a linux only thing?
>
> Perhaps not at some point in the future.  Now that's under a section
> of the docs only for Linux.
>


Hmm. So what's the solution on windows? I guess maybe it's not as important
there because there is no limit on huge pages, but generally getting the
expected shared memory usage might be useful? Just significantly less
important.

/Magnus

Commits

  1. Silence extra logging when using "postgres -C" on runtime-computed GUCs

  2. doc: Improve postgres command for shared_memory_size_in_huge_pages

  3. Introduce GUC shared_memory_size_in_huge_pages

  4. Support "postgres -C" with runtime-computed GUCs

  5. Make shared_memory_size a preset option

  6. Introduce GUC shared_memory_size

  7. Move the shared memory size calculation to its own function

  8. Add new GUC, max_worker_processes, limiting number of bgworkers.