Re: Estimating HugePages Requirements?
Nathan Bossart <bossartn@amazon.com>
From: "Bossart, Nathan" <bossartn@amazon.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: Justin Pryzby <pryzby@telsasoft.com>, Andres Freund <andres@anarazel.de>, Magnus Hagander <magnus@hagander.net>, Mark Dilger <mark.dilger@enterprisedb.com>, Don Seiler <don@seiler.us>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2021-09-03T17:36:43Z
Lists: pgsql-hackers
Attachments
- v5-0001-Move-the-shared-memory-size-calculation-to-its-ow.patch (application/octet-stream) patch v5-0001
- v5-0004-Introduce-huge_pages_required-GUC.patch (application/octet-stream) patch v5-0004
- v5-0003-Introduce-shared_memory_size-GUC.patch (application/octet-stream) patch v5-0003
- v5-0002-Move-postmaster-C-logic-to-after-processing-prelo.patch (application/octet-stream) patch v5-0002
On 9/2/21, 6:46 PM, "Michael Paquier" <michael@paquier.xyz> wrote: > On Thu, Sep 02, 2021 at 04:46:56PM +0000, Bossart, Nathan wrote: >> Yeah, huge_pages_required might not serve much purpose for Windows. >> We could always set it to -1 for Windows if it seems like it'll do >> more harm than good. > > I'd be fine with this setup on environments where there is no need for > it. I did this in v5. > One thing that would be incorrect upon more review is that we'd still > have data_checksums wrong with -C, meaning that the initial read of > the control file should be moved further up, and getting the control > file checks done before registering workers would be better. Keeping > the lock file at the end would be fine AFAIK, but should we worry > about the interactions with _PG_init() here? I think we can avoid so much reordering by moving the -C handling instead. That should also fix things like data_checksums. I split the reordering part out into its own patch in v5. You bring up an interesting point about _PG_init(). Presently, you can safely assume that the data directory is locked during _PG_init(), so there's no need to worry about breaking something on a running server. I don't know how important this is. Most _PG_init() functions that I've seen will define some GUCs, request some shared memory, register some background workers, and/or install some hooks. Those all seem like safe things to do, but I wouldn't be at all surprised to hear examples to the contrary. In any case, it looks like the current ordering of these two steps has been there for 15+ years. If this is a concern, one option would be to disallow running "-C shared_memory_size" on running servers. That would have to extend to GUCs like data_checksums and huge_pages_required, too. > 0001, that refactors the calculation of the shmem size into a > different routine, is fine as-is, so I'd like to move on and apply > it. Sounds good to me. Nathan
Commits
-
Silence extra logging when using "postgres -C" on runtime-computed GUCs
- 8bbf8461a3a2 15.0 landed
-
doc: Improve postgres command for shared_memory_size_in_huge_pages
- bbd4951b73ec 15.0 landed
-
Introduce GUC shared_memory_size_in_huge_pages
- 43c1c4f65eab 15.0 landed
-
Support "postgres -C" with runtime-computed GUCs
- 0c39c292077e 15.0 landed
-
Make shared_memory_size a preset option
- 3b231596ccfc 15.0 landed
-
Introduce GUC shared_memory_size
- bd1788051b02 15.0 landed
-
Move the shared memory size calculation to its own function
- 0bd305ee1d42 15.0 landed
-
Add new GUC, max_worker_processes, limiting number of bgworkers.
- 6bc8ef0b7f1f 9.4.0 cited