Horton

Re: Large writable variables

Andres Freund <andres@anarazel.de>

View thread View on postgr.es
From: Andres Freund <andres@anarazel.de>
To: Robert Haas <robertmhaas@gmail.com>
Cc: "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>, Thomas Munro <thomas.munro@enterprisedb.com>
Date: 2018-10-16T19:33:03Z
Lists: pgsql-hackers 
Hi,

On 2018-10-16 15:13:43 -0400, Robert Haas wrote:
> On Mon, Oct 15, 2018 at 4:08 PM Andres Freund <andres@anarazel.de> wrote:
> > So we have 500kb of not-initialized memory mapped into every
> > process. That's, uh, not nothing.
> 
> Thinking about this a bit more, why is this bad?  I mean, if the
> memory is never touched, the OS does not really need to allocate or
> zero any pages, or even make any page table entries.  If somebody
> actually accesses the data, then we'll take a page fault and have to
> really allocate, but otherwise I would think we could have 50MB of
> unused bss floating around and it wouldn't really matter, let alone
> 500kB.
> 
> What am I missing?

For one the OS will actually reserve all that memory when you use memory
overcommit = 2 (which you should). There's no need to reserve memory for
data that's guaranteed to be shared.  It also reduces the frequency of
page faults when looking at other global variables laid out nearby - we
could do something about that by giving the linker placement
instructions, but that's also work.  The size of the pagetable also has
performance effects, although it'll often be dwarfed by the shared
buffers entry if not using huge pages.  WRT .data entries, marking them
as const where appropriate both reduces memory usage, possibly allows
more compiler optimizations, and allows to catch bugs.

There's also the fact that not every OS has COW pagetables, or we use
processes in a way that don't allow that (say windows, where we don't
fork).

Greetings,

Andres Freund

Commits

  1. Apply unconstify() in more places

  2. Improve unconstify() documentation

  3. Drop const cast from dlsym() calls

  4. Const-ify a few more large static tables.

  5. Improve tzparse's handling of TZDEFRULES ("posixrules") zone data.

  6. Avoid statically allocating statement cache in ecpglib/prepare.c.

  7. Reorder FmgrBuiltin members, saving 25% in size.

  8. Add macro to cast away const without allowing changes to underlying type.

  9. Mark constantly allocated dest receiver as const.

  10. Avoid statically allocating formatting.c's format string caches.

  11. Correct constness of system attributes in heap.c & prerequisites.

  12. Avoid statically allocating gmtsub()'s timezone workspace.

  13. Correct constness of a few variables.

  14. Move the replication lag tracker into heap memory.