Horton

Large writable variables

Andres Freund <andres@anarazel.de>

View thread View on postgr.es
From: Andres Freund <andres@anarazel.de>
To: pgsql-hackers@postgresql.org, Thomas Munro <thomas.munro@enterprisedb.com>
Date: 2018-10-15T20:07:54Z
Lists: pgsql-hackers 
Hi,

while briefly thinking about the per-process overhead of postgres, I
looked at the section sizes of a modern postgres. In particular which
memory areas are *not* shared between processes.

If you look at the section sizes that are mapped read-write:

$ size --format=sysv src/backend/postgres
src/backend/postgres  :
section                  size      addr
.rodata               1593617   5730304 (read-only, for reference)
.data.rel.ro           134944   8039904 (read only after start)
.data                   53888   8178912 (read-write, initialized)
.bss                   510416   8232800 (read-write, uninitialized)
Total                52417197

So we have 500kb of not-initialized memory mapped into every
process. That's, uh, not nothing.

top unitialized allocations:
$ nm -t d --size-sort -r -S src/backend/postgres|grep '\b[bB]\b'|head
0000000008251872 0000000000131144 b LagTracker
0000000008585088 0000000000131104 b hist_entries
0000000008435040 0000000000085280 b DCHCache
0000000008391168 0000000000043840 b NUMCache
0000000008560224 0000000000023440 b tzdefrules_s
0000000008536704 0000000000023440 b gmtmem.7009
0000000008716192 0000000000016384 b hist_start
0000000008238336 0000000000008192 b PqRecvBuffer
0000000008734208 0000000000005136 B BackendWritebackContext
0000000008386368 0000000000003200 b held_lwlocks

So we have a two variables sized 130kb. Yikes.

struct {
        XLogRecPtr                 last_lsn;             /*     0     8 */
        WalTimeSample              buffer[8192];         /*     8 131072 */
        /* --- cacheline 2048 boundary (131072 bytes) was 8 bytes ago --- */
        int                        write_head;           /* 131080     4 */
        int                        read_heads[3];        /* 131084    12 */
        WalTimeSample              last_read[3];         /* 131096    48 */
        /* --- cacheline 2049 boundary (131136 bytes) was 8 bytes ago --- */

        /* size: 131144, cachelines: 2050, members: 5 */
        /* last cacheline: 8 bytes */
};

that's not actually used very often, nor in all processes... Thomas?

hist_entries is pg_lzcompress; DCHCache,NUMCache are formatting.c.

top itialized allocations:
$ nm -t d --size-sort -r -S src/backend/postgres|grep '\b[dD]\b'|head
0000000008086944 0000000000087904 D fmgr_builtins
0000000008201120 0000000000017280 d ConfigureNamesInt
0000000008218400 0000000000013680 d ConfigureNamesBool
0000000008189248 0000000000008512 d ConfigureNamesString
0000000008077344 0000000000007040 D ScanKeywords
0000000008184928 0000000000004320 d ConfigureNamesEnum
0000000008197760 0000000000003360 d ConfigureNamesReal
0000000008062976 0000000000002304 d DCH_keywords
0000000008069952 0000000000002016 D pg_wchar_table
0000000008075552 0000000000001776 d encoding_match_list

fmgr_builtins isn't readonly even though declared a const - I assume
because it's full of addresses that will be mapped differently from
execution to execution.

ConfigureNames* isn't marked as const, because we update them:
		/* Rather than requiring vartype to be filled in by hand, do this: */
		conf->gen.vartype = PGC_BOOL;

I'm unclear as to why ScanKeywords, DCH_keywords aren't in a readonly
section.

Greetings,

Andres Freund

Commits

  1. Apply unconstify() in more places

  2. Improve unconstify() documentation

  3. Drop const cast from dlsym() calls

  4. Const-ify a few more large static tables.

  5. Improve tzparse's handling of TZDEFRULES ("posixrules") zone data.

  6. Avoid statically allocating statement cache in ecpglib/prepare.c.

  7. Reorder FmgrBuiltin members, saving 25% in size.

  8. Add macro to cast away const without allowing changes to underlying type.

  9. Mark constantly allocated dest receiver as const.

  10. Avoid statically allocating formatting.c's format string caches.

  11. Correct constness of system attributes in heap.c & prerequisites.

  12. Avoid statically allocating gmtsub()'s timezone workspace.

  13. Correct constness of a few variables.

  14. Move the replication lag tracker into heap memory.