Horton

Re: Large writable variables

Andres Freund <andres@anarazel.de>

View thread View on postgr.es
From: Andres Freund <andres@anarazel.de>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: pgsql-hackers@postgresql.org, Thomas Munro <thomas.munro@enterprisedb.com>
Date: 2018-10-15T20:45:57Z
Lists: pgsql-hackers 
Hi,

On 2018-10-15 16:36:26 -0400, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > So we have 500kb of not-initialized memory mapped into every
> > process. That's, uh, not nothing.
> 
> Bleah.

Yea...

> > 0000000008585088 0000000000131104 b hist_entries
> > 0000000008716192 0000000000016384 b hist_start
> 
> I'm unsure what fraction of processes would have use for these.

Yea, I'm not sure either. Although I suspect that given the cost of
compression having an "allocate on first use" check should be quite
doable.


> > 0000000008435040 0000000000085280 b DCHCache
> > 0000000008391168 0000000000043840 b NUMCache
> 
> We could surely allocate these on first use.

yep.


> > 0000000008560224 0000000000023440 b tzdefrules_s
> > 0000000008536704 0000000000023440 b gmtmem.7009
> 
> I think that tzdefrules_s is not used in common cases (though I could be
> wrong about that), so we could win by alloc-on-first-use.  The same might
> be true for gmtmem, but there's a sticking point: there is no provision
> for failure there, so I'm unsure how we avoid crashing on OOM.

I guess we could return false / NULL to the caller. Not perfect, but
there's not that many callers. We could wrap them in wrappers that throw
errors...


> > 0000000008238336 0000000000008192 b PqRecvBuffer
> > 0000000008734208 0000000000005136 B BackendWritebackContext
> > 0000000008386368 0000000000003200 b held_lwlocks
> 
> These are below my personal threshold of pain.

Yea, agreed. PqRecvBuffer and held_lwlocks are common enough that it
makes sense to pre-allocate them anyway. I guess you could argue that
BackendWritebackContext should be dynamically allocated.


> > I'm unclear as to why ScanKeywords, DCH_keywords aren't in a readonly
> > section.
> 
> I think it's the same problem: pointers can't be truly const because
> they have to be changed if one relocates the executable.

Right. It's, as I noticed when looking via objdupm, in .data.rel.ro, so
I think it's not that bad.


> We could possibly fix these by changing the data structure so that
> what's in a ScanKeywords entry is an offset into some giant string
> constant somewhere.  No idea how that would affect performance, but
> I do notice that we could reduce the sizeof(ScanKeyword), which can't
> hurt.

Yea, that might even help performancewise. Alternatively we could change
ScanKeyword to store the keyword name inline, but that'd be a measurable
size increase...

Greetings,

Andres Freund

Commits

  1. Apply unconstify() in more places

  2. Improve unconstify() documentation

  3. Drop const cast from dlsym() calls

  4. Const-ify a few more large static tables.

  5. Improve tzparse's handling of TZDEFRULES ("posixrules") zone data.

  6. Avoid statically allocating statement cache in ecpglib/prepare.c.

  7. Reorder FmgrBuiltin members, saving 25% in size.

  8. Add macro to cast away const without allowing changes to underlying type.

  9. Mark constantly allocated dest receiver as const.

  10. Avoid statically allocating formatting.c's format string caches.

  11. Correct constness of system attributes in heap.c & prerequisites.

  12. Avoid statically allocating gmtsub()'s timezone workspace.

  13. Correct constness of a few variables.

  14. Move the replication lag tracker into heap memory.