Horton

Re: Reducing the chunk header sizes on all memory context types

Tomas Vondra <tomas.vondra@enterprisedb.com>

View thread View on postgr.es
From: Tomas Vondra <tomas.vondra@enterprisedb.com>
To: David Rowley <dgrowleyml@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Andres Freund <andres@anarazel.de>, Robert Haas <robertmhaas@gmail.com>, Yura Sokolov <y.sokolov@postgrespro.ru>, PostgreSQL Developers <pgsql-hackers@lists.postgresql.org>
Date: 2022-08-30T09:05:47Z
Lists: pgsql-hackers 

On 8/30/22 04:31, David Rowley wrote:
> On Tue, 30 Aug 2022 at 13:55, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> I wonder if slab ought to artificially bump up such requests when
>> MEMORY_CONTEXT_CHECKING is enabled, so there's room for a sentinel.
>> I think it's okay for aset.c to not do that, because its power-of-2
>> behavior means there usually is room for a sentinel; but slab's
>> policy makes it much more likely that there won't be.
> 
> I think it's fairly likely that small allocations are a power of 2,
> and I think most of our allocates are small, so I imagine that if we
> didn't do that for aset.c, we'd miss out on most of the benefits.
> 

Yeah. I think we have a fair number of "larger" allocations (once you
get to ~100B it probably won't be a 2^N), but we may easily miss whole
sections of allocations.

I guess the idea was to add a sentinel only when there already is space
for it, but perhaps that's a bad tradeoff limiting the benefits. Either
we add the sentinel fairly often (and then why not just add it all the
time - it'll need a bit more space), or we do it only very rarely (and
then it's a matter of luck if it catches an issue). Considering we only
do this with asserts, I doubt the extra bytes / CPU is a major issue,
and a (more) reliable detection of issues seems worth it. But maybe I
underestimate the costs. The only alternative seems to be valgrind, and
that's way costlier, though.

regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Commits

  1. Harden memory context allocators against bogus chunk pointers.

  2. Improve our ability to detect bogus pointers passed to pfree et al.

  3. Remove MemoryContextContains().

  4. Remove uses of MemoryContextContains in nodeAgg.c and nodeWindowAgg.c.

  5. Temporarily make MemoryContextContains return false

  6. Make MemoryContextContains work correctly again

  7. Make more effort to put a sentinel at the end of allocated memory

  8. Fix some possibly latent bugs in slab.c

  9. Various cleanups of the new memory context header code

  10. Revert "Add missing padding from MemoryChunk struct"

  11. Use MAXALIGN() in calculations using sizeof(SlabBlock)

  12. Add missing padding from MemoryChunk struct

  13. Improve performance of and reduce overheads of memory management