Horton

Re: Reducing the chunk header sizes on all memory context types

David Rowley <dgrowleyml@gmail.com>

View thread View on postgr.es
From: David Rowley <dgrowleyml@gmail.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Andres Freund <andres@anarazel.de>, Robert Haas <robertmhaas@gmail.com>, Yura Sokolov <y.sokolov@postgrespro.ru>, PostgreSQL Developers <pgsql-hackers@lists.postgresql.org>
Date: 2022-09-07T12:29:22Z
Lists: pgsql-hackers 
On Tue, 6 Sept 2022 at 15:17, David Rowley <dgrowleyml@gmail.com> wrote:
>
> On Tue, 6 Sept 2022 at 14:43, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > I think MemoryContextContains' charter is to return
> >
> >         GetMemoryChunkContext(pointer) == context
> >
> > *except* that instead of asserting what GetMemoryChunkContext asserts,
> > it should treat those cases as reasons to return false.  So if you
> > can still do GetMemoryChunkContext then you can still do
> > MemoryContextContains.  The point of having the separate function
> > is to be as forgiving as we can of bogus pointers.
>
> Ok.  I've readded the Asserts that c6e0fe1f2 mistakenly removed from
> GetMemoryChunkContext() and changed MemoryContextContains() to do
> those same pre-checks before calling GetMemoryChunkContext().
>
> I've also boosted the Assert in mcxt.c to
> Assert(MemoryContextContains(context, ret)) in each place we call the
> context's callback function to obtain a newly allocated pointer.  I
> think this should cover the testing.
>
> I felt the need to keep the adjustments I made to the header comment
> in MemoryContextContains() to ward off anyone who thinks it's ok to
> pass this any random pointer and have it do something sane. It's much
> more prone to misbehaving/segfaulting now given the extra dereferences
> that c6e0fe1f2 added to obtain a pointer to the owning context.

I spent some time looking at our existing usages of
MemoryContextContains() to satisfy myself that we'll only ever pass in
a pointer to memory allocated by a MemoryContext and pushed this
patch.

I put some notes in the commit message about it being unsafe now to
pass in arbitrary pointers to MemoryContextContains(). Just a note to
the archives that I'd personally feel much better if we just removed
this function in favour of using GetMemoryChunkContext() instead. That
would force extension authors using MemoryContextContains() to rewrite
and revalidate their code. I feel that it's unlikely anyone will
notice until something crashes otherwise. Hopefully that'll happen
before their extension is released.

David

Commits

  1. Harden memory context allocators against bogus chunk pointers.

  2. Improve our ability to detect bogus pointers passed to pfree et al.

  3. Remove MemoryContextContains().

  4. Remove uses of MemoryContextContains in nodeAgg.c and nodeWindowAgg.c.

  5. Temporarily make MemoryContextContains return false

  6. Make MemoryContextContains work correctly again

  7. Make more effort to put a sentinel at the end of allocated memory

  8. Fix some possibly latent bugs in slab.c

  9. Various cleanups of the new memory context header code

  10. Revert "Add missing padding from MemoryChunk struct"

  11. Use MAXALIGN() in calculations using sizeof(SlabBlock)

  12. Add missing padding from MemoryChunk struct

  13. Improve performance of and reduce overheads of memory management