Horton

Re: Memory Accounting

Tom Lane <tgl@sss.pgh.pa.us>

View thread View on postgr.es
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Jeff Davis <pgsql@j-davis.com>
Cc: Tomas Vondra <tomas.vondra@2ndquadrant.com>, Melanie Plageman <melanieplageman@gmail.com>, Soumyadeep Chakraborty <sochakraborty@pivotal.io>, Robert Haas <robertmhaas@gmail.com>, Pg Hackers <pgsql-hackers@postgresql.org>, soumyadeep2007@gmail.com
Date: 2019-10-04T14:43:18Z
Lists: pgsql-hackers 
Jeff Davis <pgsql@j-davis.com> writes:
> On Fri, 2019-10-04 at 10:26 +0200, Tomas Vondra wrote:
>> Yeah, I think that's an oversight. Maybe there's a reason why Jeff
>> used int64, but I can't think of any.

> I had chosen a 64-bit value to account for the situation Tom mentioned:
> that, in theory, Size might not be large enough to represent all
> allocations in a memory context. Apparently, that theoretical situation
> is not worth being concerned about.

Well, you could also argue it the other way: maybe in our children's
time, int64 won't be as wide as Size.  (Yeah, I know that sounds
ridiculous, but needing pointers wider than 32 bits was a ridiculous
idea too when I started in this business.)

The committed fix seems OK to me except that I think you should've
also changed MemoryContextMemAllocated() to return Size.

			regards, tom lane

Commits

  1. Change MemoryContextMemAllocated to return Size

  2. Use Size instead of int64 to track allocated memory

  3. Add transparent block-level memory accounting

  4. Change the way pre-reading in external sort's merge phase works.

  5. Improve memory management for external sorts.

  6. In array_agg(), don't create a new context for every group.