Horton

Re: Memory Accounting

Jeff Davis <pgsql@j-davis.com>

View thread View on postgr.es
From: Jeff Davis <pgsql@j-davis.com>
To: Tomas Vondra <tomas.vondra@2ndquadrant.com>, Heikki Linnakangas <hlinnaka@iki.fi>
Cc: pgsql-hackers@postgresql.org
Date: 2019-07-22T16:33:03Z
Lists: pgsql-hackers 
On Mon, 2019-07-22 at 18:16 +0200, Tomas Vondra wrote:
> * I changed it to only update mem_allocated for the current
> > > context,
> > > not recursively for all parent contexts. It's now up to the
> > > function
> > > that reports memory usage to recurse or not (as needed).
> > 
> > Is that OK for memory bounded hash aggregation? Might there be a
> > lot 
> > of sub-contexts during hash aggregation?
> > 
> 
> There shouldn't be, at least not since b419865a814abbc. There might
> be
> cases where custom aggregates still do that, but I think that's
> simply a
> design we should discourage.

Right, I don't think memory-context-per-group is something we should
optimize for.

Discussion:
https://www.postgresql.org/message-id/3839201.Nfa2RvcheX%40techfox.foxi
https://www.postgresql.org/message-id/5334D7A5.2000907%40fuzzy.cz

Commit link:

https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=b419865a814abbca12bdd6eef6a3d5ed67f432e1

Regards,
	Jeff Davis

Commits

  1. Change MemoryContextMemAllocated to return Size

  2. Use Size instead of int64 to track allocated memory

  3. Add transparent block-level memory accounting

  4. Change the way pre-reading in external sort's merge phase works.

  5. Improve memory management for external sorts.

  6. In array_agg(), don't create a new context for every group.