Horton

Re: Memory Accounting

Jeff Davis <pgsql@j-davis.com>

View thread View on postgr.es
From: Jeff Davis <pgsql@j-davis.com>
To: Soumyadeep Chakraborty <sochakraborty@pivotal.io>, Tomas Vondra <tomas.vondra@2ndquadrant.com>, Robert Haas <robertmhaas@gmail.com>
Cc: Melanie Plageman <melanieplageman@gmail.com>, Pg Hackers <pgsql-hackers@postgresql.org>, soumyadeep2007@gmail.com
Date: 2019-09-19T18:00:37Z
Lists: pgsql-hackers 
On Wed, 2019-09-18 at 13:50 -0700, Soumyadeep Chakraborty wrote:
> Hi Jeff,

Hi Soumyadeep and Melanie,

Thank you for the review!

> max_stack_depth	max level	lazy (ms)	eager (ms)	(eage
> r/lazy)
> 2MB	82	302.715	427.554	1.4123978
> 3MB	3474	567.829	896.143	1.578191674
> 7.67MB	8694	2657.972	4903.063	1.844663149

Thank you for collecting data on this. Were you able to find any
regression when compared to no memory accounting at all?

It looks like you agree with the approach and the results. Did you find
any other issues with the patch?

I am also including Robert in this thread. He had some concerns the
last time around due to a small regression on POWER.

Regards,
	Jeff Davis

Commits

  1. Change MemoryContextMemAllocated to return Size

  2. Use Size instead of int64 to track allocated memory

  3. Add transparent block-level memory accounting

  4. Change the way pre-reading in external sort's merge phase works.

  5. Improve memory management for external sorts.

  6. In array_agg(), don't create a new context for every group.