Re: Memory Accounting
Robert Haas <robertmhaas@gmail.com>
From: Robert Haas <robertmhaas@gmail.com>
To: Melanie Plageman <melanieplageman@gmail.com>
Cc: Jeff Davis <pgsql@j-davis.com>,
Soumyadeep Chakraborty <sochakraborty@pivotal.io>, Tomas Vondra <tomas.vondra@2ndquadrant.com>,
Pg Hackers <pgsql-hackers@postgresql.org>, soumyadeep2007@gmail.com
Date: 2019-10-04T20:11:35Z
Lists: pgsql-hackers
On Tue, Sep 24, 2019 at 2:47 PM Melanie Plageman <melanieplageman@gmail.com> wrote: > I think it would be helpful if we could repeat the performance tests > Robert did on that machine with the current patch (unless this version > of the patch is exactly the same as the ones he tested previously). I still have access to a POWER machine, but it's currently being used by somebody else for a benchmarking project, so I can't test this immediately. It's probably worth noting that, in addition to whatever's changed in this patch, tuplesort.c's memory management has been altered significantly since 2015 - see 0011c0091e886b874e485a46ff2c94222ffbf550 and e94568ecc10f2638e542ae34f2990b821bbf90ac. I'm not immediately sure how that would affect the REINDEX case that I tested back then, but it seems at least possible that they would have the effect of making palloc overhead less significant. More generally, so much of the sorting machinery has been overhauled by Peter Geoghegan since then that what happens now may just not be very comparable to what happened back then. I do agree that this approach looks pretty light weight. Tomas's point about the difference between updating only the current context and updating all parent contexts seems right on target to me. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
Commits
-
Change MemoryContextMemAllocated to return Size
- 36425ece5d6c 13.0 landed
-
Use Size instead of int64 to track allocated memory
- f2369bc610a1 13.0 landed
-
Add transparent block-level memory accounting
- 5dd7fc151946 13.0 landed
-
Change the way pre-reading in external sort's merge phase works.
- e94568ecc10f 10.0 cited
-
Improve memory management for external sorts.
- 0011c0091e88 9.6.0 cited
-
In array_agg(), don't create a new context for every group.
- b419865a814a 9.5.0 cited