Horton

Re: tuplesort memory usage: grow_memtuples

Tom Lane <tgl@sss.pgh.pa.us>

View thread
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Peter Geoghegan <peter@2ndquadrant.com>
Cc: Andres Freund <andres@2ndquadrant.com>, Jeff Janes <jeff.janes@gmail.com>, Robert Haas <robertmhaas@gmail.com>, Simon Riggs <simon@2ndquadrant.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, Greg S <stark@mit.edu>
Date: 2013-01-17T18:22:23Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Incorporate a couple of recent tuplesort.c improvements into tuplestore.c.

Peter Geoghegan <peter@2ndquadrant.com> writes:
> On 8 December 2012 14:41, Andres Freund <andres@2ndquadrant.com> wrote:
>> Is anybody planning to work on this? There hasn't been any activity
>> since the beginning of the CF and it doesn't look like there is much
>> work left?

> I took another look at this.

Applied with some changes:

* Use float8 arithmetic to prevent intermediate-result overflow, as I
suggested last night.

* Rearrange the subsequent checks so that they provide bulletproof
clamping behavior on out-of-range newmemtupsize values; this allows
dropping the very ad-hoc range limiting used in the previous patch (and
in what I had last night).

* Fix the check against availMem; it was supposed to be testing that the
increment in the array size was within availMem.  (In the original
coding the increment was always the same as the original size, but not
so much anymore.)

* Allow the array size to grow to the MaxAllocSize limit, instead of
punting if we'd exceed that.  If we're attempting to use as much of
work_mem as we possibly can, I don't see why that should be a reject
case.

* Improve the comments, including the existing one about the availMem
check, since that evidently wasn't clear enough.

* Copy the whole mess into tuplestore.c too.

			regards, tom lane