Re: Out of Memory errors are frustrating as heck!

Gunther <raj@gusw.net>

From: Gunther <raj@gusw.net>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Tomas Vondra <tomas.vondra@2ndquadrant.com>, Justin Pryzby <pryzby@telsasoft.com>, Jeff Janes <jeff.janes@gmail.com>, pgsql-performance@lists.postgresql.org, Alvaro Herrera <alvherre@2ndquadrant.com>
Date: 2019-08-24T15:40:09Z
Lists: pgsql-performance
Thanks Tom, yes I'd say it's using a lot of memory, but wouldn't call it 
"leak" as it doesn't grow during the 30 min or so that this query runs. 
It explodes to 4GB and then stays flat until done.

Yes, and this time the query is super complicated with many joins and 
tables involved. The query plan has 100 lines. Not easy to share for 
reproduce and I have my issue under control by adding some swap just in 
case. The swap space was never actually used.

thanks,
-Gunther

On 8/23/2019 10:20, Tom Lane wrote:
> Gunther <raj@gusw.net> writes:
>> Hi all, I am connecting to a discussion back from April this year. My
>> data has grown and now I am running into new out of memory situations.
> It doesn't look like this has much of anything to do with the hash-table
> discussion.  The big hog is an ExprContext:
>
>> ExprContext: 1107296256 total in 142 blocks; 6328 free (101 chunks);
>> 1107289928 used
> So there's something leaking in there, but this isn't enough info
> to guess what.
>
> 			regards, tom lane



Commits

  1. Consider BufFiles when adjusting hashjoin parameters

  2. Allocate hash join files in a separate memory context