Re: Out of Memory errors are frustrating as heck!
Justin Pryzby <pryzby@telsasoft.com>
From: Justin Pryzby <pryzby@telsasoft.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Tomas Vondra <tomas.vondra@2ndquadrant.com>, Gunther <raj@gusw.net>, pgsql-performance@lists.postgresql.org, Jeff Janes <jeff.janes@gmail.com>, Alvaro Herrera <alvherre@2ndquadrant.com>
Date: 2019-04-20T21:45:41Z
Lists: pgsql-performance
On Sat, Apr 20, 2019 at 04:46:03PM -0400, Tom Lane wrote: > Tomas Vondra <tomas.vondra@2ndquadrant.com> writes: > > I think it's really a matter of underestimate, which convinces the planner > > to hash the larger table. In this case, the table is 42GB, so it's > > possible it actually works as expected. With work_mem = 4MB I've seen 32k > > batches, and that's not that far off, I'd day. Maybe there are more common > > values, but it does not seem like a very contrived data set. > > Maybe we just need to account for the per-batch buffers while estimating > the amount of memory used during planning. That would force this case > into a mergejoin instead, given that work_mem is set so small. Do you mean by adding disable_cost if work_mem is so small that it's estimated to be exceeded ? Justin
Commits
-
Consider BufFiles when adjusting hashjoin parameters
- a1b4f289beec 18.0 landed
-
Allocate hash join files in a separate memory context
- 8c4040edf456 16.0 landed