Re: BUG #14932: SELECT DISTINCT val FROM table gets stuck in an infinite loop
Todd A. Cook <tcook@blackducksoftware.com>
From: "Todd A. Cook" <tcook@blackducksoftware.com>
To: Andres Freund <andres@anarazel.de>, Tom Lane <tgl@sss.pgh.pa.us>
Cc: Tomas Vondra <tomas.vondra@2ndquadrant.com>, Thomas Munro <thomas.munro@enterprisedb.com>, PostgreSQL Bugs <pgsql-bugs@postgresql.org>
Date: 2018-01-29T18:20:25Z
Lists: pgsql-bugs
On 01/28/18 20:02, Andres Freund wrote: > On 2018-01-26 18:48:35 -0500, Tom Lane wrote: >> Andres Freund <andres@anarazel.de> writes: >>> On 2017-12-10 23:09:42 +0100, Tomas Vondra wrote: >>>> FWIW I do agree the data sets shared in this thread are pretty extreme >>>> and it doesn't make much sense to slow the regular cases. I'll be >>>> perfectly happy if we stop the OOM, making those cases fast is a bonus. >> >>> Yea, agreed on that. I'm kinda inclined to go for stop-growing in 10, >>> and so something better in 11. And then later possibly backpatch if >>> we've grown some confidence? >> >> +1. I'd like to see some response to this included in 10.2, and time >> grows short for that. > > Here are two patches that I think we want for 10.2, and the start of one > that I think we want for master. 0002 is needed because otherwise the > lack of extra growth leads to noticeably worse performance when filling > an underestimated a coordinator hash table from the workers - turns out > our hash combine (and most hash combines) let a lot of clustering > survive. By adding a final hashing round the bit perturbation is near > perfect. The commit messages need to be polished a bit, but other than > that I think these are reasonable fixes. Plan to push by Monday evening > at the latest. I applied 0001 and 0002 to REL_10_STABLE at 1b2a3860d3ea81825e9bbad2c7dbf66db87445c1 and got the following build error: execGrouping.c:26:29: fatal error: utils/hashutils.h: No such file or directory #include "utils/hashutils.h" My build sequence was git pull git status (showing clean tree) make distclean ./configure ... make -- todd
Commits
-
Improve bit perturbation in TupleHashTableHash.
- d18d4bca81f8 10.2 landed
- c068f87723ca 11.0 landed
-
Prevent growth of simplehash tables when they're "too empty".
- d1aac2998789 10.2 landed
- ab9f2c429d8f 11.0 landed
-
Add stack-overflow guards in set-operation planning.
- 1b2a3860d3ea 10.2 cited