Re: BUG #14932: SELECT DISTINCT val FROM table gets stuck in an infinite loop
Andres Freund <andres@anarazel.de>
From: Andres Freund <andres@anarazel.de>
To: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Cc: "Todd A. Cook" <tcook@blackducksoftware.com>, Tom Lane <tgl@sss.pgh.pa.us>, Thomas Munro <thomas.munro@enterprisedb.com>, PostgreSQL Bugs <pgsql-bugs@postgresql.org>
Date: 2017-12-06T19:35:45Z
Lists: pgsql-bugs
On 2017-12-06 20:32:24 +0100, Tomas Vondra wrote: > On 12/05/2017 11:01 PM, Tomas Vondra wrote: > > > > Based on the initial discussion, the problem here is twofold. > > > > Firstly, the code assumes that if it gets certain number of bucket > > collisions (essentially, the initial bucket and certain number of > > neighboring buckets already being full), making the table larger is > > guaranteed to reduce the number of collisions. > > > > Which is obviously untrue for duplicate hash values, but it may also > > happen for distinct hash values that form a chain (think a sequence of > > hash values 1,2,3,4,5,6,7,...,K - that is never gonna get fixed). > I've managed to construct (by sheer brute force) an example that breaks > simplehash in this way. It triggers the growth by having a sequence of > distinct but consecutive hash values, e.g. 1, 2, 3, 4, ..., 1000, and > then trying to insert a value with duplicate hash (say, 4). I fail to be excited by this. That's possible for any sort of hashtable in some way. You might hit issues due to resizing or due to lookup performance, but you'll hit problem. That's a fairly fundamental issue of pure hashtables. Greetings, Andres Freund
Commits
-
Improve bit perturbation in TupleHashTableHash.
- d18d4bca81f8 10.2 landed
- c068f87723ca 11.0 landed
-
Prevent growth of simplehash tables when they're "too empty".
- d1aac2998789 10.2 landed
- ab9f2c429d8f 11.0 landed
-
Add stack-overflow guards in set-operation planning.
- 1b2a3860d3ea 10.2 cited