Re: BUG #14932: SELECT DISTINCT val FROM table gets stuck in an infinite loop
Tom Lane <tgl@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>
To: "Todd A. Cook" <tcook@blackducksoftware.com>
Cc: Tomas Vondra <tomas.vondra@2ndquadrant.com>,
Thomas Munro <thomas.munro@enterprisedb.com>,
Andres Freund <andres@anarazel.de>,
PostgreSQL Bugs <pgsql-bugs@postgresql.org>
Date: 2017-12-06T17:14:24Z
Lists: pgsql-bugs
"Todd A. Cook" <tcook@blackducksoftware.com> writes: > If hashint8() is ultimately invoked by TupleHashTableHash() in execGroups.c, > it might be magnifying the difficulties here. The least significant bits, > which are used as the bucket number in simplehash.h, are not very well > distributed: > select val, to_hex(val), to_hex(hashint8(val)) from temp_f_03 limit 15 ; > val | to_hex | to_hex > ----------------------+------------------+---------- > 4444319256653758784 | 3dad64d121468140 | 805ffffe > 554179993563924608 | 7b0d7c49a018880 | 84dffffb > -3383965646518123872 | d109bd2c6b2982a0 | 9c3ffff7 > -4706811054739454944 | beae0c48915f8420 | 191ffff6 > 618200668902031424 | 8944a3ba5d08040 | 2a3ffff0 > 5074043922812601024 | 466aa01079f982c0 | 7effffee > -8783188184262212928 | 861bd8e1b9a482c0 | a6bfffea > -4597800992953433792 | c031545b6b128140 | b1dfffea > 8563040839807173408 | 76d608465dde8320 | 7d9fffe6 > 6092569112843158816 | 548d27540c888520 | 6f9fffe6 > -7313351060369079936 | 9a81c1f558f98180 | 68ffffe5 > -1786712428165627488 | e7345283536981a0 | 73ffffe5 > -6153596242570280896 | aa9a08d20e6b8040 | ac3fffd8 > 88426174078092128 | 13a272306c58360 | b57fffd8 > -5305589938458295680 | b65ec20faa4e8280 | ba9fffd3 Checking out hashint8() on random data shows no such obvious fault. You've apparently got a data set that exposes a weakness in hashint8, but it's not very clear what that is. In any case, the hashtable code needs to not fall over in the presence of a lot of collisions, regardless of the exact reason for there being a lot. regards, tom lane
Commits
-
Improve bit perturbation in TupleHashTableHash.
- d18d4bca81f8 10.2 landed
- c068f87723ca 11.0 landed
-
Prevent growth of simplehash tables when they're "too empty".
- d1aac2998789 10.2 landed
- ab9f2c429d8f 11.0 landed
-
Add stack-overflow guards in set-operation planning.
- 1b2a3860d3ea 10.2 cited