Re: BUG #14932: SELECT DISTINCT val FROM table gets stuck in an infinite loop

Andres Freund <andres@anarazel.de>

From: Andres Freund <andres@anarazel.de>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Tomas Vondra <tomas.vondra@2ndquadrant.com>, "Todd A. Cook" <tcook@blackducksoftware.com>, Thomas Munro <thomas.munro@enterprisedb.com>, PostgreSQL Bugs <pgsql-bugs@postgresql.org>
Date: 2018-01-29T18:30:49Z
Lists: pgsql-bugs
On 2018-01-29 13:25:35 -0500, Tom Lane wrote:
> The first 2 seem OK to me by eyeball, though I've not done performance
> testing.

Thanks for the look!


> > The third patch is a version of the random IV discussed in this
> > thread. I do think we want to add usage of the extended hash functions,
> > as prototyped by Tomas, as that actually helps to fix issues with actual
> > hash conflicts. But we additionally need a fallback path for types
> > without extended hashtables, and the random IV is a good idea
> > nonetheless.  There's no ABI difference in my patch, so I think this is
> > actually something we could backpatch. But I don't think it's urgent, so
> > I'm not planning to do that for 10.2.  The one thing that could confuse
> > people is that it can lead to output order changes from run to run - I
> > think that's actually good, nobody should rely on hashagg etc output
> > being stable, but it might be a bit much in a stable release?
> 
> I disagree: people should reasonably expect the same query and same
> data and same plan to give consistent results.  When we stuck in the
> "synchronous seqscans" feature, which broke that property, we were very
> quickly forced by user complaints to provide a way to shut it off.
> I'm also concerned that we'd have to lobotomize a bunch of regression
> test cases to ensure stable results there.

I don't think the cases really compare. For one there's no "natural
order" like insertion order for seqscans (in simplistic cases). For
another hashaggs/joins/whatnot are *already* architecture dependent, so
there's ORDER BYs for pretty much everything already - I needed to add a
single ORDER BY to make things stable over ~50 regression runs.

Greetings,

Andres Freund


Commits

  1. Improve bit perturbation in TupleHashTableHash.

  2. Prevent growth of simplehash tables when they're "too empty".

  3. Add stack-overflow guards in set-operation planning.