Fix assorted oversights in range selectivity estimation.
Tom Lane <tgl@sss.pgh.pa.us>
Fix assorted oversights in range selectivity estimation. calc_rangesel() failed outright when comparing range variables to empty constant ranges with < or >=, as a result of missing cases in a switch. It also produced a bogus estimate for > comparison to an empty range. On top of that, the >= and > cases were mislabeled throughout. For nonempty constant ranges, they managed to produce the right answers anyway as a result of counterbalancing typos. Also, default_range_selectivity() omitted cases for elem <@ range, range &< range, and range &> range, so that rather dubious defaults were applied for these operators. In passing, rearrange the code in rangesel() so that the elem <@ range case is handled in a less opaque fashion. Report and patch by Emre Hasegeli, some additional work by me
Files
| Path | Change | +/− |
|---|---|---|
| src/backend/utils/adt/rangetypes_selfuncs.c | modified | +34 −16 |
| src/include/catalog/pg_operator.h | modified | +2 −2 |
| src/test/regress/expected/rangetypes.out | modified | +32 −0 |
| src/test/regress/sql/rangetypes.sql | modified | +4 −0 |