Fix behavior of float aggregates for single Inf or NaN inputs.
Tom Lane <tgl@sss.pgh.pa.us>
Fix behavior of float aggregates for single Inf or NaN inputs. When there is just one non-null input value, and it is infinity or NaN, aggregates such as stddev_pop and covar_pop should produce a NaN result, because the calculation is not well-defined. They used to do so, but since we adopted Youngs-Cramer aggregation in commit e954a727f, they produced zero instead. That's an oversight, so fix it. Add tests exercising these edge cases. Affected aggregates are var_pop(double precision) stddev_pop(double precision) var_pop(real) stddev_pop(real) regr_sxx(double precision,double precision) regr_syy(double precision,double precision) regr_sxy(double precision,double precision) regr_r2(double precision,double precision) regr_slope(double precision,double precision) regr_intercept(double precision,double precision) covar_pop(double precision,double precision) corr(double precision,double precision) Back-patch to v12 where the behavior change was accidentally introduced. Report and patch by me; thanks to Dean Rasheed for review. Discussion: https://postgr.es/m/353062.1591898766@sss.pgh.pa.us
Files
| Path | Change | +/− |
|---|---|---|
| src/backend/utils/adt/float.c | modified | +35 −0 |
| src/test/regress/expected/aggregates.out | modified | +104 −1 |
| src/test/regress/sql/aggregates.sql | modified | +20 −1 |
Discussion
- Definitional issue: stddev_pop (and related) for 1 input 4 messages · 2020-06-11 → 2020-06-13