Re: Definitional issue: stddev_pop (and related) for 1 input
Tom Lane <tgl@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Dean Rasheed <dean.a.rasheed@gmail.com>
Cc: pgsql-hackers@lists.postgresql.org
Date: 2020-06-12T19:53:38Z
Lists: pgsql-hackers
Attachments
- 0001-add-single-input-test-cases.patch (text/x-diff) patch 0001
- 0002-fix-float-aggregates.patch (text/x-diff) patch 0002
- 0003-fix-numeric-aggregates.patch (text/x-diff) patch 0003
I wrote: > Before v12, stddev_pop() had the following behavior with just a > single input value: > ... > As of v12, though, all three cases produce 0. I am not sure what > to think about that with respect to an infinity input, but I'm > quite sure I don't like it for NaN input. While I'm still not sure whether there's an academic argument that zero is a reasonable stddev value for a single input that is Inf, it seems to me that backwards compatibility is a sufficient reason for going back to producing NaN for that. Hence, attached are some proposed patches. 0001 just adds test cases demonstrating the current behavior; then 0002 makes the proposed code change. It's easy to check that the test case results after 0002 match what v11 produces. 0003 deals with a different problem which I noted in [1]: the numeric variants of var_samp and stddev_samp also do the wrong thing for a single special input. Their disease is that they produce NaN for a single NaN input, where it seems more sensible to produce NULL. At least, NULL is what we get for the same case with the float aggregates, so we have to change one or the other set of functions if we want consistency. I propose back-patching 0001/0002 as far as v12, since the failure to match the old outputs seems like a pretty clear bug/regression. However, I'd be content to apply 0003 only to HEAD. That misbehavior is very ancient, and the lack of complaints suggests that few people really care about this fine point. regards, tom lane [1] https://www.postgresql.org/message-id/606717.1591924582%40sss.pgh.pa.us
Commits
-
Sync behavior of var_samp and stddev_samp for single NaN inputs.
- 23cbeda50b94 14.0 landed
-
Fix behavior of float aggregates for single Inf or NaN inputs.
- 33dd9bb3b0a8 13.0 landed
- 03109a53020e 14.0 landed
- 874372a941a6 12.4 landed