Thread

Re: BUG #19340: Wrong result from CORR() function

Tom Lane <tgl@sss.pgh.pa.us> — 2025-12-04T00:11:06Z

I wrote:
> Poking at this, I soon found a test case where even with the separate
> sqrt() calls we'd produce a result slightly outside [-1, 1] (running
> this test over more values of x is sufficient).  So now I think we
> should do both the separate sqrt and the clamp.

Per CI results, on some platforms the roundoff error is different from
what I observe, producing a value just less than 1 rather than just
more.  That doesn't invalidate needing the clamp, but it does mean
that we can't use that test case just like that.  I'm inclined to
remove the change of extra_float_digits, but keep the test case.

			regards, tom lane