Avoid gratuitous inaccuracy in numeric width_bucket().

Tom Lane <tgl@sss.pgh.pa.us>

Commit: 7538708394e7a70105a4e601e253adf80f47cca8
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: 2020-10-08T17:06:27Z
Releases: 14.0
Avoid gratuitous inaccuracy in numeric width_bucket().

Multiply before dividing, not the reverse, so that cases that should
produce exact results do produce exact results.  (width_bucket_float8
got this right already.)  Even when the result is inexact, this avoids
making it more inexact, since only the division step introduces any
imprecision.

While at it, fix compute_bucket() to not uselessly repeat the sign
check already done by its caller, and avoid duplicating the
multiply/divide steps by adjusting variable usage.

Per complaint from Martin Visser.  Although this seems like a bug fix,
I'm hesitant to risk changing width_bucket()'s results in stable
branches, so no back-patch.

Discussion: https://postgr.es/m/6FA5117D-6AED-4656-8FEF-B74AC18FAD85@brytlyt.com

Files

Discussion