Clarify the choice of rscale in numeric_sqrt().
Dean Rasheed <dean.a.rasheed@gmail.com>
Clarify the choice of rscale in numeric_sqrt(). Improve the comment explaining the choice of rscale in numeric_sqrt(), and ensure that the code works consistently when other values of NBASE/DEC_DIGITS are used. Note that, in practice, we always expect DEC_DIGITS == 4, and this does not change the computation in that case. Joel Jacobson and Dean Rasheed Discussion: https://postgr.es/m/06712c29-98e9-43b3-98da-f234d81c6e49%40app.fastmail.com
Files
| Path | Change | +/− |
|---|---|---|
| src/backend/utils/adt/numeric.c | modified | +15 −2 |
Discussion
- [PATCH] Fix old thinko in formula to compute sweight in numeric_sqrt(). 8 messages · 2023-01-28 → 2023-02-02