Re: [PATCH] Fix old thinko in formula to compute sweight in numeric_sqrt().
Joel Jacobson <joel@compiler.org>
From: "Joel Jacobson" <joel@compiler.org>
To: "Dean Rasheed" <dean.a.rasheed@gmail.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2023-01-31T21:59:10Z
Lists: pgsql-hackers
Attachments
- fix-sweight-v4.patch (application/octet-stream) patch v4
- fix-sweight-v4-test-output.txt (text/plain)
On Tue, Jan 31, 2023, at 20:25, Dean Rasheed wrote: > That seems a bit wordy, given the context of this comment. I think > it's sufficient to just give the formula, and note that it simplifies > when DEC_DIGITS is even (not just 4): > > /* > * Assume the input was normalized, so arg.weight is accurate. The result > * then has at least sweight = floor(arg.weight * DEC_DIGITS / 2 + 1) > * digits before the decimal point. When DEC_DIGITS is even, we can save > * a few cycles, since the division is exact and there is no need to > * round down. > */ > #if DEC_DIGITS == ((DEC_DIGITS / 2) * 2) > sweight = arg.weight * DEC_DIGITS / 2 + 1; > #else > if (arg.weight >= 0) > sweight = arg.weight * DEC_DIGITS / 2 + 1; > else > sweight = 1 - (1 - arg.weight * DEC_DIGITS) / 2; > #endif Nice, you managed to simplify it even further. I think the comment and the code now are crystal clear together. I've tested it successfully, test report attached. In summary: DEC_DIGITS=1 now produce 16 sig. figs. in the range sqrt(2e-31) .. sqrt(2e+32), which before had a mix of 17 and 18 sig. figs in the result. DEC_DIGTIS=2 now produce 16 sig. figs. in the range sqrt(2e-31) .. sqrt(2e+31), which before always had 17 sig. figs in the result. DEC_DIGITS=4 is unchanged. Exact tested patch attached, code copy/pasted verbatim from your email. Test
Commits
-
Clarify the choice of rscale in numeric_sqrt().
- 0736fc1ceb06 16.0 landed
-
Ensure that numeric.c compiles with other NBASE values.
- 9a84f2947bf9 16.0 landed