Horton

Re: Incorrect results from numeric round() and trunc()

Dean Rasheed <dean.a.rasheed@gmail.com>

View thread View on postgr.es
From: Dean Rasheed <dean.a.rasheed@gmail.com>
To: Joel Jacobson <joel@compiler.org>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2024-07-08T09:45:22Z
Lists: pgsql-hackers 
On Mon, 8 Jul 2024 at 00:40, Joel Jacobson <joel@compiler.org> wrote:
>
> On Sun, Jul 7, 2024, at 13:28, Dean Rasheed wrote:
> > I've also tidied up a bit by replacing all instances of SHRT_MAX with
> > a new constant NUMERIC_WEIGHT_MAX, whose name more accurately
> > describes the limit, as used in various other overflow checks.
>
> Having thought a bit more on this, I think we probably need a
> DEC_DIGITS sensitive definition of NUMERIC_WEIGHT_MAX,
> since per spec the max range for numeric is 0x20000 (131072)
> decimal digits.
>

No, the maximum weight is determined by the use of int16 to store the
weight. Therefore if you did reduce DEC_DIGITS to 1 or 2, the number
of decimal digits allowed before the decimal point would be reduced
too.

Regards,
Dean

Commits

  1. Fix scale clamping in numeric round() and trunc().