Horton

Re: BUG #15519: Casting float4 into int4 gets the wrong sign instead of "integer out of range" error

Andrew Gierth <andrew@tao11.riddles.org.uk>

View thread View on postgr.es
From: Andrew Gierth <andrew@tao11.riddles.org.uk>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: victor@magic.io, pgsql-bugs@lists.postgresql.org
Date: 2018-11-23T23:42:50Z
Lists: pgsql-bugs 
>>>>> "Tom" == Tom Lane <tgl@sss.pgh.pa.us> writes:

 Tom>     /*
 Tom>      * Range check.  We must be careful here that the boundary values are
 Tom>      * expressed exactly in the appropriate float domain; we assume float4
 Tom>      * is going to round off INT_MAX to a power of 2.  Also note assumption
 Tom>      * that rint() will pass through a NaN or Inf unchanged.
 Tom>      */
 Tom>      if (unlikely(num < (float4) INT_MIN || num >= (float4) INT_MAX || isnan(num)))
 Tom>         ereport(...);

Looking around, another approach I've seen (which I like better) is to
use

    if (num < (float4)INT_MIN || num >= -(float4)INT_MIN || ...

which doesn't rely on assumptions about how the compiler is going to
round INT_MAX. (It does rely on two's complement representation of ints,
but that assumption is known to be safe.)

-- 
Andrew (irc:RhodiumToad)

Commits

  1. Update additional float4/8 expected-output files.

  2. Fix float-to-integer coercions to handle edge cases correctly.

  3. Adjust new test case for more portability.