Re: BUG in postgres mathematic
Tom Lane <tgl@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>
To: "Robert B. Easter" <reaster@comptechnews.com>
Cc: Max Vaschenko <max@nino.ru>, pgsql-bugs@postgresql.org, Thomas Lockhart <lockhart@alumni.caltech.edu>
Date: 2001-01-26T23:07:05Z
Lists: pgsql-bugs
>> The fact that 5*27.81*100 != 27.81*100*5 is certainly a garden-variety
>> floating-point roundoff error. However, I think Max has a fair
>> complaint here: it seems float-to-int8 conversion is truncating, not
>> rounding like the other conversions to integer do.
I have changed float8-to-int8 to start with an rint() call, the same as
float8-to-int4 and float8-to-int2. This should give the same roundoff
behavior as the other cases, including round-to-nearest-even if your
hardware supports IEEE-compliant float math.
Curiously, this change exposed what I take to be a platform dependency
in the int8 regress test. It was computing
int8(float8(4567890123456789::int8)) and expecting to get back exactly
4567890123456789. However, that value is 53 bits long and so there is
no margin for error in a standard IEEE float8 value. I find that at
least on HP hardware, rint() treats the value as inexact and rounds to
nearest even:
regression=# select round(4567890123456788::float8) - 4567890123456780::float8;
?column?
----------
8
(1 row)
regression=# select round(4567890123456789::float8) - 4567890123456780::float8;
?column?
----------
8
(1 row)
regression=# select round(4567890123456790::float8) - 4567890123456780::float8;
?column?
----------
10
(1 row)
regression=#
Whether this is a bug in rint or spec-compliant behavior is unclear, but
I'll bet HP's hardware is not the only platform that behaves this way.
Since I'm not eager to try to develop a new set of platform-specific
int8 expected files at this late hour, I just diked out that test
instead...
regards, tom lane