Horton

Re: Negative value of numGroups

Tom Lane <tgl@sss.pgh.pa.us>

View thread View on postgr.es
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Andrey Lepikhov <a.lepikhov@postgrespro.ru>
Cc: pgsql-bugs@lists.postgresql.org
Date: 2022-05-17T15:34:17Z
Lists: pgsql-bugs 
Andrey Lepikhov <a.lepikhov@postgrespro.ru> writes:
> On 13/5/2022 11:56, Andrey Lepikhov wrote:
>> (long) Min(numGroups, (double) LONG_MAX);
>> 
>> can return negative value, if numGroups > LONG_MAX.

I see your point, but I don't think that repeating the same finicky
and undocumented coding pattern in multiple places is a future-proof
fix.  I'm inclined to think we should invent a function along the
lines of "long clamp_double_to_long(double x)".  Given where it's
used, maybe putting it beside clamp_row_est() would be good.

			regards, tom lane

Commits

  1. Avoid overflow hazard when clamping group counts to "long int".