Fix assorted integer-overflow hazards in varbit.c.

Tom Lane <tgl@sss.pgh.pa.us>

Commit: 32fdf42cf546f613aab9ca98935c40a046187fa9
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: 2016-10-14T20:28:34Z
Releases: 10.0
Fix assorted integer-overflow hazards in varbit.c.

bitshiftright() and bitshiftleft() would recursively call each other
infinitely if the user passed INT_MIN for the shift amount, due to integer
overflow in negating the shift amount.  To fix, clamp to -VARBITMAXLEN.
That doesn't change the results since any shift distance larger than the
input bit string's length produces an all-zeroes result.

Also fix some places that seemed inadequately paranoid about input typmods
exceeding VARBITMAXLEN.  While a typmod accepted by anybit_typmodin() will
certainly be much less than that, at least some of these spots are
reachable with user-chosen integer values.

Andreas Seltenreich and Tom Lane

Discussion: <87d1j2zqtz.fsf@credativ.de>

Files

PathChange+/−
src/backend/utils/adt/varbit.c modified +15 −5