Avoid overflow hazard when clamping group counts to "long int".
Tom Lane <tgl@sss.pgh.pa.us>
Avoid overflow hazard when clamping group counts to "long int". Several places in the planner tried to clamp a double value to fit in a "long" by doing (long) Min(x, (double) LONG_MAX); This is subtly incorrect, because it casts LONG_MAX to double and potentially back again. If long is 64 bits then the double value is inexact, and the platform might round it up to LONG_MAX+1 resulting in an overflow and an undesirably negative output. While it's not hard to rewrite the expression into a safe form, let's put it into a common function to reduce the risk of someone doing it wrong in future. In principle this is a bug fix, but since the problem could only manifest with group count estimates exceeding 2^63, it seems unlikely that anyone has actually hit this or will do so anytime soon. We're fixing it mainly to satisfy fuzzer-type tools. That being the case, a HEAD-only fix seems sufficient. Andrey Lepikhov Discussion: https://postgr.es/m/ebbc2efb-7ef9-bf2f-1ada-d6ec48f70e58@postgrespro.ru
Files
| Path | Change | +/− |
|---|---|---|
| src/backend/executor/nodeSubplan.c | modified | +2 −2 |
| src/backend/optimizer/path/costsize.c | modified | +27 −0 |
| src/backend/optimizer/plan/createplan.c | modified | +3 −4 |
| src/include/optimizer/optimizer.h | modified | +1 −0 |
Discussion
- Negative value of numGroups 5 messages · 2022-05-13 → 2022-05-21