Re: Reference to - BUG #18349: ERROR: invalid DSA memory alloc request size 1811939328, CONTEXT: parallel worker
Tom Lane <tgl@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Andrei Lepikhov <lepihov@gmail.com>
Cc: Thomas Munro <thomas.munro@gmail.com>,
Craig Milhiser <craig@milhiser.com>,
pgsql-bugs@lists.postgresql.org
Date: 2024-10-14T06:26:09Z
Lists: pgsql-bugs
Andrei Lepikhov <lepihov@gmail.com> writes: > My explanation (correct if I'm wrong): > OUTER JOINs allow NULLs to be in a hash table. At the same time, a hash > value for NULL is 0, and it goes to the batch==0. > If batch number 0 gets overfilled, the > ExecParallelHashIncreaseNumBatches routine attempts to increase the > number of batches - but nothing happens. The initial batch is still too > big, and the number of batches doubles up to the limit. Interesting point. If memory serves (I'm too tired to actually look) the planner considers the statistical most-common-value when estimating whether an unsplittable hash bucket is likely to be too big. It does *not* think about null values ... but it ought to. However, this does not explain why PHJ would be more subject to the problem than non-parallel HJ. regards, tom lane
Commits
-
Fix extreme skew detection in Parallel Hash Join.
- 98c7c7152d2d 18.0 landed
- 4ac5d33a8b08 17.1 landed
- 53edc948580f 16.5 landed
- 1831545ca186 15.9 landed
- 20d94899414a 14.14 landed
- 45329466fdc9 13.17 landed
- 4fa80a6d7d4b 12.21 landed