Re: Reference to - BUG #18349: ERROR: invalid DSA memory alloc request size 1811939328, CONTEXT: parallel worker

On 10/16/24 16:19, Thomas Munro wrote:
> On Mon, Oct 14, 2024 at 10:16 PM Andrei Lepikhov <lepihov@gmail.com> wrote:
>> See the attachment for a sketch of the solution.
> 
> Thanks Andrei, I mostly agree with your analysis, but I came up with a
> slightly different patch.  I think we should check for extreme skew if
> old_batch->space_exhausted (the parent partition).  Your sketch always
> does it for batch 0, which works for these examples but I don't think
> it's strictly correct: if batch 0 didn't run out of memory, it might
> falsely report extreme skew just because it had (say) 0 or 1 tuples.
Yeah, I misunderstood the meaning of the estimated_size variable. Your 
solution is more universal. Also, I confirm, it passes my synthetic  test.
Also, it raises the immediate question: What if we have too many 
duplicates? Sometimes, in user complaints, I see examples where they, 
analysing the database's logical consistency, pass through millions of 
duplicates to find an unexpected value. Do we need a top memory 
consumption limit here? I recall a thread in the mailing list with a 
general approach to limiting backend memory consumption, but it is 
finished with no result.
The patch looks good as well as commentary.

-- 
regards, Andrei Lepikhov