Re: Optimize planner memory consumption for huge arrays

On 19/2/2024 20:47, Tomas Vondra wrote:
> On 9/8/23 07:11, Lepikhov Andrei wrote:
>> Just for comparison, without partitioning:
>> elems	1		1E1		1E2		1E3		1E4	
>> master:	12kB	14kB	37kB	266kB	2.5MB
>> patched:	12kB	11.5kB	13kB	24kB	141kB
>>
> 
> These improvements look pretty nice, considering how simple the patch
> seems to be. I can't even imagine how much memory we'd need with even
> more partitions (say, 1000) if 100 partitions means 274MB.
> 
> BTW when releasing memory in scalararraysel, wouldn't it be good to also
> free the elem_values/elem_nulls? I haven't tried and maybe it's not that
> significant amount.
Agree. Added into the next version of the patch.
Moreover, I see a slight planning speedup. Looking into the reason for 
that, I discovered that it is because sometimes the planner utilizes the 
same memory piece for the next array element. It finds this piece more 
quickly than before that optimization.

-- 
regards,
Andrei Lepikhov
Postgres Professional