Re: Significant performance issues with array_agg() + HashAggregate plans on Postgres 17

Jeff Davis <pgsql@j-davis.com>

From: Jeff Davis <pgsql@j-davis.com>
To: Tom Lane <tgl@sss.pgh.pa.us>, Scott Carey <scott.carey@algonomy.com>
Cc: David Rowley <dgrowleyml@gmail.com>, pgsql-performance@lists.postgresql.org
Date: 2026-04-03T19:36:23Z
Lists: pgsql-performance
On Fri, 2026-04-03 at 15:24 -0400, Tom Lane wrote:
> After further study I no longer think there's a leak.  This
> test query involves 50000 GROUP BY groups, and we have an
> array being accumulated for each one.

I was coming to a similar conclusion, though trying to work through the
details of expanded arrays and how the datums are copied around during
aggregation. I don't (yet) see a problem with the correctness of the
memory handling there.

A lot of tiny memory contexts are not ideal, but as long as it's one
per group, that seems within reason.

> So really the whole thing is kind of unsatisfactory.
> I'm not seeing cheap ways to make it better though.
> 
> A very quick and dirty idea is to tell MemoryContextMemAllocated
> not to recurse, which would restore it to constant-time.  But
> that would make the results much less accurate in cases like
> this one.

One idea would be to update parent contexts' memory totals recursively
each time a subcontext allocates a new block. Block allocations are
infrequent enough that may be acceptable.

If we are worried about affecting unrelated cases, we could set an
"accounting_enabled" flag for the contexts we care about, which would
be automatically inherited by subcontexts, and then stop recursing up
when that flag is false.

Regards,
	Jeff Davis




Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →
  1. Disk-based Hash Aggregation.