Re: Poor memory context performance in large hash joins

Andres Freund <andres@anarazel.de>

From: Andres Freund <andres@anarazel.de>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Jeff Janes <jeff.janes@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2017-03-07T18:37:14Z
Lists: pgsql-hackers
On 2017-03-07 13:06:39 -0500, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> >>> On 2017-02-24 18:04:18 -0500, Tom Lane wrote:
> >>>> Concretely, something like the attached.  This passes regression tests
> >>>> but I've not pushed on it any harder than that.
> 
> > I think we should go forward with something like this patch in all
> > branches, and only use Tomas' patch in master, because they're
> > considerably larger.
> 
> While I'm willing to back-patch the proposed patch to some extent,
> I'm a bit hesitant to shove it into all branches, because I don't
> think that usage patterns that create a problem occurred till recently.
> You won't get a whole lot of standalone-block allocations unless you
> have code that allocates many chunks bigger than 8K without applying a
> double-the-request-each-time type of strategy.  And even then, it doesn't
> hurt unless you try to resize those chunks, or delete them in a non-LIFO
> order.
> 
> The hashjoin issue is certainly new, and the reorderbuffer issue can't
> go back further than 9.4.  So my inclination is to patch back to 9.4
> and call it good.

That works for me.  If we find further cases later, we can easily enough
backpatch it then.

Regards,

Andres


Commits

  1. Use doubly-linked block lists in aset.c to reduce large-chunk overhead.