Re: Use generation context to speed up tuplesorts

On Fri, Jul 30, 2021 at 2:42 AM David Rowley <dgrowleyml@gmail.com> wrote:
> Master:
>    Sort Method: quicksort  Memory: 5541kB
> Patched:
>    Sort Method: quicksort  Memory: 3197kB

Whoa.

> work_mem = '4GB';
> Test    master    gen sort    compare
> Test1    317.2    665.6    210%
> Test2    228.6    388.9    170%
> Test3    207.4    330.7    159%
> Test4    185.5    279.4    151%
> Test5    292.2    563.9    193%

Very impressive.

An early version of what eventually became DSA worked with
backend-local memory and I saw very substantial memory usage
improvements on large sorts, similar to what you show here. I am not
sure I saw the same CPU improvements, and in any case I abandoned the
idea of using that infrastructure to manage backend-local memory at
some point, since the whole thing had lots of problems that I didn't
know how to solve. What you've done here looks like a much more
promising approach.

-- 
Robert Haas
EDB: http://www.enterprisedb.com