Horton

Re: PoC: Partial sort

Marti Raudsepp <marti@juffo.org>

View thread View on postgr.es
From: Marti Raudsepp <marti@juffo.org>
To: Alexander Korotkov <aekorotkov@gmail.com>
Cc: Andreas Karlsson <andreas@proxel.se>, David Rowley <dgrowleyml@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, Andres Freund <andres@2ndquadrant.com>, Martijn van Oosterhout <kleptog@svana.org>, Jeremy Harris <jgh@wizmail.org>
Date: 2014-01-18T19:47:39Z
Lists: pgsql-hackers

Attachments

On Sat, Jan 18, 2014 at 7:22 PM, Marti Raudsepp <marti@juffo.org> wrote:
> Total runtime: 5.418 ms

Oops, shouldn't have rushed this. Clearly the timings should have
tipped me off that it's broken. I didn't notice that cmpSortSkipCols
was re-using tuplesort's sortkeys.

Here's a patch that actually works; I added a new skipKeys attribute
to SortState. I had to extract the SortSupport-creation code from
tuplesort_begin_heap to a new function; but that's fine, because it
was already duplicated in ExecInitMergeAppend too.

I reverted the addition of tuplesort_get_sortkeys, which is not needed now.

Now the timings are:
Unpatched partial sort: 13478.158 ms
Full sort: 6802.063 ms
Patched partial sort: 6618.962 ms

Regards,
Marti

Commits

  1. Implement Incremental Sort

  2. Improve memory management for external sorts.