Horton

Re: [HACKERS] [PATCH] Incremental sort

Alexander Korotkov <a.korotkov@postgrespro.ru>

View thread View on postgr.es
From: Alexander Korotkov <a.korotkov@postgrespro.ru>
To: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Cc: Antonin Houska <ah@cybertec.at>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2018-03-20T16:33:42Z
Lists: pgsql-hackers

Attachments

Hi!

Revised patch is attached.  It's rebased to the last master.

On Fri, Mar 16, 2018 at 3:55 PM, Tomas Vondra <tomas.vondra@2ndquadrant.com>
wrote:

> On 03/16/2018 09:47 AM, Alexander Korotkov wrote:
> > On Fri, Mar 16, 2018 at 5:12 AM, Tomas Vondra
> > <tomas.vondra@2ndquadrant.com <mailto:tomas.vondra@2ndquadrant.com>>
> wrote:
> >
> >     I agree those don't seem like an issue in the Incremental Sort patch,
> >     but like a more generic costing problems.
> >
> >
> > Yes, I think so too.
>
> I wonder if we could make the costing a bit more pessimistic, to make
> these loses less likely, while still keeping the main wins (particularly
> for the LIMIT queries). But that seems a bit like a lost case, I guess.
>

Making costing more pessimistic makes sense.  Revised patch does
it in quite rough way: volumes of groups in incremental sort are multiplied
by 1.5.  That makes one query in regression tests to fallback to fullsort
from incremental sort.  Could you test it?  If this will shorten number of
cases where incremental sort causes regression, then it might be an
acceptable way to do more pessimistic costing.

> Do you think we can mark this patch RFC assuming that it have
> > already got pretty much of review previously.
> >
>
> Actually, I was going to propose to switch it to RFC, so I've just done
> that. I think the patch is clearly ready for a committer to take a
> closer look. I really like this improvement.
>
> I'm going to rerun the tests, but that's mostly because I'm interested
> if the change from i++ to i-- in cmpSortPresortedCols makes a measurable
> difference. I don't expect to find any issues, so why wait with the RFC?
>

Good, thanks.

------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Commits

  1. Further adjustments to Hashagg EXPLAIN ANALYZE output

  2. Rework EXPLAIN format for incremental sort

  3. Fix typos and improve incremental sort comments

  4. Stabilize incremental_sort tests

  5. Minor improvements in Incremental Sort explain

  6. Consider Incremental Sort paths at additional places

  7. Fix representation of SORT_TYPE_STILL_IN_PROGRESS.

  8. Fix failures in incremental_sort due to number of workers

  9. Fix show_incremental_sort_info with force_parallel_mode

  10. Implement Incremental Sort

  11. Fix handling of "Subplans Removed" field in EXPLAIN output.

  12. Fix EXPLAIN (SETTINGS) to follow policy about when to print empty fields.

  13. Ensure plpgsql result tuples have the right composite type marking.

  14. Propagate sort instrumentation from workers back to leader.

  15. Make new regression test case parallel-safe, and improve its output.

  16. Push limit through subqueries to underlying sort, where possible.

  17. Fix inappropriate printing of never-measured times in EXPLAIN.

  18. Fix some infelicities in EXPLAIN output for parallel query plans.