Horton

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

Tomas Vondra <tomas.vondra@2ndquadrant.com>

View thread View on postgr.es
From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: James Coleman <jtc331@gmail.com>, Alvaro Herrera <alvherre@2ndquadrant.com>, Michael Paquier <michael@paquier.xyz>, Rafia Sabih <rafia.pghackers@gmail.com>, Peter Geoghegan <pg@bowt.ie>, Simon Riggs <simon@2ndquadrant.com>, Shaun Thomas <shaun.thomas@2ndquadrant.com>, Dmitry Dolgov <9erthalion6@gmail.com>, Alexander Korotkov <a.korotkov@postgrespro.ru>, PostgreSQL Developers <pgsql-hackers@lists.postgresql.org>, Andreas Karlsson <andreas@proxel.se>
Date: 2020-04-07T03:09:15Z
Lists: pgsql-hackers 
On Mon, Apr 06, 2020 at 11:00:37PM -0400, Tom Lane wrote:
>Tomas Vondra <tomas.vondra@2ndquadrant.com> writes:
>> I came to the same conclusion (that the change in TuplesortMethod
>> definiton is the culprit) a while ago and was about to push a fix that
>> initialized it correctly in ExecSortInitializeDSM. But I agree reverting
>> it back to the old definition is probably better.
>
>Yeah, for the moment.  James would like to not have
>SORT_TYPE_STILL_IN_PROGRESS be part of the enum at all, I think,
>and I can see his point --- but then we need some out-of-band
>representation of "worker not done", so I'm not sure there'll be
>any net reduction of cruft.  Anyway that can be dealt with after
>we have a stable buildfarm.
>

Agreed.

>Note also that there's a separate comment-only patch in
><CAAaqYe9qzKbxCvSp3dfLkuS1v8KKnB7kW3z-hZ2jnAQaveSm8w@mail.gmail.com>
>that shouldn't be forgotten about.
>

OK, I'll take care of that tomorrow. I have two smaller patches to
commit in the incremental sort patchset, so I'll add it to that.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Commits

  1. Further adjustments to Hashagg EXPLAIN ANALYZE output

  2. Rework EXPLAIN format for incremental sort

  3. Fix typos and improve incremental sort comments

  4. Stabilize incremental_sort tests

  5. Minor improvements in Incremental Sort explain

  6. Consider Incremental Sort paths at additional places

  7. Fix representation of SORT_TYPE_STILL_IN_PROGRESS.

  8. Fix failures in incremental_sort due to number of workers

  9. Fix show_incremental_sort_info with force_parallel_mode

  10. Implement Incremental Sort

  11. Fix handling of "Subplans Removed" field in EXPLAIN output.

  12. Fix EXPLAIN (SETTINGS) to follow policy about when to print empty fields.

  13. Ensure plpgsql result tuples have the right composite type marking.

  14. Propagate sort instrumentation from workers back to leader.

  15. Make new regression test case parallel-safe, and improve its output.

  16. Push limit through subqueries to underlying sort, where possible.

  17. Fix inappropriate printing of never-measured times in EXPLAIN.

  18. Fix some infelicities in EXPLAIN output for parallel query plans.