Horton

[PATCH] Incremental sort (was: PoC: Partial sort)

Alexander Korotkov <a.korotkov@postgrespro.ru>

View thread View on postgr.es
From: Alexander Korotkov <a.korotkov@postgrespro.ru>
To: pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2017-02-18T10:31:29Z
Lists: pgsql-hackers

Attachments

Hi all!

I decided to start new thread for this patch for following two reasons.
 * It's renamed from "Partial sort" to "Incremental sort" per suggestion by
Robert Haas [1].  New name much better characterizes the essence of
algorithm.
 * I think it's not PoC anymore.  Patch received several rounds of review
and now it's in the pretty good shape.

Attached revision of patch has following changes.
 * According to review [1], two new path and plan nodes are responsible for
incremental sort: IncSortPath and IncSort which are inherited from SortPath
and Sort correspondingly.  That allowed to get rid of set of hacks with
minimal code changes.
 * According to review [1] and comment [2], previous tuple is stored in
standalone tuple slot of SortState rather than just HeapTuple.
 * New GUC parameter enable_incsort is introduced to control planner
ability to choose incremental sort.
 * Test of postgres_fdw with not pushed down cross join is corrected.  It
appeared that with incremental sort such query is profitable to push down.
I changed ORDER BY columns so that index couldn't be used.  I think this
solution is more elegant than setting enable_incsort = off.

Also patch has set of assorted code and comments improvements.

Links
1.
https://www.postgresql.org/message-id/CA+TgmoZapyHRm7NVyuyZ+yAV=U1a070BOgRe7PkgyrAegR4JDA@mail.gmail.com
2.
https://www.postgresql.org/message-id/CAM3SWZQL4yD2SnDheMCGL0Q2b2oTdKUvv_L6Zg_FcGoLuwMffg@mail.gmail.com

------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Commits

  1. Further adjustments to Hashagg EXPLAIN ANALYZE output

  2. Rework EXPLAIN format for incremental sort

  3. Fix typos and improve incremental sort comments

  4. Stabilize incremental_sort tests

  5. Minor improvements in Incremental Sort explain

  6. Consider Incremental Sort paths at additional places

  7. Fix representation of SORT_TYPE_STILL_IN_PROGRESS.

  8. Fix failures in incremental_sort due to number of workers

  9. Fix show_incremental_sort_info with force_parallel_mode

  10. Implement Incremental Sort

  11. Fix handling of "Subplans Removed" field in EXPLAIN output.

  12. Fix EXPLAIN (SETTINGS) to follow policy about when to print empty fields.

  13. Ensure plpgsql result tuples have the right composite type marking.

  14. Propagate sort instrumentation from workers back to leader.

  15. Make new regression test case parallel-safe, and improve its output.

  16. Push limit through subqueries to underlying sort, where possible.

  17. Fix inappropriate printing of never-measured times in EXPLAIN.

  18. Fix some infelicities in EXPLAIN output for parallel query plans.