Horton

Re: POC: GROUP BY optimization

Dmitry Dolgov <9erthalion6@gmail.com>

View thread
From: Dmitry Dolgov <9erthalion6@gmail.com>
To: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Cc: Teodor Sigaev <teodor@sigaev.ru>, Gavin Flower <GavinFlower@archidevsys.co.nz>, Andres Freund <andres@anarazel.de>, Michael Paquier <michael@paquier.xyz>, PostgreSQL Developers <pgsql-hackers@lists.postgresql.org>
Date: 2020-05-16T12:24:31Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Restore preprocess_groupclause()

  2. Rename PathKeyInfo to GroupByOrdering

  3. Add invariants check to get_useful_group_keys_orderings()

  4. Fix asymmetry in setting EquivalenceClass.ec_sortref

  5. Multiple revisions to the GROUP BY reordering tests

  6. Get rid of pg_class usage in SJE regression tests

  7. Rename index "abc" in aggregates.sql

  8. Explore alternative orderings of group-by pathkeys during optimization.

  9. Generalize the common code of adding sort before processing of grouping

  10. Fix out-dated comment in preprocess_groupclause()

  11. Force parallelism in partition_aggregate

  12. Optimize order of GROUP BY keys

> On Fri, May 15, 2020 at 01:52:20AM +0200, Tomas Vondra wrote:
>
> I wonder if anyone has plans to try again with this optimization in v14
> cycle? The patches no longer apply thanks to the incremental sort patch,
> but I suppose fixing that should not be extremely hard.
>
> The 2020-07 CF is still a couple weeks away, but it'd be good to know if
> there are any plans to revive this. I'm willing to spend some time on
> reviewing / testing this, etc.

Yes, if you believe that this patch has potential, I would love to pick
it up again.

> I've only quickly skimmed the old thread, but IIRC there were two main
> challenges in getting the optimization right:
>
>
> 1) deciding which orderings are interesting / worth additional work
>
> I think we need to consider these orderings, in addition to the one
> specified in GROUP BY:
>
> 1) as specified in ORDER BY (if different from 1)

What is the idea behind considering this ordering?