Horton

Re: [HACKERS] Partition-wise aggregation/grouping

Jeevan Chalke <jeevan.chalke@enterprisedb.com>

View thread View on postgr.es
From: Jeevan Chalke <jeevan.chalke@enterprisedb.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Ashutosh Bapat <ashutosh.bapat@enterprisedb.com>, Dilip Kumar <dilipbalaut@gmail.com>, David Rowley <david.rowley@2ndquadrant.com>, Rajkumar Raghuwanshi <rajkumar.raghuwanshi@enterprisedb.com>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Date: 2018-03-20T14:46:29Z
Lists: pgsql-hackers

Attachments

Hi,

On Mon, Mar 19, 2018 at 10:56 PM, Robert Haas <robertmhaas@gmail.com> wrote:

> On Fri, Mar 16, 2018 at 1:50 PM, Ashutosh Bapat
> <ashutosh.bapat@enterprisedb.com> wrote:
> > Ok. That looks good.
>
> Here's an updated version.  In this version, based on a voice
> discussion with Ashutosh and Jeevan, I adjusted 0001 to combine it
> with an earlier idea of splitting Gather/Gather Merge path generation
> out of the function that creates partially aggregated paths.  The idea
> here is that create_ordinary_gather_paths() could first call
> create_partial_grouping_paths(), then add additional paths which might
> be partial or non-partial by invoking the partition-wise aggregate
> logic, then call gather_grouping_paths() and set_cheapest() to
> finalize the partially grouped rel.  Also, I added draft commit
> messages.
>

I have added all these three patches in the attached patch-set and rebased
my changes over it.

However, I have not yet made this patch-set dependednt on UPPERREL_TLIST
changes you have proposed on another mail-thread and thus it has 0001 patch
refactoring the scanjoin issue.
0002, 0003 and 0004 are your patches added in this patchset.
0005 and 0006 are further refactoring patches. 0006 adds a
GroupPathExtraData which stores mostly child variant data and costs.
0007 is main partitionwise aggregation patch which is then rebased
accordingly.
0008 contains testcase and 0009 contains FDW changes.

Let me know if I missed any point to consider while rebasing.

-- 
Jeevan Chalke
Technical Architect, Product Development
EnterpriseDB Corporation
The Enterprise PostgreSQL Company

Commits

  1. postgres_fdw: Push down partition-wise aggregation.

  2. Remove 'target' from GroupPathExtraData.

  3. Implement partition-wise grouping/aggregation.

  4. Don't pass the grouping target around unnecessarily.

  5. Determine grouping strategies in create_grouping_paths.

  6. Defer creation of partially-grouped relation until it's needed.

  7. Split create_grouping_paths into degenerate and non-degenerate cases.

  8. Pass additional arguments to a couple of grouping-related functions.

  9. Fix logic error in add_paths_to_partial_grouping_rel.

  10. Minor cleanup of code related to partially_grouped_rel.

  11. Add a new upper planner relation for partially-aggregated results.

  12. Charge cpu_tuple_cost * 0.5 for Append and MergeAppend nodes.

  13. Rename enable_partition_wise_join to enable_partitionwise_join

  14. Factor some code out of create_grouping_paths.

  15. Pad XLogReaderState's main_data buffer more aggressively.

  16. Prevent int128 from requiring more than MAXALIGN alignment.

  17. Fix DROP SUBSCRIPTION hang

  18. Inject $(ICU_LIBS) regardless of platform.

  19. Some preliminary refactoring towards partitionwise join.