Horton

Re: Partition-wise join for join between (declaratively) partitioned tables

Robert Haas <robertmhaas@gmail.com>

View thread View on postgr.es
From: Robert Haas <robertmhaas@gmail.com>
To: Ashutosh Bapat <ashutosh.bapat@enterprisedb.com>
Cc: Thomas Munro <thomas.munro@enterprisedb.com>, Rafia Sabih <rafia.sabih@enterprisedb.com>, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>, Rajkumar Raghuwanshi <rajkumar.raghuwanshi@enterprisedb.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2017-08-16T11:51:43Z
Lists: pgsql-hackers 
On Wed, Aug 16, 2017 at 3:31 AM, Ashutosh Bapat
<ashutosh.bapat@enterprisedb.com> wrote:
> There are two ways we can do this
> 1. In expand_inherited_rtentry(), remember (childRTE and childRTIndex)
> or just childRTIndex (using this we can fetch childRTE calling
> rtfetch()) of intermediate partitioned tables. Once we are done
> expanding immediate children, call expand_inherited_rtentry()
> recursively on this list.
>
> 2. expand_inherited_tables() scans root->parse->rtable only upto the
> end of original range table list. Make it go beyond that end,
> expanding any new entries added for intermediate partitions.
>
> FWIW, the first option allows us to keep all AppendRelInfos
> corresponding to one partitioned relation together and also expands
> the whole partition hierarchy in one go. Second will require minimal
> changes to expand_inherited_rtentry(). Both approaches will spend time
> scanning same number of RTE; the first will have them in different
> lists, and second will have them in root->parse->rtable. I don't see
> one being more attractive than the other. Do you have any opinion?

I don't like option (2).  I'm not sure about option (1).  I think
maybe we should have two nested loops in expanded_inherited_rtentry(),
the outer one iterating over partitioned tables (or just the original
parent RTE if partitioning is not involved) and then inner one looping
over individual leaf partitions for each partitioned table.  Probably
we'd end up wanting to move at least some of the logic inside the
existing loop into a subroutine.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Commits

  1. Add test for partitionwise join involving default partition.

  2. Rewrite the code that applies scan/join targets to paths.

  3. Fix code related to partitioning schemes for dropped columns.

  4. Copy information from the relcache instead of pointing to it.

  5. Basic partition-wise join functionality.

  6. Associate partitioning information with each RelOptInfo.

  7. Expand partitioned table RTEs level by level, without flattening.

  8. Set partitioned_rels appropriately when UNION ALL is used.

  9. Remove dedicated B-tree root-split record types.

  10. Assorted preparatory refactoring for partition-wise join.

  11. Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.

  12. Avoid unnecessary single-child Append nodes.

  13. Revisit handling of UNION ALL subqueries with non-Var output columns.