Re: speeding up planning with partitions

Amit Langote <langote_amit_f8@lab.ntt.co.jp>

From: Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>
To: "Imai, Yoshikazu" <imai.yoshikazu@jp.fujitsu.com>
Cc: "'Amit Langote'" <amitlangote09@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>, David Rowley <david.rowley@2ndquadrant.com>, Alvaro Herrera <alvherre@2ndquadrant.com>, Justin Pryzby <pryzby@telsasoft.com>, Pg Hackers <pgsql-hackers@postgresql.org>
Date: 2019-03-04T10:42:54Z
Lists: pgsql-hackers
Imai-san,

Thanks for the review.

On 2019/03/04 18:14, Imai, Yoshikazu wrote:
> I've taken at glance the codes and there are some minor comments about the patch.
> 
> * You changed a usage/arguments of get_relation_info, but why you did it? I made those codes back to the original code and checked it passes make check. So ISTM there are no logical problems with not changing it. Or if you change it, how about also change a usage/arguments of get_relation_info_hook in the same way?
> 
> -get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
> -				  RelOptInfo *rel)
> +get_relation_info(PlannerInfo *root, RangeTblEntry *rte, RelOptInfo *rel)
>  {
> +	bool		inhparent = rte->inh;
> -	relation = table_open(relationObjectId, NoLock);
> +	relation = heap_open(rte->relid, NoLock);
>   ...
>  	if (get_relation_info_hook)
> -		(*get_relation_info_hook) (root, relationObjectId, inhparent, rel);
> +		(*get_relation_info_hook) (root, rte->relid, rte->inh, rel);
> 
> 
> @@ -217,15 +272,13 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
>  	/* Check type of rtable entry */
>  	switch (rte->rtekind)
>  	{
>  		case RTE_RELATION:
>  			/* Table --- retrieve statistics from the system catalogs */
> -			get_relation_info(root, rte->relid, rte->inh, rel);
> +			get_relation_info(root, rte, rel);
> 
> 
> * You moved the codes of initializing of append rel's partitioned_child_rels in set_append_rel_size() to build_simple_rel(), but is it better to do? I also checked the original code passes make check by doing like above.
> 
> @@ -954,32 +948,6 @@ set_append_rel_size(PlannerInfo *root, RelOptInfo *rel,
>  	Assert(IS_SIMPLE_REL(rel));
>  
>  	/*
> -	 * Initialize partitioned_child_rels to contain this RT index.
> -	 *
> -	 * Note that during the set_append_rel_pathlist() phase, we will bubble up
> -	 * the indexes of partitioned relations that appear down in the tree, so
> -	 * that when we've created Paths for all the children, the root
> -	 * partitioned table's list will contain all such indexes.
> -	 */
> -	if (rte->relkind == RELKIND_PARTITIONED_TABLE)
> -		rel->partitioned_child_rels = list_make1_int(rti);
> 
> @@ -274,55 +327,287 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
>  										list_length(rte->securityQuals));
>  
>  	/*
> -	 * If this rel is an appendrel parent, recurse to build "other rel"
> -	 * RelOptInfos for its children.  They are "other rels" because they are
> -	 * not in the main join tree, but we will need RelOptInfos to plan access
> -	 * to them.
> +	 * Add the parent to partitioned_child_rels.
> +	 *
> +	 * Note that during the set_append_rel_pathlist() phase, values of the
> +	 * indexes of partitioned relations that appear down in the tree will
> +	 * be bubbled up into root parent's list so that when we've created
> +	 * Paths for all the children, the root table's list will contain all
> +	 * such indexes.
>  	 */
> -	if (rte->inh)
> +	if (rel->part_scheme)
> +		rel->partitioned_child_rels = list_make1_int(relid);

Both of these changes are not present in the latest patches I posted,
where I also got rid of a lot of unnecessary diffs.

Thanks,
Amit



Commits

  1. Clean up handling of constraint_exclusion and enable_partition_pruning.

  2. Add test case exercising formerly-unreached code in inheritance_planner.

  3. Speed up planning when partitions can be pruned at plan time.

  4. Avoid crash in partitionwise join planning under GEQO.

  5. Avoid passing query tlist around separately from root->processed_tlist.

  6. Build "other rels" of appendrel baserels in a separate step.

  7. Get rid of duplicate child RTE for a partitioned table.

  8. Rearrange make_partitionedrel_pruneinfo to avoid work when we can't prune.

  9. Don't copy PartitionBoundInfo in set_relation_partition_info.

  10. Move building of child base quals out into a new function

  11. Reorganize planner code moved in b60c39759908

  12. Move inheritance expansion code into its own file

  13. Fix inherited UPDATE/DELETE with UNION ALL subqueries.

  14. Rearrange planner to save the whole PlannerInfo (subroot) for a subquery.