Horton

Re: speeding up planning with partitions

Amit Langote <langote_amit_f8@lab.ntt.co.jp>

View thread View on postgr.es
From: Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>
To: "Imai, Yoshikazu" <imai.yoshikazu@jp.fujitsu.com>
Cc: "'David Rowley'" <david.rowley@2ndquadrant.com>, "'Alvaro Herrera'" <alvherre@2ndquadrant.com>, "'Pg Hackers'" <pgsql-hackers@postgresql.org>
Date: 2018-12-27T11:25:38Z
Lists: pgsql-hackers

Attachments

Hi,

On 2018/12/26 13:28, Amit Langote wrote:
> On 2018/12/25 16:47, Imai, Yoshikazu wrote:
>> adjust_inherit_target_child() is in allpaths.c in your patch, but it has the
>> code like below ones which are in master's(not patch applied) planner.c or
>> planmain.c, so could it be possible in planner.c(or planmain.c)?
>> This point is less important, but I was just wondering whether 
>> adjust_inherit_target_child() should be in allpaths.c, planner.c or planmain.c.
>>
>> +	/* Translate the original query's expressions to this child. */
>> +	subroot = makeNode(PlannerInfo);
>> +	memcpy(subroot, root, sizeof(PlannerInfo));
>>
>> +	root->parse->targetList = root->unexpanded_tlist;
>> +	subroot->parse = (Query *) adjust_appendrel_attrs(root,
>> +													  (Node *) root->parse,
>> +													  1, &appinfo);
>>
>> +	tlist = preprocess_targetlist(subroot);
>> +	subroot->processed_tlist = tlist;
>> +	build_base_rel_tlists(subroot, tlist);
> 
> I'm thinking of changing where adjust_inherit_target_child gets called
> from.  In the current patch, it's in the middle of set_rel_size which I'm
> starting to think is a not a good place for it.  Maybe, I'll place the
> call call near to where inheritance is expanded.

So what I did here is that I added a new step to query_planner() itself
that adds the PlannerInfos for inheritance child target relations and
hence moved the function adjust_inherit_target_child() to planmain.c and
renamed it to add_inherit_target_child_root().  I've removed its
RelOptInfo argument and decided to modify the child RelOptInfo's
targetlist in allpaths.c.  So, add_inherit_target_child_root() only builds
the child PlannerInfo now.  Child PlannerInfo's are now stored in an array
of simple_rel_array_size elements, instead of a list.  It's filled with
NULLs except for the slots corresponding to child target relations.

In allpaths.c, I've added set_inherit_target_rel_sizes and
set_inherit_target_rel_pathlists, which look like set_append_rel_size and
set_append_rel_pathlist, respectively.  The reason to add separate
functions for the target relation case is that we don't want to combine
the outputs of children to form an "append relation" in that case.  So,
this patch no longer modifies set_append_rel_size for handling the case
where the parent relation is query's target relation.

> I will send an updated patch hopefully before my new year vacation that
> starts on Friday this week.

Here's the v10 of the patch.  I didn't get chance to do more changes than
those described above and address Imai-san's review comments yesterday.

Have a wonderful new year!  I'll be back on January 7 to reply on this thread.

Thanks,
Amit

Commits

  1. Clean up handling of constraint_exclusion and enable_partition_pruning.

  2. Add test case exercising formerly-unreached code in inheritance_planner.

  3. Speed up planning when partitions can be pruned at plan time.

  4. Avoid crash in partitionwise join planning under GEQO.

  5. Avoid passing query tlist around separately from root->processed_tlist.

  6. Build "other rels" of appendrel baserels in a separate step.

  7. Get rid of duplicate child RTE for a partitioned table.

  8. Rearrange make_partitionedrel_pruneinfo to avoid work when we can't prune.

  9. Don't copy PartitionBoundInfo in set_relation_partition_info.

  10. Move building of child base quals out into a new function

  11. Reorganize planner code moved in b60c39759908

  12. Move inheritance expansion code into its own file

  13. Fix inherited UPDATE/DELETE with UNION ALL subqueries.

  14. Rearrange planner to save the whole PlannerInfo (subroot) for a subquery.