Horton

Re: speeding up planning with partitions

David Rowley <david.rowley@2ndquadrant.com>

View thread View on postgr.es
From: David Rowley <david.rowley@2ndquadrant.com>
To: "Imai, Yoshikazu" <imai.yoshikazu@jp.fujitsu.com>
Cc: "Tsunakawa, Takayuki" <tsunakawa.takay@jp.fujitsu.com>, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>, Alvaro Herrera <alvherre@2ndquadrant.com>, Pg Hackers <pgsql-hackers@postgresql.org>
Date: 2019-02-08T01:45:16Z
Lists: pgsql-hackers 
On Fri, 8 Feb 2019 at 14:34, Imai, Yoshikazu
<imai.yoshikazu@jp.fujitsu.com> wrote:
>
> On Wed, Feb 6, 2019 at 2:04 AM, Tsunakawa, Takayuki wrote:
> > Can you compare the performance of auto and force_custom_plan again with
> > the attached patch?  It uses PGPROC's LOCALLOCK list instead of the hash
> > table.
>
> Thanks for the patch, but it seems to have some problems.
> When I executed create/drop/select commands to large partitions, like over than 512 partitions, backend died unexpectedly. Since I could see the difference of the performance of auto and force_custom_plan when partitions is large, patch needs to be modified to check whether performance is improved or not.

It's good to see work being done to try and improve this, but I think
it's best to do it on another thread. I think there was some agreement
upthread about this not being Amit's patch's problem. Doing it here
will make keeping track of this more complex than it needs to be.
There's also Amit's issue of keeping his patch series up to date. The
CFbot is really useful to alert patch authors when that's required,
but having other patches posted to the same thread can cause the CFbot
to check the wrong patch.

-- 
 David Rowley                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

Commits

  1. Clean up handling of constraint_exclusion and enable_partition_pruning.

  2. Add test case exercising formerly-unreached code in inheritance_planner.

  3. Speed up planning when partitions can be pruned at plan time.

  4. Avoid crash in partitionwise join planning under GEQO.

  5. Avoid passing query tlist around separately from root->processed_tlist.

  6. Build "other rels" of appendrel baserels in a separate step.

  7. Get rid of duplicate child RTE for a partitioned table.

  8. Rearrange make_partitionedrel_pruneinfo to avoid work when we can't prune.

  9. Don't copy PartitionBoundInfo in set_relation_partition_info.

  10. Move building of child base quals out into a new function

  11. Reorganize planner code moved in b60c39759908

  12. Move inheritance expansion code into its own file

  13. Fix inherited UPDATE/DELETE with UNION ALL subqueries.

  14. Rearrange planner to save the whole PlannerInfo (subroot) for a subquery.