Horton

Re: speeding up planning with partitions

David Rowley <david.rowley@2ndquadrant.com>

View thread View on postgr.es
From: David Rowley <david.rowley@2ndquadrant.com>
To: "Imai, Yoshikazu" <imai.yoshikazu@jp.fujitsu.com>
Cc: Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>, Alvaro Herrera <alvherre@2ndquadrant.com>, Pg Hackers <pgsql-hackers@postgresql.org>
Date: 2019-01-21T01:31:59Z
Lists: pgsql-hackers 
On Mon, 21 Jan 2019 at 13:45, Imai, Yoshikazu
<imai.yoshikazu@jp.fujitsu.com> wrote:
> On Wed, Jan 17, 2019 at 10:25 AM, Amit Langote wrote:
> > Are you saying that, when using auto mode, all executions of the query
> > starting from 7th are slower than the first 5 executions?  That is, the
> > latency of creating and using a custom plan increases *after* a generic
> > plan is created and discarded on the 6th execution of the query?  If so,
> > that is inexplicable to me.
>
> Yes. And it's also inexplicable to me.
>
> I'll check if this fact is really correct by majoring the time of the
> first 5 queries before generic plan is created and the other queries
> after generic plan is created.

It would be interesting to see the profiles of having the generic plan
being built on the 6th execution vs the 400,000th execution.

I'd thought maybe one difference would be the relcache hash table
having been expanded greatly after the generic plan was created, but I
see even the generic plan is selecting a random partition, so the
cache would have ended up with that many items eventually anyway, and
since we're talking in the odds of 7.8k TPS with 4k partitions, it
would have only have taken about 2-3 seconds out of the 60 second run
to hit most or all of those partitions anyway. So I doubt it could be
that.

-- 
 David Rowley                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

Commits

  1. Clean up handling of constraint_exclusion and enable_partition_pruning.

  2. Add test case exercising formerly-unreached code in inheritance_planner.

  3. Speed up planning when partitions can be pruned at plan time.

  4. Avoid crash in partitionwise join planning under GEQO.

  5. Avoid passing query tlist around separately from root->processed_tlist.

  6. Build "other rels" of appendrel baserels in a separate step.

  7. Get rid of duplicate child RTE for a partitioned table.

  8. Rearrange make_partitionedrel_pruneinfo to avoid work when we can't prune.

  9. Don't copy PartitionBoundInfo in set_relation_partition_info.

  10. Move building of child base quals out into a new function

  11. Reorganize planner code moved in b60c39759908

  12. Move inheritance expansion code into its own file

  13. Fix inherited UPDATE/DELETE with UNION ALL subqueries.

  14. Rearrange planner to save the whole PlannerInfo (subroot) for a subquery.