Horton

Re: speeding up planning with partitions

Amit Langote <langote_amit_f8@lab.ntt.co.jp>

View thread View on postgr.es
From: Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>
To: David Rowley <david.rowley@2ndquadrant.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Imai Yoshikazu <yoshikazu_i443@live.jp>, "jesper.pedersen@redhat.com" <jesper.pedersen@redhat.com>, "Imai, Yoshikazu" <imai.yoshikazu@jp.fujitsu.com>, Amit Langote <amitlangote09@gmail.com>, Alvaro Herrera <alvherre@2ndquadrant.com>, Robert Haas <robertmhaas@gmail.com>, Justin Pryzby <pryzby@telsasoft.com>, Pg Hackers <pgsql-hackers@postgresql.org>
Date: 2019-03-27T05:39:20Z
Lists: pgsql-hackers 
On 2019/03/27 14:26, David Rowley wrote:
> On Wed, 27 Mar 2019 at 18:13, Amit Langote
> <Langote_Amit_f8@lab.ntt.co.jp> wrote:
>>
>> On 2019/03/27 13:50, Amit Langote wrote:
>>> On 2019/03/27 12:06, Amit Langote wrote:
>>>> I wonder if I should rework inherit.c so that its internal interfaces
>>>> don't pass around parent Relation, but make do with the RelOptInfo?  I'll
>>>> need to add tupdesc and reltype fields to RelOptInfo to go ahead with that
>>>> though.
>>>
>>> To give more context on the last sentence, the only place we need the
>>> TupleDesc for is make_inh_translation_list().  Maybe, instead of adding
>>> tupdesc to RelOptInfo, we could add a List of Vars of all attributes of a
>>> table.  To avoid the overhead of always setting it, maybe we could set it
>>> only for inheritance parent tables.  Adding tupdesc to RelOptInfo might be
>>> a hard sell to begin with, because it would need to be pinned and then
>>> released at the end of planning.
>>>
>>> I'll see if I can make this work.
>>
>> Hmm, scratch that.  We'd need to keep around the attribute names too for
>> comparing with the attribute names of the children during translation.
>> Maybe we could store TargetEntry's in the list instead of just Vars, but
>> maybe that's too much.
>>
>> Furthermore, if we make make_inh_translation_list too dependent on this
>> stuff, that will make it hard to use from inheritance_planner(), where we
>> expand the target inheritance without having created any RelOptInfos.
> 
> Perhaps the way to make this work, at least in the long term is to do
> in the planner what we did in the executor in d73f4c74dd34.

Maybe you meant 9ddef36278a9?

What would be nice is being able to readily access Relation pointers of
all tables accessed in a query from anywhere in the planner, whereby, a
given table is opened only once.

Note that Tom complained upthread that the patch is introducing
table_open()'s at random points within the planner, which is something to
avoid.

>  I'm not
> quite sure how exactly you'd know what size to make the array, but if
> it's not something that's happening for PG12, then maybe we can just
> use a List, once we get array based versions of them, at least.

We're going to have to expand the PlannerInfo arrays (simple_rel_*,
append_rel_array, etc.) with the patches here anyway.  Maybe, it will be
just one more array to consider?

Thanks,
Amit

Commits

  1. Clean up handling of constraint_exclusion and enable_partition_pruning.

  2. Add test case exercising formerly-unreached code in inheritance_planner.

  3. Speed up planning when partitions can be pruned at plan time.

  4. Avoid crash in partitionwise join planning under GEQO.

  5. Avoid passing query tlist around separately from root->processed_tlist.

  6. Build "other rels" of appendrel baserels in a separate step.

  7. Get rid of duplicate child RTE for a partitioned table.

  8. Rearrange make_partitionedrel_pruneinfo to avoid work when we can't prune.

  9. Don't copy PartitionBoundInfo in set_relation_partition_info.

  10. Move building of child base quals out into a new function

  11. Reorganize planner code moved in b60c39759908

  12. Move inheritance expansion code into its own file

  13. Fix inherited UPDATE/DELETE with UNION ALL subqueries.

  14. Rearrange planner to save the whole PlannerInfo (subroot) for a subquery.