Horton

RE: speeding up planning with partitions

Imai, Yoshikazu <imai.yoshikazu@jp.fujitsu.com>

View thread View on postgr.es
From: "Imai, Yoshikazu" <imai.yoshikazu@jp.fujitsu.com>
To: "Imai, Yoshikazu" <imai.yoshikazu@jp.fujitsu.com>, 'Amit Langote' <amitlangote09@gmail.com>
Cc: Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>, Tom Lane <tgl@sss.pgh.pa.us>, David Rowley <david.rowley@2ndquadrant.com>, "Alvaro Herrera" <alvherre@2ndquadrant.com>, Justin Pryzby <pryzby@telsasoft.com>, "Pg Hackers" <pgsql-hackers@postgresql.org>, Jesper Pedersen <jesper.pedersen@redhat.com>
Date: 2019-03-14T08:35:09Z
Lists: pgsql-hackers 
Amit-san,

I have done code review of v31 patches from 0004 to 0008.

0004:
* s/childern/children

0005:
* This seems reasonable for not using a lot of memory in specific case, although it needs special looking of planner experts.

0006:
* The codes initializing/setting RelOptInfo's part_rels looks like a bit complicated, but I didn't come up with any good design so far.

0007:
* This changes some processes using "for loop" to using "while(bms_next_member())" which speeds up processing when we scan few partitions in one statement, but when we scan a lot of partitions in one statement, its performance will likely degraded. I measured the performance of both cases.
I executed select statement to the table which has 4096 partitions.

[scanning 1 partition]
Without 0007 : 3,450 TPS
With 0007    : 3,723 TPS

[scanning 4096 partitions]
Without 0007 : 10.8 TPS
With 0007    : 10.5 TPS

In the above result, performance degrades 3% in case of scanning 4096 partitions compared before and after applying 0007 patch. I think when scanning a lot of tables, executor time would be also longer, so the increasement of planner time would be relatively smaller than it. So we might not have to care this performance degradation.

0008:
This seems ok.


--
Yoshikazu Imai

Commits

  1. Clean up handling of constraint_exclusion and enable_partition_pruning.

  2. Add test case exercising formerly-unreached code in inheritance_planner.

  3. Speed up planning when partitions can be pruned at plan time.

  4. Avoid crash in partitionwise join planning under GEQO.

  5. Avoid passing query tlist around separately from root->processed_tlist.

  6. Build "other rels" of appendrel baserels in a separate step.

  7. Get rid of duplicate child RTE for a partitioned table.

  8. Rearrange make_partitionedrel_pruneinfo to avoid work when we can't prune.

  9. Don't copy PartitionBoundInfo in set_relation_partition_info.

  10. Move building of child base quals out into a new function

  11. Reorganize planner code moved in b60c39759908

  12. Move inheritance expansion code into its own file

  13. Fix inherited UPDATE/DELETE with UNION ALL subqueries.

  14. Rearrange planner to save the whole PlannerInfo (subroot) for a subquery.