Re: speeding up planning with partitions

Amit Langote <langote_amit_f8@lab.ntt.co.jp>

From: Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>

To: jesper.pedersen@redhat.com

Cc: "Imai, Yoshikazu" <imai.yoshikazu@jp.fujitsu.com>, "'Amit Langote'" <amitlangote09@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>, David Rowley <david.rowley@2ndquadrant.com>, Alvaro Herrera <alvherre@2ndquadrant.com>, Justin Pryzby <pryzby@telsasoft.com>, Pg Hackers <pgsql-hackers@postgresql.org>

Date: 2019-03-05T10:24:26Z

Lists: pgsql-hackers

Attachments

v27-0001-Build-other-rels-of-appendrel-baserels-in-a-sepa.patch (text/plain) patch v27-0001
v27-0002-Delay-adding-inheritance-child-tables-until-quer.patch (text/plain) patch v27-0002
v27-0003-Adjust-inheritance_planner-to-reuse-source-child.patch (text/plain) patch v27-0003
v27-0004-Perform-pruning-in-expand_partitioned_rtentry.patch (text/plain) patch v27-0004
v27-0005-Teach-planner-to-only-process-unpruned-partition.patch (text/plain) patch v27-0005
v27-0006-Don-t-copy-PartitionBoundInfo-in-set_relation_pa.patch (text/plain) patch v27-0006

On 2019/03/05 9:50, Amit Langote wrote:
> I'll post the updated patches after diagnosing what
> I'm suspecting a memory over-allocation bug in one of the patches.  If you
> configure build with --enable-cassert, you'll see that throughput
> decreases as number of partitions run into many thousands, but it doesn't
> when asserts are turned off.

Attached an updated version.  This incorporates fixes for both Jesper's
and Imai-san's review.  I haven't been able to pin down the bug (or
whatever) that makes throughput go down as the partition count increases,
when tested with a --enable-cassert build.

Thanks,
Amit

Commits

Clean up handling of constraint_exclusion and enable_partition_pruning.
- e03ff739695c 12.0 landed
- 11ea45ffec9a 11.3 landed
Add test case exercising formerly-unreached code in inheritance_planner.
- 8fba397f0ca7 12.0 landed
Speed up planning when partitions can be pruned at plan time.
- 428b260f87e8 12.0 landed
Avoid crash in partitionwise join planning under GEQO.
- 7ad6498fd5a6 12.0 landed
- d70c147fa217 11.3 landed
Avoid passing query tlist around separately from root->processed_tlist.
- 333ed246c6f3 12.0 landed
Build "other rels" of appendrel baserels in a separate step.
- 53bcf5e3dbea 12.0 landed
Get rid of duplicate child RTE for a partitioned table.
- e8d5dd6be79e 12.0 landed
Rearrange make_partitionedrel_pruneinfo to avoid work when we can't prune.
- 734308a22072 12.0 landed
Don't copy PartitionBoundInfo in set_relation_partition_info.
- c8151e642368 12.0 landed
Move building of child base quals out into a new function
- 80579f9bb171 12.0 landed
Reorganize planner code moved in b60c39759908
- d723f56872a9 12.0 landed
Move inheritance expansion code into its own file
- b60c39759908 12.0 landed
Fix inherited UPDATE/DELETE with UNION ALL subqueries.
- c03ad5602f52 9.4.0 cited
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery.
- b3aaf9081a1a 9.2.0 cited