Re: [HACKERS] Runtime Partition Pruning

On Mon, Oct 12, 2020 at 5:48 PM Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
wrote:

> On Mon, Oct 12, 2020 at 7:59 AM Andy Fan <zhihui.fan1213@gmail.com> wrote:
>
> >
> > Sorry for the late reply!  Suppose we have partition defined like this:
> > p
> > - p1
> > - p2
> >
> > When you talk about "the statistics from the partitioned table", do you
> > mean the statistics from p or p1/p2?   I just confirmed there is no
> statistics
> > for p (at least pg_class.reltuples = -1),  so I think you are talking
> about
> > p1/p2.
>
> I am talking about p when I say statistics from the partitioned table.
> I see that pg_statistic row from p is well populated.
> pg_class.reltuples = -1 indicates that the heap doesn't have any rows.
> set_rel_size() sets the number of rows in the partitioned table based
> on the rows in individual unpruned partitions.
>
>
Glad to know that, Thanks for this info!


> >
> > Here we are talking about partkey = $1 or  partkey = RunTimeValue.
> > so even the value can hit 1 partition only,  but since we don't know
> > the exact value, so we treat all the partition equally.  so looks
> > nothing wrong with partition level estimation.  However when we cost
> > the Append path, we need know just one of them can be hit,  then
> > we need do something there.  Both AppendPath->rows/total_cost
> > should be adjusted (That doesn't happen now).
>
> I think in this case we can safely assume that only one partition will
> remain so normalize costs considering that only one partition will
> survive.
>

Exactly.  What I am trying to do is fix this at create_append_path,
do you have different suggestions? about the pkey > $1 case,  I think
even if we use the statistics from partition level,  it would be
hard-code as well since we don't know what value $1 is.

I have gone through the main part of the RunTime partition prune, hope
I can update a runnable patch soon.  The main idea is fix the rows/
costs at create_append_path stage.  So any suggestion in a different
direction will be very useful.

-- 
Best Regards
Andy Fan