Re: Propagate stadistinct through GROUP BY/DISTINCT in subqueries and CTEs

On Mon, Apr 13, 2026 at 12:27 PM wenhui qiu <qiuwenhuifx@gmail.com> wrote:

> Thanks so much for working on this! While looking at the negative stadistinct conversion, I was wondering if we might run into a potential edge case with multi-level nested subqueries. What do you think?
>
> /* Convert negative stadistinct to absolute count */
>
>     if (stats->stadistinct < 0)
>     {
> -       RelOptInfo *baserel = find_base_rel(subroot, var->varno);
> +       RelOptInfo *baserel = vardata->rel;
>
> -       if (baserel->tuples > 0)
> +       if (baserel && baserel->tuples > 0)
>         {
>             stats->stadistinct = (float4)
>                 clamp_row_est(-stats->stadistinct * baserel->tuples);
>         }
>     }

I don't think your proposed change would work.  vardata->rel is the
CTE/subquery scan rel in the outer query, and its tuples count is the
CTE's output row count, not the base table's.  Using it would be
equivalent to not converting at all, since get_variable_numdistinct()
already computes -stadistinct * vardata->rel->tuples.  What we need
here is the base table's rel in the subroot, which gives us the
correct rowcount for interpreting the negative fraction.

- Richard