Re: Fw:Re: Fw: ltree_compare in contrib/ltree/ltree_op.c overflows int32 on deep ltree comparisons, returning the wrong sign

Hi,

On Sat, 13 Jun 2026 at 09:20, 王跃林 <violin0613@tju.edu.cn> wrote:

> 主题：Re: Fw: ltree_compare in contrib/ltree/ltree_op.c overflows int32 on deep ltree comparisons, returning the wrong sign
> On Mon, May 25, 2026 at 10:58:05PM +0800, violin0613@tju.edu.cn wrote:
> > PoC
> >
> > File vuln_001.sql
> >
> >  CREATE EXTENSION IF NOT EXISTS ltree;
> >  â
> >  WITH s AS (SELECT 'a'::ltree AS v),
> >       l AS (SELECT (repeat('a.', 19999) || 'a')::ltree AS v)
> >  SELECT (l.v > s.v) AS long_gt_short_expected_true,
> >         (l.v < s.v) AS long_lt_short_expected_false
> >  FROM s, l;
> >
> > Process
> >
> >  psql -h /tmp -p 36265 -U postgres -f vuln_001.sql
> >
> > Results
> >
> >   long_gt_short_expected_true | long_lt_short_expected_false
> >  -----------------------------+------------------------------
> >   f                           | t
> >
> >    Both columns are inverted. long > short returned false, long < short
> >    returned true.
>
>
This looks like a classic case of integer overflow that's
happening in ltree_compare function in ltree_op.c.

return (al->len - bl->len) * 10 * (an + 1);
return res * 10 * (an + 1);
return (a->numlevel - b->numlevel) * 10 * (an + 1);

I think the calculation should be done as int64, something of this sort:

    int64 v = (int64) (al->len - bl->len) * 10 * (an + 1);
    if (v > PG_INT32_MAX) return PG_INT32_MAX;
    if (v < PG_INT32_MIN) return PG_INT32_MIN;
    return (int) v;

And needed to adjust the ltree_penalty function too.

Attached is a draft patch for this, I guess we can add a helper
function too for the above conversion.

Thoughts?

Regards,
Ayush