Horton

Re: POC, WIP: OR-clause support for indexes

Alexander Korotkov <aekorotkov@gmail.com>

View thread
From: Alexander Korotkov <aekorotkov@gmail.com>
To: Andrei Lepikhov <lepihov@gmail.com>
Cc: Alena Rybakina <a.rybakina@postgrespro.ru>, Pavel Borisov <pashkin.elfe@gmail.com>, Alexander Lakhin <exclusion@gmail.com>, pgsql-hackers@lists.postgresql.org
Date: 2025-03-28T11:59:30Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Make group_similar_or_args() reorder clause list as little as possible

  2. Allow usage of match_orclause_to_indexcol() for joins

  3. Skip not SOAP-supported indexes while transforming an OR clause into SAOP

  4. Remove the wrong assertion from match_orclause_to_indexcol()

  5. Teach bitmap path generation about transforming OR-clauses to SAOP's

  6. Transform OR-clauses to SAOP's during index matching

  7. Fix the value of or_to_any_transform_limit in postgresql.conf.sample

  8. Transform OR clauses to ANY expression

  9. MergeAttributes code deduplication

  10. SEARCH and CYCLE clauses

  11. Improve estimation of OR clauses using extended statistics.

  12. Teach btree to handle ScalarArrayOpExpr quals natively.

  13. Revise collation derivation method and expression-tree representation.

  14. Instead of trying to force WHERE clauses into CNF or DNF normal form,

On Fri, Mar 28, 2025 at 1:32 PM Andrei Lepikhov <lepihov@gmail.com> wrote:
> On 3/28/25 00:18, Alexander Korotkov wrote:
> > The attached patch changes the reordering algorithm of
> > group_similar_or_args() in the following way.  We reorder each group
> > of similar clauses so that the first item of the group stays in place,
> > but all the other items are moved after it.  So, if there are no
> > similar clauses, the order of clauses stays the same.  When there are
> > some groups, only required reordering happens while the rest of the
> > clauses remain in their places.
> The patch looks good to me from a technical perspective. But it seems
> like an overkill, isn't it?
> You introduce additional CPU-consuming operations in the planning OR
> operations.

I don't think this is going to be CPU-consuming.  I don't think this
is going to be measurable.  This patch introduces one additional pass
over array of OrArgIndexMatch'es, and qsort of them.  I think I've
seen places where we spend quadratic time over the number of
OR-clauses.  Even calls of match_index_to_operand() for every clause
and every index look way more expensive.

> My point is: 1) as Pavel has mentioned, Postgres doesn't guarantee the
> evaluation/output order of the clauses at all. 2) we need that to keep
> regression tests stable (don't forget extensions' and forks' developers
> too). But it should be done once if we have no fluidity in OR clauses
> order in general.
> The trade-off with tricky query writers and regression tests may be
> preserving the order until OR->ANY has happened. If it has happened,
> just ensure the order is determined somehow. Except that, any other
> spending on CPU cycles seems too expensive.

I think my patch gives better determinism too.  For instance, output
order doesn't depend on order of indexes in rel->indexlist.


------
Regards,
Alexander Korotkov
Supabase