Horton

Re: POC, WIP: OR-clause support for indexes

Andrei Lepikhov <a.lepikhov@postgrespro.ru>

View thread
From: Andrei Lepikhov <a.lepikhov@postgrespro.ru>
To: jian he <jian.universality@gmail.com>
Cc: Alena Rybakina <a.rybakina@postgrespro.ru>, Alexander Korotkov <aekorotkov@gmail.com>, Robert Haas <robertmhaas@gmail.com>, pgsql-hackers@postgresql.org, Peter Geoghegan <pg@bowt.ie>, "Finnerty, Jim" <jfinnert@amazon.com>, Marcos Pegoraro <marcos@f10.com.br>, teodor@sigaev.ru, Ranier Vilela <ranier.vf@gmail.com>, Tomas Vondra <tomas.vondra@enterprisedb.com>, Peter Eisentraut <peter@eisentraut.org>
Date: 2024-02-20T04:29:36Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Make group_similar_or_args() reorder clause list as little as possible

  2. Allow usage of match_orclause_to_indexcol() for joins

  3. Skip not SOAP-supported indexes while transforming an OR clause into SAOP

  4. Remove the wrong assertion from match_orclause_to_indexcol()

  5. Teach bitmap path generation about transforming OR-clauses to SAOP's

  6. Transform OR-clauses to SAOP's during index matching

  7. Fix the value of or_to_any_transform_limit in postgresql.conf.sample

  8. Transform OR clauses to ANY expression

  9. MergeAttributes code deduplication

  10. SEARCH and CYCLE clauses

  11. Improve estimation of OR clauses using extended statistics.

  12. Teach btree to handle ScalarArrayOpExpr quals natively.

  13. Revise collation derivation method and expression-tree representation.

  14. Instead of trying to force WHERE clauses into CNF or DNF normal form,

On 20/2/2024 11:03, jian he wrote:
>> Neither the code comments nor the commit message really explain the
>> design idea here. That's unfortunate, principally because it makes
>> review difficult.
>>
>> I'm very skeptical about the idea of using JumbleExpr for any part of
>> this. It seems fairly expensive, and it might produce false matches.
>> If expensive is OK, then why not just use equal()? If it's not, then
>> this probably isn't really OK either. But in any case there should be
>> comments explaining why this strategy was chosen.
> 
> The above message
> (https://postgr.es/m/CA%2BTgmoZCgP6FrBQEusn4yaWm02XU8OPeoEMk91q7PRBgwaAkFw%40mail.gmail.com)
> seems still not answered.
> How can we evaluate whether JumbleExpr is expensive or not?
> I used this naive script to test, but didn't find a big difference
> when enable_or_transformation is ON or OFF.
First, I am open to discussion here. But IMO, equal() operation is quite 
expensive by itself. We should use the hash table approach to avoid 
quadratic behaviour when looking for similar clauses in the 'OR' list.
Moreover, we use equal() in many places: selectivity estimations, 
proving of fitting the index, predtest, etc. So, by reducing the clause 
list, we eliminate many calls of the equal() routine, too.

> `leftop operator rightop`
> the operator can also be volatile.
> Do we need to check (op_volatile(opno) == PROVOLATILE_VOLATILE) within
> transformBoolExprOr?
As usual, could you provide a test case to discuss it more objectively?

-- 
regards,
Andrei Lepikhov
Postgres Professional