Re: POC, WIP: OR-clause support for indexes

Andrei Lepikhov <lepihov@gmail.com>

From: Andrei Lepikhov <lepihov@gmail.com>

To: Peter Geoghegan <pg@bowt.ie>, Alexander Korotkov <aekorotkov@gmail.com>

Cc: Tom Lane <tgl@sss.pgh.pa.us>, Alena Rybakina <a.rybakina@postgrespro.ru>, jian he <jian.universality@gmail.com>, Nikolay Shaplov <dhyan@nataraj.su>, pgsql-hackers@lists.postgresql.org, Robert Haas <robertmhaas@gmail.com>, pgsql-hackers@postgresql.org, Marcos Pegoraro <marcos@f10.com.br>, teodor@sigaev.ru, Peter Eisentraut <peter@eisentraut.org>, Ranier Vilela <ranier.vf@gmail.com>

Date: 2024-10-04T03:31:22Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Make group_similar_or_args() reorder clause list as little as possible
- 775a06d44c04 18.0 landed
Allow usage of match_orclause_to_indexcol() for joins
- 627d63419e22 18.0 landed
Skip not SOAP-supported indexes while transforming an OR clause into SAOP
- 5bba0546eecb 18.0 landed
Remove the wrong assertion from match_orclause_to_indexcol()
- d4d11940df94 18.0 landed
Teach bitmap path generation about transforming OR-clauses to SAOP's
- ae4569161a27 18.0 landed
Transform OR-clauses to SAOP's during index matching
- d4378c0005e6 18.0 landed
Fix the value of or_to_any_transform_limit in postgresql.conf.sample
- 2af75e117478 17.0 landed
Transform OR clauses to ANY expression
- 72bd38cc99a1 17.0 landed
MergeAttributes code deduplication
- 64444ce071f6 17.0 cited
SEARCH and CYCLE clauses
- 3696a600e229 14.0 cited
Improve estimation of OR clauses using extended statistics.
- 25a9e54d2db3 14.0 cited
Teach btree to handle ScalarArrayOpExpr quals natively.
- 9e8da0f75731 9.2.0 cited
Revise collation derivation method and expression-tree representation.
- b310b6e31ce5 9.1.0 cited
Instead of trying to force WHERE clauses into CNF or DNF normal form,
- 9888192fb773 8.0.0 cited

On 10/4/24 03:15, Peter Geoghegan wrote:
> On Tue, Oct 1, 2024 at 6:25 AM Alexander Korotkov <aekorotkov@gmail.com> wrote:
>> I think this patchset got much better, and it could possible be
>> committed after another round of cleanup and comment/docs improvement.
>> It would be very kind if you share your view on the decisions made in
>> this patchset.
Let me provide a standpoint to help Alexander.

The origin reason was - to avoid multiple BitmapOr, which has some 
effects at the planning stage (memory consumption, planning time) and 
execution (execution time growth). IndexScan also works better with a 
single array (especially a hashed one) than with a long list of clauses.
Another reason is that by spending some time identifying common operator 
family and variable-side clause equality, we open a way for future cheap 
improvements like removing duplicated constants.
Who knows, maybe we will be capable of using this code to improve 
cardinality estimations.

According to your proposal, we have had such casting to the common type 
in previous versions. Here, we avoid it intentionally: the general idea 
is about long lists of constants, and such casting causes questions 
about performance. Do I want it in the core? Yes, I do! But may we 
implement it a bit later to have time to probe the general method and 
see how it flies?

-- 
regards, Andrei Lepikhov