Re: POC, WIP: OR-clause support for indexes

Andrei Lepikhov <a.lepikhov@postgrespro.ru>

From: Andrei Lepikhov <a.lepikhov@postgrespro.ru>

To: Alena Rybakina <a.rybakina@postgrespro.ru>, jian he <jian.universality@gmail.com>, Alexander Korotkov <aekorotkov@gmail.com>

Cc: Robert Haas <robertmhaas@gmail.com>, pgsql-hackers@postgresql.org, Peter Geoghegan <pg@bowt.ie>, "Finnerty, Jim" <jfinnert@amazon.com>, Marcos Pegoraro <marcos@f10.com.br>, teodor@sigaev.ru, Ranier Vilela <ranier.vf@gmail.com>, Tomas Vondra <tomas.vondra@enterprisedb.com>, Peter Eisentraut <peter@eisentraut.org>

Date: 2024-02-08T05:34:38Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Make group_similar_or_args() reorder clause list as little as possible
- 775a06d44c04 18.0 landed
Allow usage of match_orclause_to_indexcol() for joins
- 627d63419e22 18.0 landed
Skip not SOAP-supported indexes while transforming an OR clause into SAOP
- 5bba0546eecb 18.0 landed
Remove the wrong assertion from match_orclause_to_indexcol()
- d4d11940df94 18.0 landed
Teach bitmap path generation about transforming OR-clauses to SAOP's
- ae4569161a27 18.0 landed
Transform OR-clauses to SAOP's during index matching
- d4378c0005e6 18.0 landed
Fix the value of or_to_any_transform_limit in postgresql.conf.sample
- 2af75e117478 17.0 landed
Transform OR clauses to ANY expression
- 72bd38cc99a1 17.0 landed
MergeAttributes code deduplication
- 64444ce071f6 17.0 cited
SEARCH and CYCLE clauses
- 3696a600e229 14.0 cited
Improve estimation of OR clauses using extended statistics.
- 25a9e54d2db3 14.0 cited
Teach btree to handle ScalarArrayOpExpr quals natively.
- 9e8da0f75731 9.2.0 cited
Revise collation derivation method and expression-tree representation.
- b310b6e31ce5 9.1.0 cited
Instead of trying to force WHERE clauses into CNF or DNF normal form,
- 9888192fb773 8.0.0 cited

Attachments

v16-0001-Transform-OR-clause-to-ANY-expressions.patch (text/plain) patch v16-0001
v16-0002-Teach-generate_bitmap_or_paths-to-build-BitmapOr-pat.patch (text/plain) patch v16-0002

On 3/2/2024 02:06, Alena Rybakina wrote:
> On 01.02.2024 08:00, jian he wrote:
> I added your code to the patch.
Thanks Alena and Jian for the detailed scrutiny!

A couple of questions:
1. As I see, transformAExprIn uses the same logic as we invented but 
allows composite and domain types. Could you add a comment explaining 
why we forbid row types in general, in contrast to the transformAExprIn 
routine?
2. Could you provide the tests to check issues covered by the recent (in 
v.15) changes?

Patch 0001-* in the attachment incorporates changes induced by Jian's 
notes from [1].
Patch 0002-* contains a transformation of the SAOP clause, which allows 
the optimizer to utilize partial indexes if they cover all values in 
this array. Also, it is an answer to Alexander's note [2] on performance 
degradation. This first version may be a bit raw, but I need your 
opinion: Does it resolve the issue?

Skimming through the thread, I see that, in general, all issues have 
been covered for now. Only Robert's note on a lack of documentation is 
still needs to be resolved.

[1] 
https://www.postgresql.org/message-id/CACJufxGXhJ823cdAdp2Ho7qC-HZ3_-dtdj-myaAi_u9RQLn45g%40mail.gmail.com
[2] 
https://www.postgresql.org/message-id/CAPpHfduJtO0s9E%3DSHUTzrCD88BH0eik0UNog1_q3XBF2wLmH6g%40mail.gmail.com

-- 
regards,
Andrei Lepikhov
Postgres Professional