Re: POC, WIP: OR-clause support for indexes

jian he <jian.universality@gmail.com>

From: jian he <jian.universality@gmail.com>
To: Alexander Korotkov <aekorotkov@gmail.com>
Cc: Andrei Lepikhov <lepihov@gmail.com>, Robert Haas <robertmhaas@gmail.com>, Alena Rybakina <a.rybakina@postgrespro.ru>, Nikolay Shaplov <dhyan@nataraj.su>, pgsql-hackers@lists.postgresql.org, pgsql-hackers@postgresql.org, Peter Geoghegan <pg@bowt.ie>, Marcos Pegoraro <marcos@f10.com.br>, teodor@sigaev.ru, Peter Eisentraut <peter@eisentraut.org>, Ranier Vilela <ranier.vf@gmail.com>
Date: 2024-10-28T07:19:00Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →
  1. Make group_similar_or_args() reorder clause list as little as possible

  2. Allow usage of match_orclause_to_indexcol() for joins

  3. Skip not SOAP-supported indexes while transforming an OR clause into SAOP

  4. Remove the wrong assertion from match_orclause_to_indexcol()

  5. Teach bitmap path generation about transforming OR-clauses to SAOP's

  6. Transform OR-clauses to SAOP's during index matching

  7. Fix the value of or_to_any_transform_limit in postgresql.conf.sample

  8. Transform OR clauses to ANY expression

  9. MergeAttributes code deduplication

  10. SEARCH and CYCLE clauses

  11. Improve estimation of OR clauses using extended statistics.

  12. Teach btree to handle ScalarArrayOpExpr quals natively.

  13. Revise collation derivation method and expression-tree representation.

  14. Instead of trying to force WHERE clauses into CNF or DNF normal form,

 * NOTE:  returns NULL if clause is an OR or AND clause; it is the
 * responsibility of higher-level routines to cope with those.
 */
static IndexClause *
match_clause_to_indexcol(PlannerInfo *root,
                         RestrictInfo *rinfo,
                         int indexcol,
                         IndexOptInfo *index)

the above comments need a slight change.


EXPLAIN (COSTS OFF, settings) SELECT * FROM tenk2 WHERE  (thousand = 1
OR thousand = 3);
                        QUERY PLAN
-----------------------------------------------------------
 Bitmap Heap Scan on tenk2
   Recheck Cond: ((thousand = 1) OR (thousand = 3))
   ->  Bitmap Index Scan on tenk2_thous_tenthous
         Index Cond: (thousand = ANY ('{1,3}'::integer[]))

EXPLAIN (COSTS OFF, settings) SELECT * FROM tenk2 WHERE  (thousand in (1,3));
                        QUERY PLAN
-----------------------------------------------------------
 Bitmap Heap Scan on tenk2
   Recheck Cond: (thousand = ANY ('{1,3}'::integer[]))
   ->  Bitmap Index Scan on tenk2_thous_tenthous
         Index Cond: (thousand = ANY ('{1,3}'::integer[]))

tenk2 index:
Indexes:
    "tenk2_thous_tenthous" btree (thousand, tenthous)

Looking at the above cases, I found out the "Recheck Cond" is
different from "Index Cond".
I wonder why there is a difference, or if they should be the same.
then i come to:
match_orclause_to_indexcol

    /*
     * Finally, build an IndexClause based on the SAOP node. Use
     * make_simple_restrictinfo() to get RestrictInfo with clean selectivity
     * estimations because it may differ from the estimation made for an OR
     * clause. Although it is not a lossy expression, keep the old version of
     * rinfo in iclause->rinfo to detect duplicates and recheck the original
     * clause.
     */
    iclause = makeNode(IndexClause);
    iclause->rinfo = rinfo;
    iclause->indexquals = list_make1(make_simple_restrictinfo(root,
                                                              &saopexpr->xpr));
    iclause->lossy = false;
    iclause->indexcol = indexcol;
    iclause->indexcols = NIL;

looking at create_bitmap_scan_plan.
I think "iclause->rinfo" itself won't be able to detect duplicates.
since the upper code would mostly use "iclause->indexquals" for comparison?


typedef struct IndexClause comments says:
"
 * indexquals is a list of RestrictInfos for the directly-usable index
 * conditions associated with this IndexClause.  In the simplest case
 * it's a one-element list whose member is iclause->rinfo.  Otherwise,
 * it contains one or more directly-usable indexqual conditions extracted
 * from the given clause.  The 'lossy' flag indicates whether the
 * indexquals are semantically equivalent to the original clause, or
 * represent a weaker condition.
"
should lossy be iclause->lossy be true at the end of match_orclause_to_indexcol?
since it meets the comment condition: "semantically equivalent to the
original clause"
or is the above comment slightly wrong?

in match_orclause_to_indexcol
i changed from
iclause->rinfo = rinfo;
to
 iclause->rinfo = make_simple_restrictinfo(root,
                                                &saopexpr->xpr);

as expected. now the "Recheck Cond" is same as "Index Cond"
   Recheck Cond: (thousand = ANY ('{1,3}'::integer[]))
   ->  Bitmap Index Scan on tenk2_thous_tenthous
         Index Cond: (thousand = ANY ('{1,3}'::integer[]))

I am not sure of the implication of this change.