Horton

Re: POC, WIP: OR-clause support for indexes

Peter Geoghegan <pg@bowt.ie>

View thread
From: Peter Geoghegan <pg@bowt.ie>
To: Andrei Lepikhov <lepihov@gmail.com>
Cc: Alexander Korotkov <aekorotkov@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>, Alena Rybakina <a.rybakina@postgrespro.ru>, jian he <jian.universality@gmail.com>, Nikolay Shaplov <dhyan@nataraj.su>, pgsql-hackers@lists.postgresql.org, Robert Haas <robertmhaas@gmail.com>, pgsql-hackers@postgresql.org, Marcos Pegoraro <marcos@f10.com.br>, teodor@sigaev.ru, Peter Eisentraut <peter@eisentraut.org>, Ranier Vilela <ranier.vf@gmail.com>
Date: 2024-10-04T15:00:23Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Make group_similar_or_args() reorder clause list as little as possible

  2. Allow usage of match_orclause_to_indexcol() for joins

  3. Skip not SOAP-supported indexes while transforming an OR clause into SAOP

  4. Remove the wrong assertion from match_orclause_to_indexcol()

  5. Teach bitmap path generation about transforming OR-clauses to SAOP's

  6. Transform OR-clauses to SAOP's during index matching

  7. Fix the value of or_to_any_transform_limit in postgresql.conf.sample

  8. Transform OR clauses to ANY expression

  9. MergeAttributes code deduplication

  10. SEARCH and CYCLE clauses

  11. Improve estimation of OR clauses using extended statistics.

  12. Teach btree to handle ScalarArrayOpExpr quals natively.

  13. Revise collation derivation method and expression-tree representation.

  14. Instead of trying to force WHERE clauses into CNF or DNF normal form,

On Thu, Oct 3, 2024 at 11:31 PM Andrei Lepikhov <lepihov@gmail.com> wrote:
> The origin reason was - to avoid multiple BitmapOr, which has some
> effects at the planning stage (memory consumption, planning time) and
> execution (execution time growth). IndexScan also works better with a
> single array (especially a hashed one) than with a long list of clauses.

I understand that that was the original goal. But I think that
preserving ordered index scans by using a SAOP (not filter quals and
not a BitmapOr) is actually the more important reason to have this
patch. It allows the OR syntax to be used in a way that preserves
crucial context.

I'm not really trying to add new requirements for this patch. The case
I highlighted wasn't a particularly tricky one. It's a case that the
existing IN() syntax somehow manages to produce a useful SAOP for. It
would be nice to get that part right.

> Another reason is that by spending some time identifying common operator
> family and variable-side clause equality, we open a way for future cheap
> improvements like removing duplicated constants.

I don't think that removing duplicated constants is all that
important, since we already do that during execution proper. The
nbtree code does this in _bt_preprocess_array_keys. It even does
things like merge together a pair of duplicate SAOPs against the same
column. It doesn't matter if the arrays are of different types,
either.

It doesn't look like index AMs lacking native support for SAOPs can do
stuff like that right now. It could be implemented by sorting and
deduplicating the IndexArrayKeyInfo.elem_values[] array in the same
way as nbtree.

-- 
Peter Geoghegan