Horton

Re: Adding skip scan (including MDAM style range skip scan) to nbtree

Peter Geoghegan <pg@bowt.ie>

View thread
From: Peter Geoghegan <pg@bowt.ie>
To: Natalya Aksman <natalya@tigerdata.com>
Cc: Masahiro Ikeda <ikedamsh@oss.nttdata.com>, Tomas Vondra <tomas@vondra.me>, Masahiro.Ikeda@nttdata.com, pgsql-hackers@lists.postgresql.org, Masao.Fujii@nttdata.com
Date: 2025-09-10T16:45:38Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. nbtree: Always set skipScan flag on rescan.

  2. meson: Build numeric.c with -ftree-vectorize.

  3. Fix "variable not found in subplan target lists" in semijoin de-duplication.

  4. Revert "nbtree: Remove useless row compare arg."

  5. nbtree: Remove useless row compare arg.

  6. Prevent premature nbtree array advancement.

  7. nbtree: tighten up array recheck rules.

  8. Avoid treating nonrequired nbtree keys as required.

  9. Adjust overstrong nbtree skip array assertion.

  10. Make NULL tuple values always advance skip arrays.

  11. Avoid extra index searches through preprocessing.

  12. Improve nbtree skip scan primitive scan scheduling.

  13. Further optimize nbtree search scan key comparisons.

  14. Add nbtree skip scan optimization.

  15. Improve nbtree array primitive scan scheduling.

  16. nbtree: Make BTMaxItemSize into object-like macro.

  17. Show index search count in EXPLAIN ANALYZE, take 2.

  18. Make parallel nbtree index scans use an LWLock.

  19. Show index search count in EXPLAIN ANALYZE.

  20. Avoid nbtree parallel scan currPos confusion.

  21. nbtree: Remove useless 'strat' local variable.

  22. Normalize nbtree truncated high key array behavior.

  23. Refactor handling of nbtree array redundancies.

  24. Fix nbtree pgstats accounting with parallel scans.

  25. Avoid parallel nbtree index scan hangs with SAOPs.

  26. Show Parallel Bitmap Heap Scan worker stats in EXPLAIN ANALYZE

  27. Enhance nbtree ScalarArrayOp execution.

  28. Skip checking of scan keys required for directional scan in B-tree

  29. Instead of using a numberOfRequiredKeys count to distinguish required

On Wed, Sep 10, 2025 at 9:53 AM Natalya Aksman <natalya@tigerdata.com> wrote:
> Our Timescaledb extension has scenarios changing ">" quals to "=" and back on rescan and it breaks when so->Skipscan needs to be reset from true to false.

But the amrescan docs say:

"In practice the restart feature is used when a new outer tuple is
selected by a nested-loop join and so a new key comparison value is
needed, but the scan key structure remains the same" [1].

I don't understand why it is that our not resetting the so->Skipscan
flag within btrescan has any particular significance to Timescaledb,
relative to all of the other fields that are supposed to be set by
_bt_preprocess_keys. What is the actual failure you see? Is it an
assertion failure within _bt_readpage/_bt_checkkeys?

Note that btrescan *does* set "so->numberOfKeys = 0", which will make
the next call to _bt_preprocess_keys (from _bt_first) perform
preprocessing from scratch. This should set so->Skipscan from scratch
on each rescan (along with every other field set by preprocessing). It
seems like that should work for you (in spite of the fact that you're
doing something that seems at odds with the index AM API).

[1] https://www.postgresql.org/docs/current/index-functions.html
-- 
Peter Geoghegan