Horton

Re: Adding skip scan (including MDAM style range skip scan) to nbtree

Tomas Vondra <tomas@vondra.me>

View thread
From: Tomas Vondra <tomas@vondra.me>
To: Matthias van de Meent <boekewurm+postgres@gmail.com>, Peter Geoghegan <pg@bowt.ie>, rmt@lists.postgresql.org
Cc: Mark Dilger <mark.dilger@enterprisedb.com>, Heikki Linnakangas <hlinnaka@iki.fi>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Alexander Korotkov <aekorotkov@gmail.com>
Date: 2025-08-29T13:09:57Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. nbtree: Always set skipScan flag on rescan.

  2. meson: Build numeric.c with -ftree-vectorize.

  3. Fix "variable not found in subplan target lists" in semijoin de-duplication.

  4. Revert "nbtree: Remove useless row compare arg."

  5. nbtree: Remove useless row compare arg.

  6. Prevent premature nbtree array advancement.

  7. nbtree: tighten up array recheck rules.

  8. Avoid treating nonrequired nbtree keys as required.

  9. Adjust overstrong nbtree skip array assertion.

  10. Make NULL tuple values always advance skip arrays.

  11. Avoid extra index searches through preprocessing.

  12. Improve nbtree skip scan primitive scan scheduling.

  13. Further optimize nbtree search scan key comparisons.

  14. Add nbtree skip scan optimization.

  15. Improve nbtree array primitive scan scheduling.

  16. nbtree: Make BTMaxItemSize into object-like macro.

  17. Show index search count in EXPLAIN ANALYZE, take 2.

  18. Make parallel nbtree index scans use an LWLock.

  19. Show index search count in EXPLAIN ANALYZE.

  20. Avoid nbtree parallel scan currPos confusion.

  21. nbtree: Remove useless 'strat' local variable.

  22. Normalize nbtree truncated high key array behavior.

  23. Refactor handling of nbtree array redundancies.

  24. Fix nbtree pgstats accounting with parallel scans.

  25. Avoid parallel nbtree index scan hangs with SAOPs.

  26. Show Parallel Bitmap Heap Scan worker stats in EXPLAIN ANALYZE

  27. Enhance nbtree ScalarArrayOp execution.

  28. Skip checking of scan keys required for directional scan in B-tree

  29. Instead of using a numberOfRequiredKeys count to distinguish required

On 8/29/25 10:38, Matthias van de Meent wrote:
> Hi,
> 
> On Thu, 22 May 2025 at 19:57, Matthias van de Meent
> <boekewurm+postgres@gmail.com> wrote:
>>
>> On Tue, 20 May 2025, 22:14 Peter Geoghegan, <pg@bowt.ie> wrote:
>>>
>>> On Mon, May 12, 2025 at 8:58 AM Peter Geoghegan <pg@bowt.ie> wrote:
>>>> I wonder if we can fix this problem by getting rid of the old support
>>>> routine #5, "options". It currently doesn't do anything, and I always
>>>> thought it was strange that it was added "for uniformity" with other
>>>> index AMs.
>>>
>>> Attached patch completely removes the nbtree "options" support
>>> function, while changing the support function number of skip support:
>>> it becomes support function #5 (the number previously used by
>>> "options"). This patch should fix the regression that Tomas complained
>>> about in an expedient way.
>>>
>>> It's likely that somebody else will run into the same problem in the
>>> future, the next time that a new support function is needed. But I
>>> think that it makes sense to do this much now -- we need a short term
>>> solution for Postgres 18.
> 
> I just realized I hadn't checked in on this in a while, and I haven't
> seen Peter's patch get committed, nor my 0001. Do we consider this an
> Open Item and should this be improved in PG18, or is this something
> the user is expected to figure out and configure their systems for?
> 
> If we want to fix it let's make a decision before RC1, so we don't
> have further breaking catalog changes between RC1 and 18.0.
> 

The thing is that if we want to make this to RC1, it needs to go in
*today*. The RC1 is planned for next week, and there's a freeze starting
tomorrow.

> cc-ed RMT as this might be Open Item-worthy, and the patches up for
> debate both change catalog behaviour.
> 

I agree with this, personally. Maybe the other RMT members will see it
differently, but it's probably to add an open item and then remove it
than miss an issue.

> Peter's patch at [0] changes opclass procedure numbers to reuse an
> existing but unused options regproc number.
> My 0001 at [1] changes the memory residence status of index access
> methods' handler_function output to const static, from dynamic in
> memctx.
> 

IIRC both approaches address the issue. I'd go with Peter's patch for
18. The other patch is much more invasive / bigger, and we're right
before RC1 freeze. Maybe it's a good idea, but I'd say it's for 19.

Peter, any thoughts on this. Do you think it's reasonable / feasible to
push the fix?


regards

-- 
Tomas Vondra