Re: Making all nbtree entries unique by having heap TIDs participate in comparisons
Peter Geoghegan <pg@bowt.ie>
From: Peter Geoghegan <pg@bowt.ie>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Heikki Linnakangas <hlinnaka@iki.fi>,
Peter Eisentraut <peter.eisentraut@2ndquadrant.com>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>, Alexander Korotkov <a.korotkov@postgrespro.ru>,
Thomas Munro <thomas.munro@enterprisedb.com>, Claudio Freire <klaussfreire@gmail.com>, Anastasia Lubennikova <a.lubennikova@postgrespro.ru>, "Andrey V. Lepikhov" <a.lepikhov@postgrespro.ru>
Date: 2018-11-24T23:13:42Z
Lists: pgsql-hackers
Attachments
- v8-0003-Pick-nbtree-split-points-discerningly.patch (text/x-patch) patch v8-0003
- v8-0004-Add-split-at-new-tuple-page-split-optimization.patch (text/x-patch) patch v8-0004
- v8-0005-Add-high-key-continuescan-optimization.patch (text/x-patch) patch v8-0005
- v8-0006-DEBUG-Add-pageinspect-instrumentation.patch (text/x-patch) patch v8-0006
- v8-0002-Treat-heap-TID-as-part-of-the-nbtree-key-space.patch (text/x-patch) patch v8-0002
- v8-0001-Add-pg_depend-index-scan-tiebreaker-column.patch (text/x-patch) patch v8-0001
Attached is v8 of the patch series, which has some relatively minor changes: * A new commit adds an artificial tie-breaker column to pg_depend indexes, comprehensively solving the issues with regression test instability. This is the only really notable change. * Clean-up of how the design in described in the nbtree README, and elsewhere. I want to make it clear that we're now more or less using the Lehman and Yao design. I re-read the Lehman and Yao paper to make sure that the patch acknowledges what Lehman and Yao say to expect, at least in cases that seemed to matter. * Stricter verification by contrib/amcheck. Not likely to catch a case that wouldn't have been caught by previous revisions, but should make the design a bit clearer to somebody following L&Y. * Tweaks to how _bt_findsplitloc() accumulates candidate split points. We're less aggressive in choosing a smaller tuple during an internal page split in this revision. The overall impact of the pg_depend change is that required regression test output changes are *far* less numerous than they were in v7. There are now only trivial differences in the output order of items. And, there are very few diagnostic message changes overall -- we see exactly 5 changes now, rather than dozens. Importantly, there is no longer any question about whether I could make diagnostic messages less useful to users, because the existing behavior for findDependentObjects() is retained. This is an independent improvement, since it fixes an independent problem with test flappiness that we've been papering-over for some time [2] -- I make the required order actually-deterministic, removing heap TID ordering as a factor that can cause seemingly-random regression test failures on slow/overloaded buildfarm animals. Robert Haas remarked that he thought that the pg_depend index tie-breaker commit's approach is acceptable [1] -- see the other thread that Robert weighed in on for all the gory details. The patch's draft commit message may also be interesting. Note that adding a new column turns out to have *zero* storage overhead, because we only ever end up filling up space that was already getting lost to alignment. The pg_depend thing is clearly a kludge. It's ugly, though in no small part because it acknowledges the existing reality of how findDependentObjects() already depends on scan order. I'm optimistic that I'll be able to push this groundwork commit before too long; it doesn't hinge on whether or not the nbtree patches are any good. [1] https://postgr.es/m/CA+TgmoYNeFxdPimiXGL=tCiCXN8zWosUFxUfyDBaTd2VAg-D9w@mail.gmail.com [2] https://postgr.es/m/11852.1501610262%40sss.pgh.pa.us -- Peter Geoghegan
Commits
-
Add "split after new tuple" nbtree optimization.
- f21668f328c8 12.0 landed
-
Add nbtree high key "continuescan" optimization.
- 29b64d1de7c7 12.0 landed
-
Allow amcheck to re-find tuples using new search.
- c1afd175b5b2 12.0 landed
-
Consider secondary factors during nbtree splits.
- fab250243387 12.0 landed
-
Make heap TID a tiebreaker nbtree index column.
- dd299df8189b 12.0 landed
-
Refactor nbtree insertion scankeys.
- e5adcb789d80 12.0 landed
-
Redesign the partition dependency mechanism.
- 1d92a0c9f7dd 12.0 cited
-
Avoid unnecessary palloc overhead in _bt_first(). The temporary
- d961a5689966 8.1.0 cited