Horton

Re: [WIP] Effective storage of duplicates in B-tree index.

Anastasia Lubennikova <a.lubennikova@postgrespro.ru>

View thread View on postgr.es
From: Anastasia Lubennikova <a.lubennikova@postgrespro.ru>
To: David Steele <david@pgmasters.net>, pgsql-hackers@postgresql.org
Date: 2016-03-15T17:36:52Z
Lists: pgsql-hackers 
14.03.2016 16:02, David Steele:
> Hi Anastasia,
>
> On 2/18/16 12:29 PM, Anastasia Lubennikova wrote:
>> 18.02.2016 20:18, Anastasia Lubennikova:
>>> 04.02.2016 20:16, Peter Geoghegan:
>>>> On Fri, Jan 29, 2016 at 8:50 AM, Anastasia Lubennikova
>>>> <a.lubennikova@postgrespro.ru>  wrote:
>>>>> I fixed it in the new version (attached).
>>>
>>> Thank you for the review.
>>> At last, there is a new patch version 3.0. After some refactoring it
>>> looks much better.
>>> I described all details of the compression in this document
>>> https://goo.gl/50O8Q0 (the same text without pictures is attached in
>>> btc_readme_1.0.txt).
>>> Consider it as a rough copy of readme. It contains some notes about
>>> tricky moments of implementation and questions about future work.
>>> Please don't hesitate to comment it.
>>>
>> Sorry, previous patch was dirty. Hotfix is attached.
>
> This looks like an extremely valuable optimization for btree indexes 
> but unfortunately it is not getting a lot of attention. It still 
> applies cleanly for anyone interested in reviewing.
>

Thank you for attention.
I would be indebted to all reviewers, who can just try this patch on 
real data and workload (except WAL for now).
B-tree needs very much testing.

> It's not clear to me that you answered all of Peter's questions in 
> [1].  I understand that you've provided a README but it may not be 
> clear if the answers are in there (and where).

I described in README all the points Peter asked.
But I see that it'd be better to answer directly.
Thanks for reminding, I'll do it tomorrow.

> Also, at the end of the README it says:
>
> 13. Xlog. TODO.
>
> Does that mean the patch is not yet complete?

Yes, you're right.
Frankly speaking, I supposed that someone will help me with that stuff,
but now I almost completed it. I'll send updated patch in the next letter.

I'm still doubtful about some patch details. I mentioned them in readme 
(bold type).
But they are mostly about future improvements.

-- 
Anastasia Lubennikova
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Commits

  1. Teach pageinspect about nbtree deduplication.

  2. Doc: Fix deduplicate_items index term.

  3. Revise BTP_HAS_GARBAGE nbtree VACUUM comments.

  4. Remove unneeded "pin scan" nbtree VACUUM code.

  5. Cleanup code in reloptions.h regarding reloption handling

  6. Catch invalid typlens in a couple of places

  7. Compute XID horizon for page level index vacuum on primary.

  8. Make heap TID a tiebreaker nbtree index column.

  9. Avoid pin scan for replay of XLOG_BTREE_VACUUM in all cases

  10. Revert buggy optimization of index scans

  11. Restructure index access method API to hide most of it at the C level.

  12. Reduce pinning and buffer content locking for btree scans.

  13. Avoid scanning nulls at the beginning of a btree index scan.