Horton

Re: [WIP] Effective storage of duplicates in B-tree index.

David Steele <david@pgmasters.net>

View thread View on postgr.es
From: David Steele <david@pgmasters.net>
To: Anastasia Lubennikova <a.lubennikova@postgrespro.ru>, pgsql-hackers@postgresql.org
Date: 2016-03-14T13:02:06Z
Lists: pgsql-hackers 
Hi Anastasia,

On 2/18/16 12:29 PM, Anastasia Lubennikova wrote:
> 18.02.2016 20:18, Anastasia Lubennikova:
>> 04.02.2016 20:16, Peter Geoghegan:
>>> On Fri, Jan 29, 2016 at 8:50 AM, Anastasia Lubennikova
>>> <a.lubennikova@postgrespro.ru>  wrote:
>>>> I fixed it in the new version (attached).
>>
>> Thank you for the review.
>> At last, there is a new patch version 3.0. After some refactoring it
>> looks much better.
>> I described all details of the compression in this document
>> https://goo.gl/50O8Q0 (the same text without pictures is attached in
>> btc_readme_1.0.txt).
>> Consider it as a rough copy of readme. It contains some notes about
>> tricky moments of implementation and questions about future work.
>> Please don't hesitate to comment it.
>>
> Sorry, previous patch was dirty. Hotfix is attached.

This looks like an extremely valuable optimization for btree indexes but 
unfortunately it is not getting a lot of attention.  It still applies 
cleanly for anyone interested in reviewing.

It's not clear to me that you answered all of Peter's questions in [1]. 
  I understand that you've provided a README but it may not be clear if 
the answers are in there (and where).

Also, at the end of the README it says:

13. Xlog. TODO.

Does that mean the patch is not yet complete?

Thanks,
-- 
-David
david@pgmasters.net

[1] 
http://www.postgresql.org/message-id/CAM3SWZQ3_PLQCH4w7uQ8q_f2t4HEseKTr2n0rQ5pxA18OeRTJw@mail.gmail.com

Commits

  1. Teach pageinspect about nbtree deduplication.

  2. Doc: Fix deduplicate_items index term.

  3. Revise BTP_HAS_GARBAGE nbtree VACUUM comments.

  4. Remove unneeded "pin scan" nbtree VACUUM code.

  5. Cleanup code in reloptions.h regarding reloption handling

  6. Catch invalid typlens in a couple of places

  7. Compute XID horizon for page level index vacuum on primary.

  8. Make heap TID a tiebreaker nbtree index column.

  9. Avoid pin scan for replay of XLOG_BTREE_VACUUM in all cases

  10. Revert buggy optimization of index scans

  11. Restructure index access method API to hide most of it at the C level.

  12. Reduce pinning and buffer content locking for btree scans.

  13. Avoid scanning nulls at the beginning of a btree index scan.