Horton

Re: [HACKERS] [WIP] Effective storage of duplicates in B-tree index.

Anastasia Lubennikova <a.lubennikova@postgrespro.ru>

View thread View on postgr.es
From: Anastasia Lubennikova <a.lubennikova@postgrespro.ru>
To: Peter Geoghegan <pg@bowt.ie>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2019-08-23T11:45:47Z
Lists: pgsql-hackers 
23.08.2019 7:33, Peter Geoghegan wrote:
> On Wed, Aug 21, 2019 at 10:19 AM Anastasia Lubennikova
> <a.lubennikova@postgrespro.ru> wrote:
>> I'm going to look through the patch once more to update nbtxlog
>> comments, where needed and
>> answer to your remarks that are still left in the comments.
> Have you been using amcheck's rootdescend verification?

No, I haven't checked it with the latest version yet.

> There were many large indexes that amcheck didn't detect a problem
> with. I don't yet understand what the problem is, or why we only see
> the problem for a small number of indexes. Note that all of these
> indexes passed verification with v5, so this is some kind of
> regression.
>
> I also noticed that there were some regressions in the size of indexes
> -- indexes were not nearly as small as they were in v5 in some cases.
> The overall picture was a clear regression in how effective
> deduplication is.
Do these indexes have something in common? Maybe some specific workload?
Are there any error messages in log?

I'd like to specify what caused the problem.
There were several major changes between v5 and v8:
- dead tuples handling added in v6;
- _bt_split changes for posting tuples in v7;
- WAL logging of posting tuple changes in v8.

I don't think the last one could break regular indexes on master.
Do you see the same regression in v6, v7?

> I think that it would save time if you had direct access to my test
> data, even though it's a bit cumbersome. You'll have to download about
> 10GB of dumps, which require plenty of disk space when restored:
>
>
> Want me to send this data and the associated tests script over to you?
>
Yes, I think it will help me to debug the patch faster.

-- 
Anastasia Lubennikova
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Commits

  1. Teach pageinspect about nbtree deduplication.

  2. Doc: Fix deduplicate_items index term.

  3. Revise BTP_HAS_GARBAGE nbtree VACUUM comments.

  4. Remove unneeded "pin scan" nbtree VACUUM code.

  5. Cleanup code in reloptions.h regarding reloption handling

  6. Catch invalid typlens in a couple of places

  7. Compute XID horizon for page level index vacuum on primary.

  8. Make heap TID a tiebreaker nbtree index column.

  9. Avoid pin scan for replay of XLOG_BTREE_VACUUM in all cases

  10. Revert buggy optimization of index scans

  11. Restructure index access method API to hide most of it at the C level.

  12. Reduce pinning and buffer content locking for btree scans.

  13. Avoid scanning nulls at the beginning of a btree index scan.