Horton

Re: new heapcheck contrib module

Mark Dilger <mark.dilger@enterprisedb.com>

View thread View on postgr.es
From: Mark Dilger <mark.dilger@enterprisedb.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Robert Haas <robertmhaas@gmail.com>, Peter Geoghegan <pg@bowt.ie>, "Andrey M. Borodin" <x4mmm@yandex-team.ru>, Stephen Frost <sfrost@snowman.net>, Michael Paquier <michael@paquier.xyz>, Amul Sul <sulamul@gmail.com>, Dilip Kumar <dilipbalaut@gmail.com>, Andres Freund <andres@anarazel.de>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2020-10-23T01:47:50Z
Lists: pgsql-hackers 

> On Oct 22, 2020, at 6:41 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> 
> I wrote:
>> So now I think this is a REDIRECT on either architecture, but the
>> offset and length fields have different values, causing the redirect
>> pointer to point to different places.  Maybe it happens to point
>> at a DEAD tuple in the big-endian case.
> 
> Just to make sure, I tried this test program:
> 
> #include <stdio.h>
> #include <string.h>
> 
> typedef struct ItemIdData
> {
>    unsigned    lp_off:15,      /* offset to tuple (from start of page) */
>                lp_flags:2,     /* state of line pointer, see below */
>                lp_len:15;      /* byte length of tuple */
> } ItemIdData;
> 
> int main()
> {
>    ItemIdData lp;
> 
>    memset(&lp, 0x77, sizeof(lp));
>    printf("off = %x, flags = %x, len = %x\n",
>           lp.lp_off, lp.lp_flags, lp.lp_len);
>    return 0;
> }
> 
> I get
> 
> off = 7777, flags = 2, len = 3bbb
> 
> on a little-endian machine, and
> 
> off = 3bbb, flags = 2, len = 7777
> 
> on big-endian.  It'd be less symmetric if the bytes weren't
> all the same ...

I think we're going in the wrong direction here.  The idea behind this test was to have as little knowledge about the layout of pages as possible and still verify that damaging the pages would result in corruption reports.  Of course, not all damage will result in corruption reports, because some damage looks legit.  I think it was just luck (good or bad depending on your perspective) that the damage in the test as committed works on little-endian but not big-endian.

I can embed this knowledge that you have researched into the test if you want me to, but my instinct is to go the other direction and have even less knowledge about pages in the test.  That would work if instead of expecting corruption for every time the test writes the file, instead to have it just make sure that it gets corruption reports at least some of the times that it does so.  That seems more maintainable long term.

—
Mark Dilger
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Commits

  1. Add pg_amcheck, a CLI for contrib/amcheck.

  2. Refactor and generalize the ParallelSlot machinery.

  3. Generalize parallel slot result handling.

  4. Move some code from src/bin/scripts to src/fe_utils to permit reuse.

  5. Factor pattern-construction logic out of processSQLNamePattern.

  6. Doc: clean up verify_heapam() documentation.

  7. Fix more portability issues in new amcheck code.

  8. Fix portability issues in new amcheck test.

  9. Try to avoid a compiler warning about using fxid uninitialized.

  10. Extend amcheck to check heap pages.

  11. Adjust walsender usage of xlogreader, simplify APIs

  12. Improve checking of child pages in contrib/amcheck.

  13. Sanitize line pointers within contrib/amcheck.

  14. Fix possible sorting error when aborting use of abbreviated keys.