Horton

Re: new heapcheck contrib module

Mark Dilger <mark.dilger@enterprisedb.com>

View thread View on postgr.es
From: Mark Dilger <mark.dilger@enterprisedb.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Robert Haas <robertmhaas@gmail.com>, Peter Geoghegan <pg@bowt.ie>, "Andrey M. Borodin" <x4mmm@yandex-team.ru>, Stephen Frost <sfrost@snowman.net>, Michael Paquier <michael@paquier.xyz>, Amul Sul <sulamul@gmail.com>, Dilip Kumar <dilipbalaut@gmail.com>, Andres Freund <andres@anarazel.de>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2020-10-22T21:10:55Z
Lists: pgsql-hackers 

> On Oct 22, 2020, at 2:06 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> 
> I wrote:
>> Mark Dilger <mark.dilger@enterprisedb.com> writes:
>>> It is seeking to position 32 and writing '\x77\x77\x77\x77'.  x86_64 is
>>> little-endian, and ppc32 and sparc64 are both big-endian, right?
> 
>> They are, but that should not meaningfully affect the results of
>> that corruption step.  You zapped only one line pointer not
>> several, but it would look the same regardless of endiannness.
> 
> Oh, wait a second.  ItemIdData has the flag bits in the middle:
> 
> typedef struct ItemIdData
> {
>    unsigned    lp_off:15,        /* offset to tuple (from start of page) */
>                lp_flags:2,       /* state of line pointer, see below */
>                lp_len:15;        /* byte length of tuple */
> } ItemIdData;
> 
> meaning that for that particular bit pattern, one endianness
> is going to see the flags as 01 (LP_NORMAL) and the other as 10
> (LP_REDIRECT).  The offset/len are corrupt either way, but
> I'd certainly expect that amcheck would produce different
> complaints about those two cases.  So it's unsurprising if
> this test case's output is endian-dependent.

Yeah, I'm already looking at that.  The logic in verify_heapam skips over line pointers that are unused or dead, and the test is reporting zero corruption (and complaining about that), so it's probably not going to help to overwrite all the line pointers with this particular bit pattern any more than to just overwrite the first one, as it would just skip them all.

I think the test should overwrite the line pointers with a variety of different bit patterns, or one calculated to work on all platforms.  I'll have to write that up.


—
Mark Dilger
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Commits

  1. Add pg_amcheck, a CLI for contrib/amcheck.

  2. Refactor and generalize the ParallelSlot machinery.

  3. Generalize parallel slot result handling.

  4. Move some code from src/bin/scripts to src/fe_utils to permit reuse.

  5. Factor pattern-construction logic out of processSQLNamePattern.

  6. Doc: clean up verify_heapam() documentation.

  7. Fix more portability issues in new amcheck code.

  8. Fix portability issues in new amcheck test.

  9. Try to avoid a compiler warning about using fxid uninitialized.

  10. Extend amcheck to check heap pages.

  11. Adjust walsender usage of xlogreader, simplify APIs

  12. Improve checking of child pages in contrib/amcheck.

  13. Sanitize line pointers within contrib/amcheck.

  14. Fix possible sorting error when aborting use of abbreviated keys.