Horton

Re: new heapcheck contrib module

Mark Dilger <mark.dilger@enterprisedb.com>

View thread View on postgr.es
From: Mark Dilger <mark.dilger@enterprisedb.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Robert Haas <robertmhaas@gmail.com>, Peter Geoghegan <pg@bowt.ie>, "Andrey M. Borodin" <x4mmm@yandex-team.ru>, Stephen Frost <sfrost@snowman.net>, Michael Paquier <michael@paquier.xyz>, Amul Sul <sulamul@gmail.com>, Dilip Kumar <dilipbalaut@gmail.com>, Andres Freund <andres@anarazel.de>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2020-10-22T21:18:59Z
Lists: pgsql-hackers 

> On Oct 22, 2020, at 2:06 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> 
> I wrote:
>> Mark Dilger <mark.dilger@enterprisedb.com> writes:
>>> It is seeking to position 32 and writing '\x77\x77\x77\x77'.  x86_64 is
>>> little-endian, and ppc32 and sparc64 are both big-endian, right?
> 
>> They are, but that should not meaningfully affect the results of
>> that corruption step.  You zapped only one line pointer not
>> several, but it would look the same regardless of endiannness.
> 
> Oh, wait a second.  ItemIdData has the flag bits in the middle:
> 
> typedef struct ItemIdData
> {
>    unsigned    lp_off:15,        /* offset to tuple (from start of page) */
>                lp_flags:2,       /* state of line pointer, see below */
>                lp_len:15;        /* byte length of tuple */
> } ItemIdData;
> 
> meaning that for that particular bit pattern, one endianness
> is going to see the flags as 01 (LP_NORMAL) and the other as 10
> (LP_REDIRECT).  The offset/len are corrupt either way, but
> I'd certainly expect that amcheck would produce different
> complaints about those two cases.  So it's unsurprising if
> this test case's output is endian-dependent.

Well, the issue is that on big-endian machines it is not reporting any corruption at all.  Are you sure the difference will be LP_NORMAL vs LP_REDIRECT?  I was thinking it was LP_DEAD vs LP_REDIRECT, as the little endian platforms are seeing corruption messages about bad redirect line pointers, and the big-endian are apparently skipping over the line pointer entirely, which makes sense if it is LP_DEAD but not if it is LP_NORMAL.  It would also skip over LP_UNUSED, but I don't see how that could be stored in lp_flags, because 0x77 is going to either be 01110111 or 11101110, and in neither case do you get two zeros adjacent, but you could get two ones adjacent.  (LP_UNUSED = binary 00 and LP_DEAD = binary 11)

—
Mark Dilger
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Commits

  1. Add pg_amcheck, a CLI for contrib/amcheck.

  2. Refactor and generalize the ParallelSlot machinery.

  3. Generalize parallel slot result handling.

  4. Move some code from src/bin/scripts to src/fe_utils to permit reuse.

  5. Factor pattern-construction logic out of processSQLNamePattern.

  6. Doc: clean up verify_heapam() documentation.

  7. Fix more portability issues in new amcheck code.

  8. Fix portability issues in new amcheck test.

  9. Try to avoid a compiler warning about using fxid uninitialized.

  10. Extend amcheck to check heap pages.

  11. Adjust walsender usage of xlogreader, simplify APIs

  12. Improve checking of child pages in contrib/amcheck.

  13. Sanitize line pointers within contrib/amcheck.

  14. Fix possible sorting error when aborting use of abbreviated keys.