Horton

Re: new heapcheck contrib module

Tom Lane <tgl@sss.pgh.pa.us>

View thread View on postgr.es
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Alvaro Herrera <alvherre@2ndquadrant.com>
Cc: Robert Haas <robertmhaas@gmail.com>, Peter Geoghegan <pg@bowt.ie>, Mark Dilger <mark.dilger@enterprisedb.com>, Andres Freund <andres@anarazel.de>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2020-05-14T19:50:52Z
Lists: pgsql-hackers 
Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> On 2020-May-14, Robert Haas wrote:
>> If you mean that we shouldn't have the buildfarm run the proposed heap
>> corruption checker against heap pages full of randomly-generated
>> garbage, I tend to agree. Such a test wouldn't be very stable and
>> might fail in lots of low-probability ways that could require
>> unreasonable effort to find and fix.

> This is what I meant.  I was thinking of blocks generated randomly.

Yeah, -1 for using random data --- when it fails, how you gonna
reproduce the problem?

>> If you mean that we shouldn't have the buildfarm run the proposed heap
>> corruption checker against any corrupted heap pages at all, I tend to
>> disagree.

> Yeah, IMV those would not be arbitrarily corrupted -- instead they're
> crafted to be corrupted in some specific way.

I think there's definitely value in corrupting data in some predictable
(reproducible) way and verifying that the check code catches it and
responds as expected.  Sure, this will not be 100% coverage, but it'll be
a lot better than 0% coverage.

			regards, tom lane

Commits

  1. Add pg_amcheck, a CLI for contrib/amcheck.

  2. Refactor and generalize the ParallelSlot machinery.

  3. Generalize parallel slot result handling.

  4. Move some code from src/bin/scripts to src/fe_utils to permit reuse.

  5. Factor pattern-construction logic out of processSQLNamePattern.

  6. Doc: clean up verify_heapam() documentation.

  7. Fix more portability issues in new amcheck code.

  8. Fix portability issues in new amcheck test.

  9. Try to avoid a compiler warning about using fxid uninitialized.

  10. Extend amcheck to check heap pages.

  11. Adjust walsender usage of xlogreader, simplify APIs

  12. Improve checking of child pages in contrib/amcheck.

  13. Sanitize line pointers within contrib/amcheck.

  14. Fix possible sorting error when aborting use of abbreviated keys.