Horton

Re: new heapcheck contrib module

Peter Geoghegan <pg@bowt.ie>

View thread View on postgr.es
From: Peter Geoghegan <pg@bowt.ie>
To: Alvaro Herrera <alvherre@2ndquadrant.com>
Cc: Mark Dilger <mark.dilger@enterprisedb.com>, Robert Haas <robertmhaas@gmail.com>, Andres Freund <andres@anarazel.de>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2020-05-14T00:01:55Z
Lists: pgsql-hackers 
On Wed, May 13, 2020 at 4:32 PM Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
> I think my initial approach for this would be to use a fuzzing tool that
> generates data blocks semi-randomly, then uses them as Postgres data
> pages somehow, and see what happens -- examine any resulting crashes and
> make individual judgement calls about the fix(es) necessary to prevent
> each of them.  I expect that many such pages would be rejected as
> corrupt by page header checks.

As I mentioned in my response to Robert earlier, that's more or less
been my experience with adversarial corruption generated using
pg_hexedit. Within nbtree, as well as heapam. I put a lot of work into
that tool, and have used it to simulate all kinds of weird scenarios.
I've done things like corrupt individual tuple header fields, swap
line pointers, create circular sibling links in indexes, corrupt
varlena headers, and corrupt line pointer flags/status bits. Postgres
itself rarely segfaults, and amcheck will only segfault with a truly
contrived input.

-- 
Peter Geoghegan

Commits

  1. Add pg_amcheck, a CLI for contrib/amcheck.

  2. Refactor and generalize the ParallelSlot machinery.

  3. Generalize parallel slot result handling.

  4. Move some code from src/bin/scripts to src/fe_utils to permit reuse.

  5. Factor pattern-construction logic out of processSQLNamePattern.

  6. Doc: clean up verify_heapam() documentation.

  7. Fix more portability issues in new amcheck code.

  8. Fix portability issues in new amcheck test.

  9. Try to avoid a compiler warning about using fxid uninitialized.

  10. Extend amcheck to check heap pages.

  11. Adjust walsender usage of xlogreader, simplify APIs

  12. Improve checking of child pages in contrib/amcheck.

  13. Sanitize line pointers within contrib/amcheck.

  14. Fix possible sorting error when aborting use of abbreviated keys.