Horton

new heapcheck contrib module

Mark Dilger <mark.dilger@enterprisedb.com>

View thread View on postgr.es
From: Mark Dilger <mark.dilger@enterprisedb.com>
To: PostgreSQL-development <pgsql-hackers@postgresql.org>
Cc: Robert Haas <robert.haas@enterprisedb.com>
Date: 2020-04-20T17:59:28Z
Lists: pgsql-hackers

Attachments

Hackers,

I have been talking with Robert about table corruption that occurs from time to time. The page checksum feature seems sufficient to detect most random corruption problems, but it can't detect "logical" corruption, where the page is valid but inconsistent with the rest of the database cluster. This can happen due to faulty or ill-conceived backup and restore tools, or bad storage, or user error, or bugs in the server itself. (Also, not everyone enables checksums.)

The attached module provides the means to scan a relation and sanity check it. Currently, it checks xmin and xmax values against relfrozenxid and relminmxid, and also validates TOAST pointers. If people like this, it could be expanded to perform additional checks.

There was a prior v1 patch, discussed offlist with Robert, not posted.  Here is v2:

Commits

  1. Add pg_amcheck, a CLI for contrib/amcheck.

  2. Refactor and generalize the ParallelSlot machinery.

  3. Generalize parallel slot result handling.

  4. Move some code from src/bin/scripts to src/fe_utils to permit reuse.

  5. Factor pattern-construction logic out of processSQLNamePattern.

  6. Doc: clean up verify_heapam() documentation.

  7. Fix more portability issues in new amcheck code.

  8. Fix portability issues in new amcheck test.

  9. Try to avoid a compiler warning about using fxid uninitialized.

  10. Extend amcheck to check heap pages.

  11. Adjust walsender usage of xlogreader, simplify APIs

  12. Improve checking of child pages in contrib/amcheck.

  13. Sanitize line pointers within contrib/amcheck.

  14. Fix possible sorting error when aborting use of abbreviated keys.