Horton

Re: pg_amcheck contrib application

Noah Misch <noah@leadboat.com>

View thread View on postgr.es
From: Noah Misch <noah@leadboat.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Mark Dilger <mark.dilger@enterprisedb.com>, Robert Haas <robertmhaas@gmail.com>, Peter Geoghegan <pg@bowt.ie>, Thomas Munro <thomas.munro@gmail.com>, "Andrey M. Borodin" <x4mmm@yandex-team.ru>, Stephen Frost <sfrost@snowman.net>, Michael Paquier <michael@paquier.xyz>, Amul Sul <sulamul@gmail.com>, Dilip Kumar <dilipbalaut@gmail.com>, Andres Freund <andres@anarazel.de>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2021-03-16T06:09:13Z
Lists: pgsql-hackers 
On Mon, Mar 15, 2021 at 02:57:20PM -0400, Tom Lane wrote:
> Mark Dilger <mark.dilger@enterprisedb.com> writes:
> > On Mar 15, 2021, at 10:04 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >> These animals have somewhat weird alignment properties: MAXALIGN is 8
> >> but ALIGNOF_DOUBLE is only 4.  I speculate that that is affecting their
> >> choices about whether an out-of-line TOAST value is needed, breaking
> >> this test case.

That machine also has awful performance for filesystem metadata operations,
like open(O_CREAT).  Its CPU and read()/write() performance are normal.

> > The logic in verify_heapam only looks for a value in the toast table if
> > the tuple it gets from the main table (in this case, from pg_statistic)
> > has an attribute that claims to be toasted.  The error message we're
> > seeing that you quoted above simply means that no entry exists in the
> > toast table.
> 
> Yeah, that could be phrased better.  Do we have a strong enough lock on
> the table under examination to be sure that autovacuum couldn't remove
> a dead toast entry before we reach it?  But this would only be an
> issue if we are trying to check validity of toasted fields within
> known-dead tuples, which I would argue we shouldn't, since lock or
> no lock there's no guarantee the toast entry is still there.
> 
> Not sure that I believe the theory that this is from bad luck of
> concurrent autovacuum timing, though.

With autovacuum_naptime=1s, on hornet, the failure reproduced twice in twelve
runs.  With v6-0001-Turning-off-autovacuum-during-corruption-tests.patch
applied, 196 runs all succeeded.

> The fact that we're seeing
> this on just those two animals suggests strongly to me that it's
> architecture-correlated, instead.

That is possible.

Commits

  1. amcheck: Improve some confusing reports about TOAST problems.

  2. amcheck: Reword some messages and fix an alignment problem.

  3. amcheck: fix multiple problems with TOAST pointer validation

  4. amcheck: Remove duplicate XID/MXID bounds checks.

  5. amcheck: Fix verify_heapam's tuple visibility checking rules.

  6. nbtree VACUUM: Cope with buggy opclasses.

  7. Improve pg_amcheck's TAP test 003_check.pl.

  8. Fix a confusing amcheck corruption message.

  9. Doc: add note about how to run the pg_amcheck regression tests.

  10. In pg_amcheck tests, don't depend on perl's Q/q pack code.

  11. pg_amcheck: Keep trying to fix the tests.

  12. pg_amcheck: Try to fix still more test failures.

  13. Try to avoid apparent platform-dependency in IPC::Run

  14. Fix portability issues in pg_amcheck's 004_verify_heapam.pl.

  15. Try to fix compiler warnings.

  16. Add pg_amcheck, a CLI for contrib/amcheck.

  17. Refactor and generalize the ParallelSlot machinery.

  18. Remove old-style VACUUM FULL (which was known for a little while as