Horton

Re: pg_amcheck contrib application

Mark Dilger <mark.dilger@enterprisedb.com>

View thread View on postgr.es
From: Mark Dilger <mark.dilger@enterprisedb.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Noah Misch <noah@leadboat.com>, Peter Geoghegan <pg@bowt.ie>, Thomas Munro <thomas.munro@gmail.com>, "Andrey M. Borodin" <x4mmm@yandex-team.ru>, Stephen Frost <sfrost@snowman.net>, Michael Paquier <michael@paquier.xyz>, Amul Sul <sulamul@gmail.com>, Dilip Kumar <dilipbalaut@gmail.com>, Andres Freund <andres@anarazel.de>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2021-04-23T18:32:39Z
Lists: pgsql-hackers 

> On Apr 23, 2021, at 11:29 AM, Robert Haas <robertmhaas@gmail.com> wrote:
> 
> On Fri, Apr 23, 2021 at 2:15 PM Mark Dilger
> <mark.dilger@enterprisedb.com> wrote:
>> Another difference I should probably mention is that a bunch of unrelated tests are failing with errors like:
>> 
>>    toast value 13465 chunk 0 has size 1995, but expected size 1996
>> 
>> which leads me to suspect your changes to how the size is calculated.
> 
> That seems like a pretty reasonable suspicion, but I can't see the problem:
> 
> -       expected_size = curchunk < endchunk ? TOAST_MAX_CHUNK_SIZE
> -               : VARATT_EXTERNAL_GET_EXTSIZE(ta->toast_pointer) -
> (endchunk * TOAST_MAX_CHUNK_SIZE);
> +       expected_size = chunk_seq < last_chunk_seq ? TOAST_MAX_CHUNK_SIZE
> +               : extsize % TOAST_MAX_CHUNK_SIZE;
> 
> What's different?
> 
> 1. The variables are renamed.
> 
> 2. It uses a new variable extsize instead of recomputing
> VARATT_EXTERNAL_GET_EXTSIZE(ta->toast_pointer), but I think that
> should have the same value.
> 
> 3. I used modulo arithmetic (%) instead of subtracting endchunk *
> TOAST_MAX_CHUNK_SIZE.
> 
> Is TOAST_MAX_CHUNK_SIZE 1996? How long a value did you insert?

On  my laptop, yes, 1996 is TOAST_MAX_CHUNK_SIZE.

I'm not inserting anything.  These failures come from just regular tests that I have not changed.  I just applied your patch and ran `make check-world` and these fail in src/bin/pg_amcheck 

—
Mark Dilger
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Commits

  1. amcheck: Improve some confusing reports about TOAST problems.

  2. amcheck: Reword some messages and fix an alignment problem.

  3. amcheck: fix multiple problems with TOAST pointer validation

  4. amcheck: Remove duplicate XID/MXID bounds checks.

  5. amcheck: Fix verify_heapam's tuple visibility checking rules.

  6. nbtree VACUUM: Cope with buggy opclasses.

  7. Improve pg_amcheck's TAP test 003_check.pl.

  8. Fix a confusing amcheck corruption message.

  9. Doc: add note about how to run the pg_amcheck regression tests.

  10. In pg_amcheck tests, don't depend on perl's Q/q pack code.

  11. pg_amcheck: Keep trying to fix the tests.

  12. pg_amcheck: Try to fix still more test failures.

  13. Try to avoid apparent platform-dependency in IPC::Run

  14. Fix portability issues in pg_amcheck's 004_verify_heapam.pl.

  15. Try to fix compiler warnings.

  16. Add pg_amcheck, a CLI for contrib/amcheck.

  17. Refactor and generalize the ParallelSlot machinery.

  18. Remove old-style VACUUM FULL (which was known for a little while as