Re: A design for amcheck heapam verification

Peter Geoghegan <pg@bowt.ie>

From: Peter Geoghegan <pg@bowt.ie>
To: Alvaro Herrera <alvherre@2ndquadrant.com>
Cc: Thomas Munro <thomas.munro@enterprisedb.com>, Tom Lane <tgl@sss.pgh.pa.us>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Date: 2017-09-16T17:54:45Z
Lists: pgsql-hackers
On Wed, Sep 6, 2017 at 7:26 PM, Peter Geoghegan <pg@bowt.ie> wrote:
> On Wed, Aug 30, 2017 at 9:29 AM, Peter Geoghegan <pg@bowt.ie> wrote:
>> On Wed, Aug 30, 2017 at 5:02 AM, Alvaro Herrera
>> <alvherre@2ndquadrant.com> wrote:
>>> Eh, if you want to optimize it for the case where debug output is not
>>> enabled, make sure to use ereport() not elog().  ereport()
>>> short-circuits evaluation of arguments, whereas elog() does not.
>>
>> I should do that, but it's still not really noticeable.
>
> Since this patch has now bit-rotted, I attach a new revision, V2.

I should point out that I am relying on deterministic TOAST
compression within index_form_tuple() at present. This could, in
theory, become a problem later down the road, when
toast_compress_datum() compression becomes configurable via a storage
parameter or something (e.g., we use PGLZ_strategy_always, rather than
the hard coded PGLZ_strategy_default strategy).

While I should definitely have a comment above the new amcheck
index_form_tuple() call that points this out, it's not clear if that's
all that is required. Normalizing the representation of hashed index
tuples to make verification robust against unforeseen variation in
TOAST compression strategy seems like needless complexity to me, but
I'd like to hear a second opinion on that.

-- 
Peter Geoghegan


Commits

  1. Fix non-portable use of round().

  2. Add amcheck verification of heap relations belonging to btree indexes.

  3. Add Bloom filter implementation.

  4. Use ereport not elog for some corrupt-HOT-chain reports.

  5. Introduce 64-bit hash functions with a 64-bit seed.

  6. Upgrade src/port/rint.c to be POSIX-compliant.

  7. Use type "int64" for memory accounting in tuplesort.c/tuplestore.c.