Re: A design for amcheck heapam verification

Thomas Munro <thomas.munro@enterprisedb.com>

From: Thomas Munro <thomas.munro@enterprisedb.com>
To: Michael Paquier <michael.paquier@gmail.com>
Cc: Peter Geoghegan <pg@bowt.ie>, Alvaro Herrera <alvherre@2ndquadrant.com>, Tom Lane <tgl@sss.pgh.pa.us>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Date: 2017-09-29T03:34:09Z
Lists: pgsql-hackers
On Fri, Sep 29, 2017 at 4:17 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:
>> As for DSM, I think that that can come later, and can be written by
>> somebody closer to that problem. There can be more than one
>> initialization function.
>
> I don't completely disagree with that, there could be multiple
> initialization functions. Still, an advantage about designing things
> right from the beginning with a set of correct APIs is that we don't
> need extra things later and this will never bother module maintainers.
> I would think that this utility interface should be minimal and
> portable to maintain a long-term stance.

FWIW I think if I were attacking that problem the first thing I'd
probably try would be getting rid of that internal pointer
filter->bitset in favour of a FLEXIBLE_ARRAY_MEMBER and then making
the interface look something like this:

extern size_t bloom_estimate(int64 total elems, int work_mem);
extern void bloom_init(bloom_filter *filter, int64 total_elems, int work_mem);

Something that allocates new memory as the patch's bloom_init()
function does I'd tend to call 'make' or 'create' or 'new' or
something, rather than 'init'.  'init' has connotations of being the
second phase in an allocate-and-init pattern for me.  Then
bloom_filt_make() would be trivially implemented on top of
bloom_estimate() and bloom_init(), and bloom_init() could be used
directly in DSM, DSA, traditional shmem without having to add any
special DSM support.

-- 
Thomas Munro
http://www.enterprisedb.com


Commits

  1. Fix non-portable use of round().

  2. Add amcheck verification of heap relations belonging to btree indexes.

  3. Add Bloom filter implementation.

  4. Use ereport not elog for some corrupt-HOT-chain reports.

  5. Introduce 64-bit hash functions with a 64-bit seed.

  6. Upgrade src/port/rint.c to be POSIX-compliant.

  7. Use type "int64" for memory accounting in tuplesort.c/tuplestore.c.