Re: Enabling Checksums

Florian Pflug <fgp@phlo.org>

From: Florian Pflug <fgp@phlo.org>
To: Ants Aasma <ants@cybertec.at>
Cc: Andres Freund <andres@2ndquadrant.com>, Tom Lane <tgl@sss.pgh.pa.us>, Bruce Momjian <bruce@momjian.us>, Jeff Davis <pgsql@j-davis.com>, Heikki Linnakangas <hlinnakangas@vmware.com>, Simon Riggs <simon@2ndquadrant.com>, Greg Smith <greg@2ndquadrant.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2013-04-18T18:11:28Z
Lists: pgsql-hackers
On 18.04.2013, at 20:02, Ants Aasma <ants@cybertec.at> wrote:
> On Thu, Apr 18, 2013 at 8:24 PM, Ants Aasma <ants@cybertec.at> wrote:
>> On Thu, Apr 18, 2013 at 8:15 PM, Florian Pflug <fgp@phlo.org> wrote:
>>> So either the CRC32-C polynomial isn't irreducible, or there something
>>> fishy going on. Could there be a bug in your CRC implementation? Maybe
>>> a mixup between big and little endian, or something like that?
>> 
>> I'm suspecting an implementation bug myself. I already checked the
>> test harness and that was all sane, compiler hadn't taken any
>> unforgivable liberties there. I will crosscheck the output with other
>> implementations to verify that the checksum is implemented correctly.
> 
> Looks like the implementation is correct. I cross-referenced it
> against a bitwise algorithm for crc32 with the castagnoli polynomial.
> This also rules out any endianness issues as the bitwise variant
> consumes input byte at a time.
> 
> What ever it is, it is something specific to PostgreSQL page layout.
> If I use /dev/urandom as the source the issue disappears. So much for
> CRC32 being proven good.

Weird. Is the code of your test harness available publicly, or could you post it? I'd like to look into this...

best regard,
Florian Pflug