Re: Cost of XLogInsert CRC calculations

> -----Original Message-----
> From: Mark Cave-Ayland [mailto:m.cave-ayland@webbased.co.uk]
> Sent: 07 March 2005 11:04
> To: 'tgl@sss.pgh.pa.us'
> Cc: 'pgsql-hackers@postgreSQL.org'
> Subject: Re: Cost of XLogInsert CRC calculations

(cut)

> > I suppose that the bulk of the CPU cycles being attributed to
> > XLogInsert are going into > the inlined CRC calculations.  Maybe we 
> > need to think twice about the cost/benefit ratio > of using 64-bit 
> > CRCs to protect xlog records that are often only a few dozen bytes.
> 
> Wow, a 64-bit CRC does seem excessive, especially when going
> back to Zmodem days where a 50-100k file seemed to be easily 
> protected by a 32-bit CRC. I'm sure there are some error 
> rates somewhere dependent upon the polynomial and the types 
> of error detected.... Try the following link towards the 
> bottom: http://www.ee.unb.ca/tervo/ee4253/crc.htm for some 
> theory on detection rates vs. CRC size.


Hi Tom/Simon,

I was just researching some articles on compression (zlib) and I saw mention
of the Adler-32 algorithm which is supposed to be slightly less accurate
than an equivalent CRC calculation but significantly faster to compute. I
haven't located a good paper comparing the error rates of the two different
checksums, however extending the Adler-32 algorithm up to 64 bits may be a
way of increasing the speed at the expense of a slight loss in the accuracy
of error detection if we decided that we want to stay at 64 bits as opposed
to 32. 

The following seems to indicate that Adler-32 is at least twice as fast as
optimised CRC32: http://www.winimage.com/misc/readfile_test.htm.


Kind regards,

Mark.

------------------------
WebBased Ltd
17 Research Way
Plymouth
PL6 8BT 

T: +44 (0)1752 791021
F: +44 (0)1752 791023
W: http://www.webbased.co.uk