Re: Proposal for Updating CRC32C with AVX-512 Algorithm.

John Naylor <johncnaylorls@gmail.com>

From: John Naylor <johncnaylorls@gmail.com>

To: Bruce Momjian <bruce@momjian.us>

Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>, Andres Freund <andres@anarazel.de>, "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>, Nathan Bossart <nathandbossart@gmail.com>, "Shankaran, Akash" <akash.shankaran@intel.com>, "Devulapalli, Raghuveer" <raghuveer.devulapalli@intel.com>

Date: 2024-12-12T11:32:20Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Compute CRC32C using AVX-512 instructions where available
- 3c6e8c123896 18.0 landed

+ * For This Function:
+ * Copyright 2015 The Chromium Authors

I went and looked at the Chromium source, and found the following
snippet that uses the same technique, but only requires 128-bit CLMUL
and has a minimum input size of 64 bytes, rather than 256. This seems
like it might be better suited for shorter inputs. Also seems much
easier than trying to get the AVX-512 hippo to dance. It uses the IEEE
polynomial, so would need new constants calculated for ours, but that
had to be done for the shared patch, too.

https://github.com/chromium/chromium/blob/main/third_party/zlib/crc32_simd.c#L215

-- 
John Naylor
Amazon Web Services