RE: Proposal for Updating CRC32C with AVX-512 Algorithm.

Amonson, Paul D <paul.d.amonson@intel.com>

From: "Amonson, Paul D" <paul.d.amonson@intel.com>

To: Nathan Bossart <nathandbossart@gmail.com>

Cc: Bruce Momjian <bruce@momjian.us>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Andres Freund <andres@anarazel.de>, "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>, "Shankaran, Akash" <akash.shankaran@intel.com>

Date: 2024-08-26T17:09:35Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Compute CRC32C using AVX-512 instructions where available
- 3c6e8c123896 18.0 landed

Attachments

0001-v3-Refactor-Move-all-HW-checks-to-common-file.patch (application/octet-stream) patch v3-0001
0002-v3-Feat-Add-support-for-the-SIMD-AVX-512-crc32c-algorit.patch (application/octet-stream) patch v3-0002
0003-v3-Feat-Targeted-use-of-legacy-crc32c.patch (application/octet-stream) patch v3-0003

> Upthread [0], Andres suggested dispatching to a different implementation for
> compile-time-known small lengths.  Have you looked into that?  In your
> original post, you noted a 14% regression for records smaller than 256 bytes,
> which is not an uncommon case for Postgres.  IMO we should try to mitigate
> that as much as possible.

Hi,

Ok I added a patch that exposed a new macro CRC32C_COMP_SMALL for targeted fixed size < 256 use cases in Postgres. As for mitigating the regression in general, I have not been able to work up a fallback (i.e. <256 bytes) that doesn't involve runtime checks which cause latency. I also attempted to change the AVX512 fallback from the current algorithm in the avx512 implementation to the SSE original implementation, but I am not seeing any real difference for this use case in performance.

I am open to any other suggestions.

Paul