Improve CRC32C performance on SSE4.2

Devulapalli, Raghuveer <raghuveer.devulapalli@intel.com>

From: "Devulapalli, Raghuveer" <raghuveer.devulapalli@intel.com>

To: "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>

Cc: "Shankaran, Akash" <akash.shankaran@intel.com>, "Devulapalli, Raghuveer" <raghuveer.devulapalli@intel.com>

Date: 2025-02-05T20:48:56Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Include _mm512_zextsi128_si512() in AVX-512 configure probes.
- ccd5bc93fdfe 18.0 landed
Properly fix AVX-512 CRC calculation bug
- 43da394304fb 18.0 landed
Workaround code generation bug in clang
- f83f14881c7a 18.0 landed
Compute CRC32C using AVX-512 instructions where available
- 3c6e8c123896 18.0 landed
Inline CRC computation for small fixed-length input on x86
- e2809e3a1015 18.0 landed
Be more paranoid in configure's checks for CRC and POPCNT intrinsics.
- fdb5dd6331e3 18.0 cited

Attachments

v1-0001-Add-more-test-coverage-for-crc32c.patch (application/octet-stream) patch v1-0001
v1-0002-Improve-CRC32C-performance-on-SSE4.2.patch (application/octet-stream) patch v1-0002

This patch improves the performance of SSE42 CRC32C algorithm. The current SSE4.2 implementation of CRC32C relies on the native crc32 instruction and processes 8 bytes at a time in a loop. The technique in  this paper uses the pclmulqdq instruction and processing 64 bytes at time. The algorithm is based on sse42 version of crc32 computation from Chromium's copy of zlib with modified constants for crc32c computation. See:

https://chromium.googlesource.com/chromium/src/+/refs/heads/main/third_party/zlib/crc32_simd.c

Microbenchmarks (generated with google benchmark using a standalone version of the same algorithms):

Comparing scalar_crc32c to sse42_crc32c (for various buffer sizes: 64, 128, 256, 512, 1024, 2048 bytes)
Benchmark                                               Time             CPU      Time Old      Time New       CPU Old       CPU New
------------------------------------------------------------------------------------------------------------------------------------
[scalar_crc32c vs. sse42_crc32c]/64                  -0.8147         -0.8148            33             6            33             6
[scalar_crc32c vs. sse42_crc32c]/128                -0.8962         -0.8962            88             9            88             9
[scalar_crc32c vs. sse42_crc32c]/256                -0.9200         -0.9200           211            17           211            17
[scalar_crc32c vs. sse42_crc32c]/512                -0.9389         -0.9389           486            30           486            30
[scalar_crc32c vs. sse42_crc32c]/1024              -0.9452         -0.9452          1037            57          1037            57
[scalar_crc32c vs. sse42_crc32c]/2048             -0.9456         -0.9456          2140           116          2140           116

Raghuveer