Re: Improve CRC32C performance on SSE4.2

John Naylor <johncnaylorls@gmail.com>

From: John Naylor <johncnaylorls@gmail.com>

To: Nathan Bossart <nathandbossart@gmail.com>

Cc: "Devulapalli, Raghuveer" <raghuveer.devulapalli@intel.com>, "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>, "Shankaran, Akash" <akash.shankaran@intel.com>

Date: 2025-02-25T11:05:17Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Include _mm512_zextsi128_si512() in AVX-512 configure probes.
- ccd5bc93fdfe 18.0 landed
Properly fix AVX-512 CRC calculation bug
- 43da394304fb 18.0 landed
Workaround code generation bug in clang
- f83f14881c7a 18.0 landed
Compute CRC32C using AVX-512 instructions where available
- 3c6e8c123896 18.0 landed
Inline CRC computation for small fixed-length input on x86
- e2809e3a1015 18.0 landed
Be more paranoid in configure's checks for CRC and POPCNT intrinsics.
- fdb5dd6331e3 18.0 cited

Attachments

v8-0001-Dispatch-CRC-computation-by-branching-rather-than.patch (text/x-patch) patch v8-0001
v8-0003-Add-a-Postgres-SQL-function-for-crc32c-benchmarki.patch (text/x-patch) patch v8-0003
v8-0004-Improve-CRC32C-performance-on-x86_64.patch (text/x-patch) patch v8-0004
v8-0002-Rename-CRC-choose-files-to-cpucap-files.patch (text/x-patch) patch v8-0002

On Tue, Feb 18, 2025 at 1:40 PM John Naylor <johncnaylorls@gmail.com> wrote:
>
> On Tue, Feb 18, 2025 at 12:41 AM Nathan Bossart
> <nathandbossart@gmail.com> wrote:

> > While this needn't block this patch set, I do find the dispatch code to be
> > pretty complicated.  Maybe we can improve that in the future by using
> > macros to auto-generate much of it.  My concern here is less about this
> > particular patch set and more about the long term maintainability as we add
> > more and more stuff like it, each with its own tangled web of build and
> > dispatch rules.

I had a further thought on this: CRC and non-vector popcount are kind
of special in that recent OSes assume they exist, and it's worth a bit
of effort to take advantage of that. Other things we may add should be
kept as simple as possible.

> - Rename the CRC choose*.c files to pg_cpucap_{x86,arm}.c and build
> them unconditionally for each platform
> - Initialize the runtime info by CPU platform and not other symbols
> where possible (I guess anything needing AVX-512 will still be a mess)

I've made a start of this for v8:

0001 is mostly the same as before
0002 (Meson-only for now) changes 0001 per the above, to see how it
looks, but I've not tried to add popcount or anything else. I like it
overall, but some details may need tweaking.
0004 generates the pclmul loop slightly differently to simplify
integrating with our code, but shouldn't make a big difference

Another thing I found in Agner's manuals: AMD Zen, even as recently as
Zen 4, don't have as good a microarchitecture for PCLMUL, so if anyone
with such a machine would like to help test the cutoff, the script is
at

https://www.postgresql.org/message-id/CANWCAZahvhE-%2BhtZiUyzPiS5e45ukx5877mD-dHr-KSX6LcdjQ%40mail.gmail.com

(needs "CREATE EXTENSION test_crc32c;" to run it)

--
John Naylor
Amazon Web Services