Re: Popcount optimization using AVX512

Nathan Bossart <nathandbossart@gmail.com>

From: Nathan Bossart <nathandbossart@gmail.com>

To: Ants Aasma <ants.aasma@cybertec.at>

Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>, "Amonson, Paul D" <paul.d.amonson@intel.com>, Tom Lane <tgl@sss.pgh.pa.us>, David Rowley <dgrowleyml@gmail.com>, Andres Freund <andres@anarazel.de>, "Shankaran, Akash" <akash.shankaran@intel.com>, Noah Misch <noah@leadboat.com>, Matthias van de Meent <boekewurm+postgres@gmail.com>, "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>

Date: 2024-04-05T12:58:44Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Fix __attribute__((target(...))) usage.
- 41b98ddb77bf 18.0 landed
Use __attribute__((target(...))) for AVX-512 support.
- f78667bd910e 18.0 landed
Fix code for probing availability of AVX-512.
- 598e0114a3b1 17.0 landed
Optimize visibilitymap_count() with AVX-512 instructions.
- 41c51f0c68b2 17.0 landed
Optimize pg_popcount() with AVX-512 instructions.
- 792752af4eb5 17.0 landed
Inline pg_popcount() for small buffers.
- deb1486c7d36 17.0 landed
Avoid function call overhead of pg_popcount() in syslogger.c.
- 4133c1f45c54 17.0 landed
Refactor code for setting pg_popcount* function pointers.
- 6687430c98f3 17.0 landed
Inline pg_popcount{32,64} into pg_popcount().
- cc4826dd5e52 17.0 landed
Remove MSVC scripts
- 1301c80b2167 17.0 cited
Use ARMv8 CRC instructions where available.
- f044d71e331d 11.0 cited
Use Intel SSE 4.2 CRC instructions where available.
- 3dc2d62d0486 9.5.0 cited

On Fri, Apr 05, 2024 at 10:33:27AM +0300, Ants Aasma wrote:
> The main issue I saw was that clang was able to peel off the first
> iteration of the loop and then eliminate the mask assignment and
> replace masked load with a memory operand for vpopcnt. I was not able
> to convince gcc to do that regardless of optimization options.
> Generated code for the inner loop:
> 
> clang:
> <L2>:
>       50:      add rdx, 64
>       54:      cmp rdx, rdi
>       57:      jae <L1>
>       59:      vpopcntq zmm1, zmmword ptr [rdx]
>       5f:      vpaddq zmm0, zmm1, zmm0
>       65:      jmp <L2>
> 
> gcc:
> <L1>:
>       38:      kmovq k1, rdx
>       3d:      vmovdqu8 zmm0 {k1} {z}, zmmword ptr [rax]
>       43:      add rax, 64
>       47:      mov rdx, -1
>       4e:      vpopcntq zmm0, zmm0
>       54:      vpaddq zmm0, zmm0, zmm1
>       5a:      vmovdqa64 zmm1, zmm0
>       60:      cmp rax, rsi
>       63:      jb <L1>
> 
> I'm not sure how much that matters in practice. Attached is a patch to
> do this manually giving essentially the same result in gcc. As most
> distro packages are built using gcc I think it would make sense to
> have the extra code if it gives a noticeable benefit for large cases.

Yeah, I did see this, but I also wasn't sure if it was worth further
complicating the code.  I can test with and without your fix and see if it
makes any difference in the benchmarks.

-- 
Nathan Bossart
Amazon Web Services: https://aws.amazon.com