Re: Popcount optimization using AVX512

Nathan Bossart <nathandbossart@gmail.com>

From: Nathan Bossart <nathandbossart@gmail.com>

To: Ants Aasma <ants.aasma@cybertec.at>

Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>, "Amonson, Paul D" <paul.d.amonson@intel.com>, Tom Lane <tgl@sss.pgh.pa.us>, David Rowley <dgrowleyml@gmail.com>, Andres Freund <andres@anarazel.de>, "Shankaran, Akash" <akash.shankaran@intel.com>, Noah Misch <noah@leadboat.com>, Matthias van de Meent <boekewurm+postgres@gmail.com>, "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>

Date: 2024-04-05T15:38:11Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Fix __attribute__((target(...))) usage.
- 41b98ddb77bf 18.0 landed
Use __attribute__((target(...))) for AVX-512 support.
- f78667bd910e 18.0 landed
Fix code for probing availability of AVX-512.
- 598e0114a3b1 17.0 landed
Optimize visibilitymap_count() with AVX-512 instructions.
- 41c51f0c68b2 17.0 landed
Optimize pg_popcount() with AVX-512 instructions.
- 792752af4eb5 17.0 landed
Inline pg_popcount() for small buffers.
- deb1486c7d36 17.0 landed
Avoid function call overhead of pg_popcount() in syslogger.c.
- 4133c1f45c54 17.0 landed
Refactor code for setting pg_popcount* function pointers.
- 6687430c98f3 17.0 landed
Inline pg_popcount{32,64} into pg_popcount().
- cc4826dd5e52 17.0 landed
Remove MSVC scripts
- 1301c80b2167 17.0 cited
Use ARMv8 CRC instructions where available.
- f044d71e331d 11.0 cited
Use Intel SSE 4.2 CRC instructions where available.
- 3dc2d62d0486 9.5.0 cited

Attachments

v27-0001-AVX512-popcount-support.patch (text/x-diff)

On Fri, Apr 05, 2024 at 07:58:44AM -0500, Nathan Bossart wrote:
> On Fri, Apr 05, 2024 at 10:33:27AM +0300, Ants Aasma wrote:
>> The main issue I saw was that clang was able to peel off the first
>> iteration of the loop and then eliminate the mask assignment and
>> replace masked load with a memory operand for vpopcnt. I was not able
>> to convince gcc to do that regardless of optimization options.
>> Generated code for the inner loop:
>> 
>> clang:
>> <L2>:
>>       50:      add rdx, 64
>>       54:      cmp rdx, rdi
>>       57:      jae <L1>
>>       59:      vpopcntq zmm1, zmmword ptr [rdx]
>>       5f:      vpaddq zmm0, zmm1, zmm0
>>       65:      jmp <L2>
>> 
>> gcc:
>> <L1>:
>>       38:      kmovq k1, rdx
>>       3d:      vmovdqu8 zmm0 {k1} {z}, zmmword ptr [rax]
>>       43:      add rax, 64
>>       47:      mov rdx, -1
>>       4e:      vpopcntq zmm0, zmm0
>>       54:      vpaddq zmm0, zmm0, zmm1
>>       5a:      vmovdqa64 zmm1, zmm0
>>       60:      cmp rax, rsi
>>       63:      jb <L1>
>> 
>> I'm not sure how much that matters in practice. Attached is a patch to
>> do this manually giving essentially the same result in gcc. As most
>> distro packages are built using gcc I think it would make sense to
>> have the extra code if it gives a noticeable benefit for large cases.
> 
> Yeah, I did see this, but I also wasn't sure if it was worth further
> complicating the code.  I can test with and without your fix and see if it
> makes any difference in the benchmarks.

This seems to provide a small performance boost, so I've incorporated it
into v27.

-- 
Nathan Bossart
Amazon Web Services: https://aws.amazon.com