Re: Proposal for enabling auto-vectorization for checksum calculations
Andrew Kim <tenistarkim@gmail.com>
From: Andrew Kim <tenistarkim@gmail.com>
To: John Naylor <johncnaylorls@gmail.com>
Cc: pgsql-hackers@lists.postgresql.org, Oleg Tselebrovskiy <o.tselebrovskiy@postgrespro.ru>
Date: 2025-11-05T23:49:47Z
Lists: pgsql-hackers
Commits
Same data as JSON:
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Use AVX2 for calculating page checksums where available
- 5e13b0f24039 19 (unreleased) landed
-
Refactor checksumming code to make it easier to use externally.
- f04216341dd1 9.3.0 cited
Attachments
- v9-0001-Move-checksum-functions-from-backend-storage-to-port.patch (application/octet-stream) patch v9-0001
- v9-0002-Move-platform-specific-checksum-code-from-header-to-.patch (application/octet-stream) patch v9-0002
- v9-0003-Benchmark-code-for-postgres-checksums.patch (application/octet-stream) patch v9-0003
Hi John, Thank you for reviewing and bringing this up regarding checksum architecture. On Tue, Oct 28, 2025 at 7:50 PM John Naylor <johncnaylorls@gmail.com> wrote: > > On Fri, Oct 24, 2025 at 2:49 PM Andrew Kim <tenistarkim@gmail.com> wrote: > > The function signatures (pg_checksum_block, pg_checksum_page) remain > > identical, and checksum_impl.h still contains the complete > > implementation that external programs can include. The runtime > > dispatch only affects internal PostgreSQL usage. > > I don't quite understand the architecture here -- all > platform-specific definitions were put in the "checksum_impl.h" > header. My thinking was that checksum.c would have all that, with thin > wrappers around the functions included from that header. The v9 patch series is attached. I've implemented the architecture as you described. checksum_impl.h -No platform-specific code (removed all AVX2, CPUID, intrinsics) -External programs get a clean, portable standalone implementation -Uses #ifndef PG_CHECKSUM_INTERNAL guard to prevent conflicts checksum.c (full implementation for checksum): -Includes checksum_impl.h for the basic implementation and common definitions -Contains all platform-specific code (AVX2, CPUID detection, runtime dispatch) -Implements thin wrapper functions that provide the public interface -Uses #define PG_CHECKSUM_INTERNAL before including the header > > -- > John Naylor > Amazon Web Services Thanks Andrew