Re: speed up verifying UTF-8

Greg Stark <stark@mit.edu>

From: Greg Stark <stark@mit.edu>
To: Heikki Linnakangas <hlinnaka@iki.fi>
Cc: John Naylor <john.naylor@enterprisedb.com>, Amit Khandekar <amitdkhan.pg@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2021-06-03T14:41:41Z
Lists: pgsql-hackers
I haven't looked at the surrounding code. Are we processing all the
COPY data in one long stream or processing each field individually? If
we're processing much more than 128 bits and happy to detect NUL
errors only at the end after wasting some work then you could hoist
that has_zero check entirely out of the loop (removing the branch
though it's probably a correctly predicted branch anyways).

Do something like:

zero_accumulator = zero_accumulator & next_chunk

in the loop and then only at the very end check for zeros in that.



Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →
  1. Simplify coding style of is_valid_ascii()

  2. Apply auto-vectorization to the inner loop of numeric multiplication.