Horton

Re: Unicode normalization SQL functions

John Naylor <john.naylor@2ndquadrant.com>

View thread View on postgr.es
From: John Naylor <john.naylor@2ndquadrant.com>
To: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>, Daniel Verite <daniel@manitou-mail.org>, Andreas Karlsson <andreas@proxel.se>
Date: 2020-03-27T07:16:12Z
Lists: pgsql-hackers 
I wrote:
>
> Regression tests pass, but I haven't measured performance yet.

Using a test similar to one upthread:

select count(*) from (select md5(i::text) as t from
generate_series(1,100000) as i) s where t is nfc normalized ;

I get (median of three)
v4  419ms
v5  310ms

with binary size
v4  HEAD + 33kB
v5  HEAD + 57kB

-- 
John Naylor                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Commits

  1. Add SQL functions for Unicode normalization

  2. Add support for other normal forms to Unicode normalization API