Re: Unicode normalization SQL functions

On 2020-03-26 08:25, Peter Eisentraut wrote:
> On 2020-03-24 10:20, Peter Eisentraut wrote:
>> Now I have some concerns about the size of the new table in
>> unicode_normprops_table.h, and the resulting binary size.  At the very
>> least, we should probably make that #ifndef FRONTEND or something like
>> that so libpq isn't bloated by it unnecessarily.  Perhaps there is a
>> better format for that table?  Any ideas?
> 
> I have figured this out.  New patch is attached.
> 
> First, I have added #ifndef FRONTEND, as mentioned above, so libpq isn't
> bloated.  Second, I have changed the lookup structure to a bitfield, so
> each entry is only 32 bits instead of 64.  Third, I have dropped the
> quickcheck tables for the NFD and NFKD forms.  Those are by far the
> biggest tables, and you still get okay performance if you do the
> normalization check the long way, since we don't need the recomposition
> step on those cases, which is by far the slowest part.  The main use
> case of all of this, I expect, is to check for NFC normalization, so
> it's okay if the other variants are not optimized to the same extent.

I have committed this version.

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services