Horton

Re: Radix tree for character conversion

Michael Paquier <michael.paquier@gmail.com>

View thread View on postgr.es
From: Michael Paquier <michael.paquier@gmail.com>
To: Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>
Cc: Heikki Linnakangas <hlinnaka@iki.fi>, daniel@yesql.se, peter.eisentraut@2ndquadrant.com, Robert Haas <robertmhaas@gmail.com>, tsunakawa.takay@jp.fujitsu.com, Tom Lane <tgl@sss.pgh.pa.us>, Tatsuo Ishii <ishii@sraoss.co.jp>, PostgreSQL mailing lists <pgsql-hackers@postgresql.org>
Date: 2017-03-03T03:53:04Z
Lists: pgsql-hackers 
On Thu, Mar 2, 2017 at 2:20 PM, Kyotaro HORIGUCHI
<horiguchi.kyotaro@lab.ntt.co.jp> wrote:
> 5) Just remove plain map files and all related code. Addition to
>    that, Makefile stores hash digest of authority files in
>    Unicode/authoriy_hashes.txt or something that is managed by
>    git.

That may be an idea to check for differences across upstream versions.
But that sounds like a separate discussion to me.

> This digest may differ among platforms (typically from cr/nl
> difference) but we can assume *nix for the usage.
>
> I will send the next version after this discussion is settled.

Sure. There is not much point to move on without Heikki's opinion at
least, or anybody else like Ishii-san or Tom who are familiar with
this code. I would think that Heikki would be the committer to pick up
this change though.
-- 
Michael

Commits

  1. Use radix tree for character encoding conversions.

  2. Small fixes to the Perl scripts to create unicode conversion tables.

  3. Rewrite the perl scripts to produce our Unicode conversion tables.

  4. Remove leading zeros, for consistency with other map files.

  5. Remove code points < 0x80 from character conversion tables.