Re: Radix tree for character conversion

Heikki Linnakangas <hlinnaka@iki.fi>

From: Heikki Linnakangas <hlinnaka@iki.fi>
To: Daniel Gustafsson <daniel@yesql.se>, Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>
Cc: robertmhaas@gmail.com, tsunakawa.takay@jp.fujitsu.com, tgl@sss.pgh.pa.us, ishii@sraoss.co.jp, pgsql-hackers@postgresql.org
Date: 2016-11-30T12:59:44Z
Lists: pgsql-hackers
On 10/31/2016 06:11 PM, Daniel Gustafsson wrote:
>> On 27 Oct 2016, at 09:23, Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp> wrote:
>>
>> At Tue, 25 Oct 2016 12:23:48 +0300, Heikki Linnakangas <hlinnaka@iki.fi> wrote in <08e7892a-d55c-eefe-76e6-7910bc8dd1f3@iki.fi>
>>>
>>> [..]
>>> The perl scripts are still quite messy. For example, I lost the checks
>>> for duplicate mappings somewhere along the way - that ought to be put
>>> back. My Perl skills are limited.
>>
>> Perl scripts are to be messy, I believe. Anyway the duplicate
>> check as been built into the sub print_radix_trees. Maybe the
>> same check is needed by some plain map files but it would be just
>> duplication for the maps having radix tree.
>
> I took a small stab at doing some cleaning of the Perl scripts, mainly around
> using the more modern (well, modern as in +15 years old) form for open(..),
> avoiding global filehandles for passing scalar references and enforcing use
> strict.  Some smaller typos and fixes were also included.  It seems my Perl has
> become a bit rusty so I hope the changes make sense.  The produced files are
> identical with these patches applied, they are merely doing cleaning as opposed
> to bugfixing.
>
> The attached patches are against the 0001-0006 patches from Heikki and you in
> this series of emails, the separation is intended to make them easier to read.

Thanks! Patches 0001-0003 seem to have been mostly unchanged for the 
later discussion and everyone seems to be happy with those patches, so I 
picked the parts of these cleanups of yours that applied to my patches 
0001-0003, and pushed those. I'll continue reviewing the rest..

- Heikki



Commits

  1. Use radix tree for character encoding conversions.

  2. Small fixes to the Perl scripts to create unicode conversion tables.

  3. Rewrite the perl scripts to produce our Unicode conversion tables.

  4. Remove leading zeros, for consistency with other map files.

  5. Remove code points < 0x80 from character conversion tables.