Re: Radix tree for character conversion

Daniel Gustafsson <daniel@yesql.se>

From: Daniel Gustafsson <daniel@yesql.se>
To: Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>
Cc: Heikki Linnakangas <hlinnaka@iki.fi>, robertmhaas@gmail.com, tsunakawa.takay@jp.fujitsu.com, tgl@sss.pgh.pa.us, ishii@sraoss.co.jp, pgsql-hackers@postgresql.org
Date: 2016-10-31T16:11:17Z
Lists: pgsql-hackers

Attachments

> On 27 Oct 2016, at 09:23, Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp> wrote:
> 
> Hello, thank you very much for the work. My work became quite
> easier with it.
> 
> At Tue, 25 Oct 2016 12:23:48 +0300, Heikki Linnakangas <hlinnaka@iki.fi> wrote in <08e7892a-d55c-eefe-76e6-7910bc8dd1f3@iki.fi>
>> 
>> [..]
>> The perl scripts are still quite messy. For example, I lost the checks
>> for duplicate mappings somewhere along the way - that ought to be put
>> back. My Perl skills are limited.
> 
> Perl scripts are to be messy, I believe. Anyway the duplicate
> check as been built into the sub print_radix_trees. Maybe the
> same check is needed by some plain map files but it would be just
> duplication for the maps having radix tree.

I took a small stab at doing some cleaning of the Perl scripts, mainly around
using the more modern (well, modern as in +15 years old) form for open(..),
avoiding global filehandles for passing scalar references and enforcing use
strict.  Some smaller typos and fixes were also included.  It seems my Perl has
become a bit rusty so I hope the changes make sense.  The produced files are
identical with these patches applied, they are merely doing cleaning as opposed
to bugfixing.

The attached patches are against the 0001-0006 patches from Heikki and you in
this series of emails, the separation is intended to make them easier to read.

cheers ./daniel

Commits

  1. Use radix tree for character encoding conversions.

  2. Small fixes to the Perl scripts to create unicode conversion tables.

  3. Rewrite the perl scripts to produce our Unicode conversion tables.

  4. Remove leading zeros, for consistency with other map files.

  5. Remove code points < 0x80 from character conversion tables.