Re: Radix tree for character conversion

Daniel Gustafsson <daniel@yesql.se>

From: Daniel Gustafsson <daniel@yesql.se>
To: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Cc: Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>, Heikki Linnakangas <hlinnaka@iki.fi>, Robert Haas <robertmhaas@gmail.com>, tsunakawa.takay@jp.fujitsu.com, Tom Lane <tgl@sss.pgh.pa.us>, ishii@sraoss.co.jp, pgsql-hackers@postgresql.org
Date: 2016-11-09T01:19:01Z
Lists: pgsql-hackers

Attachments

> On 08 Nov 2016, at 17:37, Peter Eisentraut <peter.eisentraut@2ndquadrant.com> wrote:
> 
> On 10/31/16 12:11 PM, Daniel Gustafsson wrote:
>> I took a small stab at doing some cleaning of the Perl scripts, mainly around
>> using the more modern (well, modern as in +15 years old) form for open(..),
>> avoiding global filehandles for passing scalar references and enforcing use
>> strict.  Some smaller typos and fixes were also included.  It seems my Perl has
>> become a bit rusty so I hope the changes make sense.  The produced files are
>> identical with these patches applied, they are merely doing cleaning as opposed
>> to bugfixing.
>> 
>> The attached patches are against the 0001-0006 patches from Heikki and you in
>> this series of emails, the separation is intended to make them easier to read.
> 
> Cool.  See also here:
> https://www.postgresql.org/message-id/55E52225.4040305%40gmx.net

Nice, not having hacked much Perl in quite a while I had all but forgotten
about perlcritic.

Running it on the current version of the patchset yields mostly warnings on
string values used in the require “convutils.pm” statement.  There were however
two more interesting reports: one more open() call not using the three
parameter form and an instance of map which alters the input value.  The latter
is not causing an issue since we don’t use the input list past the map but
fixing it seems like good form.

Attached is a patch that addresses the perlcritic reports (running without any
special options).

cheers ./daniel

Commits

  1. Use radix tree for character encoding conversions.

  2. Small fixes to the Perl scripts to create unicode conversion tables.

  3. Rewrite the perl scripts to produce our Unicode conversion tables.

  4. Remove leading zeros, for consistency with other map files.

  5. Remove code points < 0x80 from character conversion tables.