Re: Radix tree for character conversion

Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>

From: Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>
To: ayumi.ishii.pg@gmail.com
Cc: hlinnaka@iki.fi, daniel@yesql.se, peter.eisentraut@2ndquadrant.com, robertmhaas@gmail.com, tsunakawa.takay@jp.fujitsu.com, tgl@sss.pgh.pa.us, ishii@sraoss.co.jp, pgsql-hackers@postgresql.org
Date: 2017-01-26T07:16:56Z
Lists: pgsql-hackers

Attachments

Hello, thank you for looking this.

At Wed, 25 Jan 2017 19:18:26 +0900, Ishii Ayumi <ayumi.ishii.pg@gmail.com> wrote in <CAOu5J714+w-TRSNHbsS+aBVE5LdsR3CEZ6w4QLQ=9NrAJNavTA@mail.gmail.com>
> I patched 4 patchset and run "make", but I got failed.
> Is this a bug or my mistake ?
> I'm sorry if I'm wrong.
> 
> [$(TOP)]$ patch -p1 < ../0001-Add-missing-semicolon.patch
> [$(TOP)]$ patch -p1 < ../0002-Correct-reference-resolution-syntax.patch
> [$(TOP)]$ patch -p1 <
> ../0003-Apply-pgperltidy-on-src-backend-utils-mb-Unicode.patch
> [$(TOP)]$ patch -p1 < ../0004-Use-radix-tree-for-character-conversion.patch
> [$(TOP)]$ ./configure
> [Unicode]$ make

The directory src/backend/mb/uilts/Unicode is not built as a part
of the top-level build, and it would be preferable that the
preexisting map files are removed.

$ cd src/backend/utils/mb/Unicode
$ make distclean      # this would require ./configure
$ make maintainer-clean
$ cd ../../../../..   # go to top
$ make clean
(make'ing here will give you an error saying a .map file is not found)
$ cd  src/backend/utils/mb/Unicode  # again
$ make
$ cd ../../../../..   # go to top
$ make

This steps still suceeds for me, even with the patches on the
current master.

The cause of the the following errors seems the other things.

> '/usr/bin/perl' UCS_to_most.pl
> Type of arg 1 to keys must be hash (not hash element) at convutils.pm
> line 443, near "})
>         "
> Type of arg 1 to values must be hash (not hash element) at
> convutils.pm line 596, near "})
>         "
> Type of arg 1 to each must be hash (not private variable) at
> convutils.pm line 755, near "$map)
>         "
> Compilation failed in require at UCS_to_most.pl line 19.
> make: *** [iso8859_2_to_utf8.map] Error 255

Surely perl 5.8.9 complained just as above but 5.16
doesn't. Google told me that at least 5.10 behaves as the same
way. The *current* requirement for perl verion to build is 5.8.

https://www.postgresql.org/docs/current/static/install-requirements.html

Fortunately, only three lines of change suffices 5.8 so I've
chosen to flatter perl 5.8.

As the result, no changes has been made on 0001-0003 so I
attached only 0004 to this mail.

regards,

-- 
Kyotaro Horiguchi
NTT Open Source Software Center

Commits

  1. Use radix tree for character encoding conversions.

  2. Small fixes to the Perl scripts to create unicode conversion tables.

  3. Rewrite the perl scripts to produce our Unicode conversion tables.

  4. Remove leading zeros, for consistency with other map files.

  5. Remove code points < 0x80 from character conversion tables.