Re: Radix tree for character conversion
Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>
From: Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>
To: hlinnaka@iki.fi
Cc: daniel@yesql.se, peter.eisentraut@2ndquadrant.com, robertmhaas@gmail.com,
tsunakawa.takay@jp.fujitsu.com, tgl@sss.pgh.pa.us, ishii@sraoss.co.jp,
pgsql-hackers@postgresql.org
Date: 2017-01-10T11:22:23Z
Lists: pgsql-hackers
Attachments
- 0001-Add-missing-semicolon.patch (text/x-patch)
Hello, I found a bug in my portion while rebasing. The attached patches apply on top of the current master HEAD, not on Heikki's previous one. And separated into 4 parts. At Tue, 13 Dec 2016 15:11:03 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp> wrote in <20161213.151103.157484378.horiguchi.kyotaro@lab.ntt.co.jp> > > Apart from the aboves, I have some trivial comments on the new > > version. > > > > > > 1. If we decide not to use old-style maps, UtfToLocal no longer > > need to take void * as map data. (Patch 0001) I changed the pointer type wrongly. Combined maps are of the type *_combined. > > 2. "use Data::Dumper" doesn't seem necessary. (Patch 0002) > > 3. A comment contains a superfluous comma. (Patch 0002) (The last > > byte of the first line below) > > 4. The following code doesn't seem so perl'ish. > > 5. download_srctxts.sh is no longer needed. (No patch) > > 6. Fixed some inconsistent indentation/folding. > 7. Fix handling of $verbose. > 8. Sort segments using leading bytes. The attached files are the following. This patchset is not complete missing changes of map files. The change is tremendously large but generatable. 0001-Add-missing-semicolon.patch UCS_to_EUC_JP.pl has a line missing teminating semicolon. This doesn't harm but surely a syntax error. This patch fixes it. This might should be a separate patch. 0002-Correct-reference-resolution-syntax.patch convutils.pm has lines with different syntax of reference resolution. This unifies the syntax. 0003-Apply-pgperltidy-on-src-backend-utils-mb-Unicode.patch Before adding radix tree stuff, applied pgperltidy and inserted format-skipping pragma for the parts where perltidy seems to do too much. 0004-Use-radix-tree-for-character-conversion.patch Radix tree body. The unattached fifth patch is generated by the following steps. [$(TOP)]$ ./configure [Unicode]$ make [Unicode]$ make distclean [Unicode]$ git add . [Unicode]$ commit === COMMITE MESSSAGE Replace map files with radix tree files. These encodings no longer uses the former map files and uses new radix tree files. === regards, -- Kyotaro Horiguchi NTT Open Source Software Center
Commits
-
Use radix tree for character encoding conversions.
- aeed17d00037 10.0 landed
-
Small fixes to the Perl scripts to create unicode conversion tables.
- bc1686f3f653 10.0 landed
-
Rewrite the perl scripts to produce our Unicode conversion tables.
- 1de9cc0dcca6 10.0 landed
-
Remove leading zeros, for consistency with other map files.
- 6c303223be34 10.0 landed
-
Remove code points < 0x80 from character conversion tables.
- 2c09c93ce1b8 10.0 landed