Re: Radix tree for character conversion
Tom Lane <tgl@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Heikki Linnakangas <hlinnaka@iki.fi>,
Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>,
"Tsunakawa,
Takayuki" <tsunakawa.takay@jp.fujitsu.com>,
Tatsuo Ishii <ishii@sraoss.co.jp>,
"pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2016-10-07T16:19:23Z
Lists: pgsql-hackers
Robert Haas <robertmhaas@gmail.com> writes: > On Fri, Oct 7, 2016 at 6:46 AM, Heikki Linnakangas <hlinnaka@iki.fi> wrote: >> Ouch. We should find and document an authoritative source for all the >> mappings we have... >> >> I think the next steps here are: >> >> 1. Find an authoritative source for all the existing mappings. >> 2. Generate the radix tree files directly from the authoritative sources, >> instead of the existing *.map files. >> 3. Completely replace the existing binary-search code with this. > It might be best to convert using the existing map files, and then > update the mappings later. Otherwise, when things break, you won't > know what to blame. I think I went through this exercise last year or so, and updated the notes about the authoritative sources where I was able to find one. In the remaining cases, I believe that the maps have been intentionally tweaked and we should be cautious about undoing that. Tatsuo-san might remember more about why they are the way they are. regards, tom lane
Commits
-
Use radix tree for character encoding conversions.
- aeed17d00037 10.0 landed
-
Small fixes to the Perl scripts to create unicode conversion tables.
- bc1686f3f653 10.0 landed
-
Rewrite the perl scripts to produce our Unicode conversion tables.
- 1de9cc0dcca6 10.0 landed
-
Remove leading zeros, for consistency with other map files.
- 6c303223be34 10.0 landed
-
Remove code points < 0x80 from character conversion tables.
- 2c09c93ce1b8 10.0 landed