Re: Radix tree for character conversion
Heikki Linnakangas <hlinnaka@iki.fi>
From: Heikki Linnakangas <hlinnaka@iki.fi>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>,
michael.paquier@gmail.com, daniel@yesql.se,
peter.eisentraut@2ndquadrant.com, robertmhaas@gmail.com,
tsunakawa.takay@jp.fujitsu.com, ishii@sraoss.co.jp,
pgsql-hackers@postgresql.org
Date: 2017-03-13T19:07:39Z
Lists: pgsql-hackers
On 03/13/2017 08:53 PM, Tom Lane wrote: > Heikki Linnakangas <hlinnaka@iki.fi> writes: >> It would be nice to run the map_checker tool one more time, though, to >> verify that the mappings match those from PostgreSQL 9.6. > > +1 > >> Just to be sure, and after that the map checker can go to the dustbin. > > Hm, maybe we should keep it around for the next time somebody has a bright > idea in this area? The map checker compares old-style maps with the new radix maps. The next time 'round, we'll need something that compares the radix maps with the next great thing. Not sure how easy it would be to adapt. Hmm. A somewhat different approach might be more suitable for testing across versions, though. We could modify the perl scripts slightly to print out SQL statements that exercise every mapping. For every supported conversion, the SQL script could: 1. create a database in the source encoding. 2. set client_encoding='<target encoding>' 3. SELECT a string that contains every character in the source encoding. You could then run those SQL statements against old and new server version, and verify that you get the same results. - Heikki
Commits
-
Use radix tree for character encoding conversions.
- aeed17d00037 10.0 landed
-
Small fixes to the Perl scripts to create unicode conversion tables.
- bc1686f3f653 10.0 landed
-
Rewrite the perl scripts to produce our Unicode conversion tables.
- 1de9cc0dcca6 10.0 landed
-
Remove leading zeros, for consistency with other map files.
- 6c303223be34 10.0 landed
-
Remove code points < 0x80 from character conversion tables.
- 2c09c93ce1b8 10.0 landed