Re: BUG #15548: Unaccent does not remove combining diacritical characters
Hugh Ranalli <hugh@whtc.ca>
From: Hugh Ranalli <hugh@whtc.ca>
To: Daniel Verite <daniel@manitou-mail.org>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, pgsql-bugs@lists.postgresql.org
Date: 2018-12-13T18:50:37Z
Lists: pgsql-bugs, pgsql-hackers
On Thu, 13 Dec 2018, 11:26 Daniel Verite <daniel@manitou-mail.org wrote: > Tom Lane wrote: > > > Hm, I thought the OP's proposal was just to make unaccent drop > > combining diacriticals independently of context, which'd avoid the > > combinatorial-growth problem. > That's what I was thinking. Given that the accent is separate from the characters, simply dropping it should result in the correct unaccented character. > > In that case, this could be achieved by simply appending the > diacriticals themselves to unaccent.rules, since replacement of a > string by an empty string is already supported as a rule. > It doesn't seem like the current file has any of these, but from > https://www.postgresql.org/docs/11/unaccent.html : > > "Alternatively, if only one character is given on a line, instances > of that character are deleted; this is useful in languages where > accents are represented by separate characters" > Yes, I had read that in the docs, and that's the approach I planned to take. I'll go ahead and develop a patch, then. Best wishes, Hugh >
Commits
-
Fix unaccent generation script in Windows
- 0afc0a784188 13.0 landed
-
Add combining characters to unaccent.rules.
- 456e3718e7b7 12.0 landed
-
Update unaccent rules with release 34 of CLDR for Latin-ASCII.xml
- e1c1d5444e43 12.0 landed
-
unaccent: Make generate_unaccent_rules.py Python 3 compatible
- 3d59da9ccdb9 12.0 landed
-
Convert unaccent tests to UTF-8
- b6f3649bba98 12.0 landed