Horton

Re: BUG #15548: Unaccent does not remove combining diacritical characters

Tom Lane <tgl@sss.pgh.pa.us>

View thread View on postgr.es
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Hugh Ranalli <hugh@whtc.ca>
Cc: Daniel Verite <daniel@manitou-mail.org>, Thomas Munro <thomas.munro@enterprisedb.com>, pgsql-bugs@lists.postgresql.org
Date: 2018-12-15T19:03:58Z
Lists: pgsql-bugs, pgsql-hackers 
I wrote:
> ... I get something that's bit-for-bit the same as what's in unaccent.rules.
> So there's clearly a platform difference between here and there.
> I'm using Python 2.6.6, which is what ships with RHEL6; have not tried
> it on anything newer.

A few minutes later on a Fedora 28 box: python 2.7.15 also gives me the
expected results, while python 3.6.6 fails with "SyntaxError: invalid
syntax".

So updating that script to also work with python3 might be a worthwhile
TODO item.  But I'm at a loss to explain why you get different results.

			regards, tom lane

Commits

  1. Fix unaccent generation script in Windows

  2. Add combining characters to unaccent.rules.

  3. Update unaccent rules with release 34 of CLDR for Latin-ASCII.xml

  4. unaccent: Make generate_unaccent_rules.py Python 3 compatible

  5. Convert unaccent tests to UTF-8