Re: BUG #15548: Unaccent does not remove combining diacritical characters

Hugh Ranalli <hugh@whtc.ca>

From: Hugh Ranalli <hugh@whtc.ca>
To: Michael Paquier <michael@paquier.xyz>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Peter Eisentraut <peter.eisentraut@2ndquadrant.com>, thomas.munro@enterprisedb.com, Daniel Verite <daniel@manitou-mail.org>, pgsql-bugs@lists.postgresql.org
Date: 2019-01-03T16:19:43Z
Lists: pgsql-bugs, pgsql-hackers
On Wed, 2 Jan 2019 at 20:15, Michael Paquier <michael@paquier.xyz> wrote:

> As a side note, if I recall correctly Python 2.7 will be EOL'd in
> 2020 by community, though I suspect that a couple of vendors will
> still maintain compatibility for a couple of years in what they ship.
> CentOS and RHEL enter in this category perhaps.  Like Peter, I would
> vote for just maintaining support for Python 3 in this script, as any
> modern development machines have it anyway, and not a lot of commits
> involve it (I am counting 4 since 2015).
>

I realise this is an incredibly minor component of the PostgreSQL
infrastructure, but as I don't want to hold up reviewers, may I ask:

   - It seems we have two votes for Python 3 only, and one for Python 2/3.
   I lean toward Python 2/3 myself because: a) many distributions still ship
   with Python 2 as the default and b) it's a single code block that can
   easily be removed. If the decision is for Python 3, I'd like at least to
   add a check that catches this and prints a message, rather than leaving
   someone with a cryptic runtime error that makes them think the script is
   broken;
   - Michael Paquier, do you have any other comments? If not, I'll adjust
   the documentation to use the URLs you have indicated. If you are
   downloading via curl or wget, the URL I used is the proper one. It gives
   you the XML file, whereas the other saves the HTML interface, leading to
   errors if you try to run it. I'll also add this to the documentation.

Once I have clarification on these, I'll update the patches.

Thanks,
Hugh

Commits

  1. Fix unaccent generation script in Windows

  2. Add combining characters to unaccent.rules.

  3. Update unaccent rules with release 34 of CLDR for Latin-ASCII.xml

  4. unaccent: Make generate_unaccent_rules.py Python 3 compatible

  5. Convert unaccent tests to UTF-8