Re: BUG #15548: Unaccent does not remove combining diacritical characters

Michael Paquier <michael@paquier.xyz>

From: Michael Paquier <michael@paquier.xyz>
To: Hugh Ranalli <hugh@whtc.ca>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Alvaro Herrera <alvherre@2ndquadrant.com>, Peter Eisentraut <peter.eisentraut@2ndquadrant.com>, thomas.munro@enterprisedb.com, Daniel Verite <daniel@manitou-mail.org>, pgsql-bugs@lists.postgresql.org
Date: 2019-01-04T13:00:48Z
Lists: pgsql-bugs, pgsql-hackers
On Thu, Jan 03, 2019 at 04:48:33PM -0500, Hugh Ranalli wrote:
> On Thu, 3 Jan 2019 at 13:22, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Alvaro Herrera <alvherre@2ndquadrant.com> writes:
>>> I think the point is that if the committee updates with a further
>>> version of the file, how do you find the new version?  We need a URL
>>> that's one step removed from the final file, so that we can see if we
>>> need to update it.  Maybe we can provide both URLs for convenience.
>>
>> +1.  Could be phrased along the lines of "documents are at URL1,
>> currently synced with URL2" so that it's clear that URL2 should
>> be updated when we re-sync with a newer release.
>>
> 
> Yes, this is what I was thinking. I was integrating this into my installer,
> used the "new" URL provided to download the file, and spent several minutes
> wondering why the script was failing (and what I had broken in it), before
> realising what had happened.

I think that we could just use the URLs I am mentioning here:
https://www.postgresql.org/message-id/20181227064958.GK2106@paquier.xyz

I haven't been able to finish what I wanted for the proposed patch set
yet, but what I was thinking about is to include:
1) The root URL where all the release folders are present
2) The full URL of the current Latin-ASCII.xml being used for the
generation, not as a URL pointing to the latest version, but as a URL
pointing to an exact version in time (I doubt that a released version
never changes in this tree, but who knows..).
3) The version used to generate the rules.
--
Michael

Commits

  1. Fix unaccent generation script in Windows

  2. Add combining characters to unaccent.rules.

  3. Update unaccent rules with release 34 of CLDR for Latin-ASCII.xml

  4. unaccent: Make generate_unaccent_rules.py Python 3 compatible

  5. Convert unaccent tests to UTF-8