Horton

Re: ICU locale validation / canonicalization

Peter Eisentraut <peter.eisentraut@enterprisedb.com>

View thread View on postgr.es
From: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
To: Jeff Davis <pgsql@j-davis.com>, pgsql-hackers@postgresql.org
Date: 2023-02-10T06:42:31Z
Lists: pgsql-hackers 
On 09.02.23 22:15, Jeff Davis wrote:
> On Thu, 2023-02-09 at 15:44 +0100, Peter Eisentraut wrote:
>> One use case is that if a user specifies a locale, say, of 'de-AT',
>> this
>> might canonicalize to 'de' today,
> Canonicalization should not lose useful information, it should just
> rearrange it, so I don't see a risk here based on what I read and the
> behavior I saw. In ICU, "de-AT" canonicalizes to "de_AT" and becomes
> the language tag "de-AT".

It turns out that 'de_AT' is actually a distinct collation from 'de' in 
CLDR, so that was not the best example.  What behavior do you see for 
'de_CH'?

Commits

  1. Add missing source file to nls.mk

  2. Fix MSVC warning introduced in ea1db8ae70.

  3. Canonicalize ICU locale names to language tags.

  4. Validate ICU locales.

  5. initdb: emit message when using default ICU locale.

  6. initdb: replace check_icu_locale() with default_icu_locale().

  7. Fix error inconsistency in older ICU versions.

  8. Avoid potential UCollator leak for older ICU versions.

  9. pg_locale.c: change ereport() to elog().

  10. Handle the "und" locale in ICU versions 54 and older.

  11. Wrap ICU ucol_open().

  12. Support language tags in older ICU versions (53 and earlier).

  13. Improve support for UNICODE collation on older ICU

  14. Use ICU by default at initdb time.