Horton

Re: ICU locale validation / canonicalization

Peter Eisentraut <peter.eisentraut@enterprisedb.com>

View thread View on postgr.es
From: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
To: Jeff Davis <pgsql@j-davis.com>, Robert Haas <robertmhaas@gmail.com>
Cc: pgsql-hackers@postgresql.org
Date: 2023-03-14T07:08:55Z
Lists: pgsql-hackers 
On 13.03.23 16:31, Jeff Davis wrote:
>> What we had discussed a while ago in one of these threads is that ICU
>> before version 54 do not support keyword lists, and we have custom
>> code
>> to do that parsing ourselves, but we don't have code to do the same
>> for
>> language tags.  Therefore, if I understand this right, if we
>> automatically convert ICU locale IDs to language tags, as shown
>> above,
>> then we break support for such locales in those older ICU versions.
> 
> Right. In versions 53 and earlier, and during pg_upgrade, we would just
> preserve the locale string as entered.

Another issue that came to mind:  Right now, you can, say, develop SQL 
schemas on a newer ICU version, say, your laptop, and then deploy them 
on a server running an older ICU version.  If we have a cutoff beyond 
which we convert ICU locale IDs to language tags, then this won't work 
anymore for certain combinations.  And RHEL/CentOS 7 is still pretty 
popular.

Commits

  1. Add missing source file to nls.mk

  2. Fix MSVC warning introduced in ea1db8ae70.

  3. Canonicalize ICU locale names to language tags.

  4. Validate ICU locales.

  5. initdb: emit message when using default ICU locale.

  6. initdb: replace check_icu_locale() with default_icu_locale().

  7. Fix error inconsistency in older ICU versions.

  8. Avoid potential UCollator leak for older ICU versions.

  9. pg_locale.c: change ereport() to elog().

  10. Handle the "und" locale in ICU versions 54 and older.

  11. Wrap ICU ucol_open().

  12. Support language tags in older ICU versions (53 and earlier).

  13. Improve support for UNICODE collation on older ICU

  14. Use ICU by default at initdb time.