Horton
Menu

Re: Order changes in PG16 since ICU introduction

Andrew Gierth <andrew@tao11.riddles.org.uk>

View thread
From: Andrew Gierth <andrew@tao11.riddles.org.uk>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Peter Eisentraut <peter.eisentraut@enterprisedb.com>, Sandro Santilli <strk@kbt.io>, Regina Obe <lr@pcorp.us>, pgsql-hackers@lists.postgresql.org, Jeff Davis <pgsql@j-davis.com>
Date: 2023-04-21T19:14:20Z
Lists: pgsql-hackers 
>>>>> "Tom" == Tom Lane <tgl@sss.pgh.pa.us> writes:

 >> Also, somewhere along the line someone broke initdb --no-locale,
 >> which should result in C locale being the default everywhere, but
 >> when I just tested it it picked 'en' for an ICU locale, which is not
 >> the right thing.

 Tom> Confirmed:

 Tom> $ LANG=en_US.utf8 initdb --no-locale
 Tom> The files belonging to this database system will be owned by user "postgres".
 Tom> This user must also own the server process.

 Tom> Using default ICU locale "en_US".
 Tom> Using language tag "en-US" for ICU locale "en_US".
 Tom> The database cluster will be initialized with this locale configuration:
 Tom>   provider:    icu
 Tom>   ICU locale:  en-US
 Tom>   LC_COLLATE:  C
 Tom>   LC_CTYPE:    C
 Tom>   ...

 Tom> That needs to be fixed: --no-locale should prevent any
 Tom> consideration of initdb's LANG/LC_foo environment.

Would it also not make sense to also take into account any --locale and
--lc-* options before choosing an ICU default locale? Right now if you
do, say, initdb --locale=fr_FR you get an ICU locale based on the
environment but lc_* settings based on the option, which seems maximally
confusing.

Also, what happens now to lc_collate_is_c() when the provider is ICU? Am
I missing something, or is it never true now, even if you specified C /
POSIX / en-US-u-va-posix as the ICU locale? This seems like it could be
an important pessimization.

Also also, we now have the problem that it is much harder to create a
'C' collation database within an existing cluster (e.g. for testing)
without knowing whether the default provider is ICU. In the past one
would have done:

CREATE DATABASE test TEMPLATE=template0 ENCODING = 'UTF8' LOCALE = 'C';

but now that creates a database that uses the same ICU locale as
template0 by default. If instead one tries:

CREATE DATABASE test TEMPLATE=template0 ENCODING = 'UTF8' LOCALE = 'C' ICU_LOCALE='C';

then one gets an error if the default locale provider is _not_ ICU. The
only option now seems to be:

CREATE DATABASE test TEMPLATE=template0 ENCODING = 'UTF8' LOCALE = 'C' LOCALE_PROVIDER = 'libc';

which of course doesn't work in older pg versions.

-- 
Andrew.

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Introduce "builtin" collation provider.

  2. ICU: do not convert locale 'C' to 'en-US-u-va-posix'.

  3. initdb: change default --locale-provider back to libc.

  4. CREATE DATABASE: make LOCALE apply to all collation providers.

  5. Remove read-only server settings lc_collate and lc_ctype

  6. ICU: use uloc_getDefault() for initdb.

  7. Doc fixes for commit 1e16af8ab5.

  8. Doc: fix some rendering problems in the PDF docs build.

  9. Doc improvements for language tags and custom ICU collations.

  10. ICU: check for U_STRING_NOT_TERMINATED_WARNING.

  11. Reduce icu_validation_level default to WARNING.

  12. Revert "ICU: do not convert locale 'C' to 'en-US-u-va-posix'."

  13. Fix initdb --no-locale.

  14. Avoid character classification in regex escape parsing.