Horton
Menu

Re: Order changes in PG16 since ICU introduction

Daniel Verite <daniel@manitou-mail.org>

View thread
From: "Daniel Verite" <daniel@manitou-mail.org>
To: "Jeff Davis" <pgsql@j-davis.com>
Cc: Andrew Gierth <andrew@tao11.riddles.org.uk>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, Sandro Santilli <strk@kbt.io>, Tom Lane <tgl@sss.pgh.pa.us>, Regina Obe <lr@pcorp.us>, pgsql-hackers@lists.postgresql.org
Date: 2023-04-27T12:23:24Z
Lists: pgsql-hackers 
	Jeff Davis wrote:

> Attached are a few small patches:
> 
>   0001: don't convert C to en-US-u-va-posix
>   0002: handle locale C the same regardless of the provider, as you
> suggest above
>   0003: make LOCALE (or --locale) apply to everything including ICU

Testing this briefly I noticed two regressions

1) all pg_collation.collversion are empty due to a trivial bug in 0002:

@ -1650,6 +1686,10 @@ get_collation_actual_version(char collprovider, const
char *collcollate)
 {
	char	   *collversion = NULL;

+	if (pg_strcasecmp("C", collcollate) ||
+		pg_strcasecmp("POSIX", collcollate))
+		return NULL;
+

This should be pg_strcasecmp(...) == 0

2) The following works with HEAD (default provider=icu) but errors out with
the patches:

postgres=# create database lat9 locale 'fr_FR@euro' encoding LATIN9 template
'template0';
ERROR:	could not convert locale name "fr_FR@euro" to language tag:
U_ILLEGAL_ARGUMENT_ERROR

fr_FR@euro is a libc locale name 

$ locale -a|grep fr_FR
fr_FR
fr_FR@euro
fr_FR.iso88591
fr_FR.iso885915@euro
fr_FR.utf8

I understand that fr_FR@euro is taken as an ICU locale name, with the idea
that the locale
syntax being more or less compatible between both providers, this should work
smoothly.  0003 seems to go further in the interpretation and fail on it.
TBH the assumption that it's OK to feed libc locale names to ICU feels quite
uncomfortable.


Best regards,
-- 
Daniel Vérité
https://postgresql.verite.pro/
Twitter: @DanielVerite

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Introduce "builtin" collation provider.

  2. ICU: do not convert locale 'C' to 'en-US-u-va-posix'.

  3. initdb: change default --locale-provider back to libc.

  4. CREATE DATABASE: make LOCALE apply to all collation providers.

  5. Remove read-only server settings lc_collate and lc_ctype

  6. ICU: use uloc_getDefault() for initdb.

  7. Doc fixes for commit 1e16af8ab5.

  8. Doc: fix some rendering problems in the PDF docs build.

  9. Doc improvements for language tags and custom ICU collations.

  10. ICU: check for U_STRING_NOT_TERMINATED_WARNING.

  11. Reduce icu_validation_level default to WARNING.

  12. Revert "ICU: do not convert locale 'C' to 'en-US-u-va-posix'."

  13. Fix initdb --no-locale.

  14. Avoid character classification in regex escape parsing.