Horton
Menu

Re: Order changes in PG16 since ICU introduction

Daniel Verite <daniel@manitou-mail.org>

View thread
From: "Daniel Verite" <daniel@manitou-mail.org>
To: "Jeff Davis" <pgsql@j-davis.com>
Cc: Peter Eisentraut <peter.eisentraut@enterprisedb.com>, Andrew Gierth <andrew@tao11.riddles.org.uk>, Tom Lane <tgl@sss.pgh.pa.us>, pgsql-hackers@lists.postgresql.org
Date: 2023-06-07T21:50:57Z
Lists: pgsql-hackers 
	Jeff Davis wrote:

> The locale "C" is a special case, documented as a non-locale. So, if
> LOCALE/--locale apply to ICU, then either ICU needs to handle locale
> "C" in the expected way (v8 patch series); or when we see locale "C" we
> need to somehow change the provider into something that can handle it
> (v6 patch series changes it to the "none" provider).

Yes it's a special case but when doing initdb --locale=C, a user does
not need or want an ICU locale. They want the same thing than what v15
does with the same arguments: a template0 database with
datlocprovider='c', datcollate='C', datctype='C', dateiculocale=NULL.

The simplest way to obtain that in v16 is to teach initdb that
--locale=C without the --locale-provider option implies that
--locale-provider=libc ([1])

OTOH what you're proposing with the 0001 patch is much more involved
in terms of tweaking the ICU code so that dateiculocale='C' and
datlocprovider='i' becomes a combination that provides the C semantics
that ICU doesn't have natively.

I don't agree with the reasoning that to make progress with
the other uses of --locale, we need to start by either making ICU
support C/POSIX (0001/0002), or add a new "none/builtin" provider
(previous set of patches).
v16 should not need it any more than v15 did, if v16 does the same as
v15 on locale='C', that is not involve ICU at all.

> Then that enables us to change LOCALE/--locale to apply to ICU, which
> means that a simple command like "initdb --locale=en_US" does a
> sensible thing regardless of the default provider.
>
> I understand you are skeptical of trying to apply an arbitrary locale
> name to ICU, but if they don't specify the provider, what do you expect
> to happen?

It's a hard question because it depends on what people have in their 
locale environment combined with what they try to do.
I think that initdb without any locale option should work well in
the majority of environments, but specifying a locale alone will not work
well in a number of cases, so users might end up concluding that they
need to specify not only the provider but lc_collate/lc_ctype.


[1]
https://www.postgresql.org/message-id/360c90b9-7c20-4cec-aade-38e6e3351c05@manitou-mail.org


Best regards,
-- 
Daniel Vérité
https://postgresql.verite.pro/
Twitter: @DanielVerite

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Introduce "builtin" collation provider.

  2. ICU: do not convert locale 'C' to 'en-US-u-va-posix'.

  3. initdb: change default --locale-provider back to libc.

  4. CREATE DATABASE: make LOCALE apply to all collation providers.

  5. Remove read-only server settings lc_collate and lc_ctype

  6. ICU: use uloc_getDefault() for initdb.

  7. Doc fixes for commit 1e16af8ab5.

  8. Doc: fix some rendering problems in the PDF docs build.

  9. Doc improvements for language tags and custom ICU collations.

  10. ICU: check for U_STRING_NOT_TERMINATED_WARNING.

  11. Reduce icu_validation_level default to WARNING.

  12. Revert "ICU: do not convert locale 'C' to 'en-US-u-va-posix'."

  13. Fix initdb --no-locale.

  14. Avoid character classification in regex escape parsing.