Horton
Menu

Re: Order changes in PG16 since ICU introduction

Tom Lane <tgl@sss.pgh.pa.us>

View thread
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Jeff Davis <pgsql@j-davis.com>
Cc: Regina Obe <lr@pcorp.us>, "'Peter Eisentraut'" <peter.eisentraut@enterprisedb.com>, "'Sandro Santilli'" <strk@kbt.io>, pgsql-hackers@lists.postgresql.org
Date: 2023-04-21T20:00:09Z
Lists: pgsql-hackers 
Jeff Davis <pgsql@j-davis.com> writes:
> I have a couple ideas:

> 1. Introduce a "none" provider to separate the concept of C/POSIX
> locales from the libc provider. It's not really using a provider
> anyway, it's just using memcmp(), and I think it causes confusion to
> combine them. Saying "LOCALE_PROVIDER=none" is less error-prone than
> "LOCALE_PROVIDER=libc LOCALE='C'".

I think I might like this idea, except for one thing: you're imagining
that the locale doesn't control anything except string comparisons.
What about to_upper/to_lower, character classifications in regexes, etc?
(I'm not sure whether those operations can get redirected to ICU today
or whether they still always go to libc, but we'll surely want to fix
it eventually if the latter is still true.)

In any case, that seems somewhat orthogonal to what we're on about here,
which is making the behavior of CREATE DATABASE less surprising and more
backwards-compatible.  I'm not sure that provider=none can help with that.
Aside from the user-surprise issues discussed up to now, pg_dump scripts
emitted by pre-v15 pg_dump are not going to contain LOCALE_PROVIDER
clauses in CREATE DATABASE, and people are going to be very unhappy
if that means they suddenly get totally different locale semantics
after restoring into a new DB.  I think we need some plan for mapping
libc-style locale specs into ICU locales so that we can make that
more nearly transparent.

> 2. Change the CREATE DATABASE syntax to catch these errors better at
> the possible expense of backwards compatibility.

That is the exact opposite of what I think we need.  Backwards
compatibility isn't optional.

Maybe this means we are not ready to do ICU-by-default in v16.
It certainly feels like there might be more here than we want to
start designing post-feature-freeze.

			regards, tom lane

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Introduce "builtin" collation provider.

  2. ICU: do not convert locale 'C' to 'en-US-u-va-posix'.

  3. initdb: change default --locale-provider back to libc.

  4. CREATE DATABASE: make LOCALE apply to all collation providers.

  5. Remove read-only server settings lc_collate and lc_ctype

  6. ICU: use uloc_getDefault() for initdb.

  7. Doc fixes for commit 1e16af8ab5.

  8. Doc: fix some rendering problems in the PDF docs build.

  9. Doc improvements for language tags and custom ICU collations.

  10. ICU: check for U_STRING_NOT_TERMINATED_WARNING.

  11. Reduce icu_validation_level default to WARNING.

  12. Revert "ICU: do not convert locale 'C' to 'en-US-u-va-posix'."

  13. Fix initdb --no-locale.

  14. Avoid character classification in regex escape parsing.