Horton
Menu

Re: ICU for global collation

Finnerty, Jim <jfinnert@amazon.com>

View thread
From: "Finnerty, Jim" <jfinnert@amazon.com>
To: Julien Rouhaud <rjuju123@gmail.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>, Daniel Verite <daniel@manitou-mail.org>
Date: 2022-01-06T13:55:55Z
Lists: pgsql-hackers 
I didn't notice anything version-specific about the patch.  Would any modifications be needed to backport it to pg13 and pg14?

After this patch goes in, the big next thing would be to support nondeterministic collations for LIKE, ILIKE and pattern matching operators in general.  Is anyone interested in working on that?

On 1/5/22, 10:36 PM, "Julien Rouhaud" <rjuju123@gmail.com> wrote:

    CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.



    On Tue, Jan 04, 2022 at 05:03:10PM +0100, Peter Eisentraut wrote:
    > On 04.01.22 03:21, Julien Rouhaud wrote:
    >
    > > > - if (!lc_collate_is_c(collid) && collid != DEFAULT_COLLATION_OID)
    > > > -         mylocale = pg_newlocale_from_collation(collid);
    > > > + if (!lc_collate_is_c(collid))
    > > > + {
    > > > +         if (collid != DEFAULT_COLLATION_OID)
    > > > +                 mylocale = pg_newlocale_from_collation(collid);
    > > > +         else if (default_locale.provider == COLLPROVIDER_ICU)
    > > > +                 mylocale = &default_locale;
    > > > + }
    > >
    > > There are really a lot of places with this new code.  Maybe it could be some
    > > new function/macro to wrap that for the normal case (e.g. not formatting.c)?
    >
    > Right, we could just put this into pg_newlocale_from_collation(), but the
    > comment there says
    >
    >  * In fact, they shouldn't call this function at all when they are dealing
    >  * with the default locale.  That can save quite a bit in hotspots.
    >
    > I don't know how to assess that.
    >
    > We could pack this into a macro or inline function if we are concerned about
    > this.

    Yes that was my idea, just have a new function (inline function or a macro
    then since pg_newlocale_from_collation() clearly warns about performance
    concerns) that have the whole
    is-not-c-collation-and-is-default-collation-or-icu-collation logic and calls
    pg_newlocale_from_collation() only when needed.

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add CHECK_FOR_INTERRUPTS while restoring changes during decoding.

  2. Improve ICU option handling in CREATE DATABASE

  3. Don't allow creation of database with ICU locale with unsupported encoding

  4. Make locale option behavior more consistent

  5. pg_dump: Dump colliculocale

  6. Remove further unwanted linker flags from perl_embed_ldflags

  7. Doc: document possible need to raise kernel's somaxconn limit.

  8. Reduce warnings with -Wshadow=compatible-local builds

  9. Remove redundant spaces in _outA_Expr() output

  10. Fix outdated --help message for postgres -f

  11. pg_upgrade: Fix version comparison for global ICU support

  12. Silence -Wmaybe-uninitialized compiler warning in dbcommands.c.

  13. Add option to use ICU as global locale provider

  14. DefineCollation() code cleanup

  15. Change collate and ctype fields to type text

  16. Call pg_newlocale_from_collation() also with default collation

  17. Use libc version as a collation version on glibc systems.

  18. Silence -Wmaybe-uninitialized compiler warnings in dbcommands.c.

  19. Make LC_COLLATE and LC_CTYPE database-level settings. Collation and