Horton

Re: Built-in CTYPE provider

Tom Lane <tgl@sss.pgh.pa.us>

View thread View on postgr.es
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Noah Misch <noah@leadboat.com>
Cc: Jeff Davis <pgsql@j-davis.com>, Peter Eisentraut <peter@eisentraut.org>, Daniel Verite <daniel@manitou-mail.org>, Robert Haas <robertmhaas@gmail.com>, Jeremy Schneider <schneider@ardentperf.com>, pgsql-hackers@postgresql.org
Date: 2024-07-06T20:19:21Z
Lists: pgsql-hackers 
Noah Misch <noah@leadboat.com> writes:
> As a released feature, NORMALIZE() has a different set of remedies to choose
> from, and I'm not proposing one.  I may have sidetracked this thread by
> talking about remedies without an agreement that pg_c_utf8 has a problem.  My
> question for the PostgreSQL maintainers is this:

>   textregexeq(... COLLATE pg_c_utf8, '[[:alpha:]]') and lower(), despite being
>   IMMUTABLE, will change behavior in some major releases.  pg_upgrade does not
>   have a concept of IMMUTABLE functions changing, so index scans will return
>   wrong query results after upgrade.  Is it okay for v17 to release a
>   pg_c_utf8 planned to behave that way when upgrading v17 to v18+?

I do not think it is realistic to define "IMMUTABLE" as meaning that
the function will never change behavior until the heat death of the
universe.  As a counterexample, we've not worried about applying
bug fixes or algorithm improvements that change the behavior of
"immutable" numeric computations.  It might be unwise to do that
in a minor release, but we certainly do it in major releases.

I'd say a realistic policy is "immutable means we don't intend to
change it within a major release".  If we do change the behavior,
either as a bug fix or a major-release improvement, that should
be release-noted so that people know they have to rebuild dependent
indexes and matviews.

It gets stickier for behaviors that aren't fully under our control,
which is the case for a lot of locale-related things.  We cannot then
promise "no changes within major releases".  But I do not think it
is helpful to react to that fact by refusing to label such things
immutable.  Then we'd just need another mutability classification,
and it would effectively act the same as immutable does now, because
people will certainly wish to use these functions in indexes etc.

			regards, tom lane

Commits

  1. Support PG_UNICODE_FAST locale in the builtin collation provider.

  2. Support Unicode full case mapping and conversion.

  3. Fix test failures when language environment is not UTF-8.

  4. Add unicode_strtitle() for Unicode Default Case Conversion.

  5. Use version for builtin collations.

  6. Fix convert_case(), introduced in 5c40364dd6.

  7. Inline basic UTF-8 functions.

  8. Support C.UTF-8 locale in the new builtin collation provider.

  9. Fix another warning, introduced by 846311051e.

  10. Address more review comments on commit 2d819a08a1.

  11. Fix unreachable code warning from commit 2d819a08a1.

  12. Introduce "builtin" collation provider.

  13. Catalog changes preparing for builtin collation provider.

  14. Unicode case mapping tables and functions.

  15. Add Unicode property tables.

  16. Documentation update for Standard Collations.

  17. Cleanup for unicode-update build target and test.

  18. Shrink Unicode category table.

  19. Make some error strings more generic

  20. pg_upgrade: copy locale and encoding information to new cluster.

  21. Update Unicode data to Unicode 15.0.0

  22. Create a new type category for "internal use" types.