Horton

Re: Collation versioning

Douglas Doole <dougdoole@gmail.com>

View thread View on postgr.es
From: Douglas Doole <dougdoole@gmail.com>
To: Greg Stark <stark@mit.edu>
Cc: Thomas Munro <thomas.munro@enterprisedb.com>, Peter Eisentraut <peter.eisentraut@2ndquadrant.com>, Christoph Berg <myon@debian.org>, Pg Hackers <pgsql-hackers@postgresql.org>
Date: 2018-09-17T19:56:49Z
Lists: pgsql-hackers 
On Mon, Sep 17, 2018 at 12:32 PM Greg Stark <stark@mit.edu> wrote:

> This seems like a terrible idea in the open source world. Surely collation
> versioning means new ICU libraries can still provide the old collation
> rules so even if you update the library you can request the old version? We
> shouldn't need that actual old code with all its security holes and bugs
> just to get the old collation version.
>

We asked long and hard for this feature from the ICU team but they kept
arguing it was too hard to do. There are apparently some tight couplings
between the code and each version of CLDR. So the only way to support old
collations is to ship the entire old library. (They even added make rules
to allow the entire API to be version extended to accommodate this
requirement.)

Even bug fixes are potentially problematic because the fix may alter how
some code points collate. The ICU team won't (or at least wouldn't - been a
few years since I dealt with them) guarantee any sort of backwards
compatibility between code drops.

As an aside, they did look at making the CLDR data a separate data file
that could be read by any version of the code (before finding there were
too many dependencies). One thing that they discovered is that this
approach didn't save much disk since the CLDR data is something like 90-95%
of the library. So while it would have made the calling code a lot cleaner,
it wasn't the huge footprint win we'd been hoping for.

Commits

  1. Doc: Document known problem with Windows collation versions.

  2. Add collation versions for FreeBSD.

  3. Tolerate version lookup failure for old style Windows locale names.

  4. Track collation versions for indexes.

  5. Add pg_depend.refobjversion.

  6. Remove pg_collation.collversion.

  7. Fix the MSVC build for versions 2015 and later.

  8. Add collation versions for Windows.

  9. Implement type regcollation

  10. Use libc version as a collation version on glibc systems.

  11. Make type "name" collation-aware.