Horton

Re: badly calculated width of emoji in psql

Jacob Champion <pchampion@vmware.com>

View thread View on postgr.es
From: Jacob Champion <pchampion@vmware.com>
To: "john.naylor@enterprisedb.com" <john.naylor@enterprisedb.com>
Cc: "pavel.stehule@gmail.com" <pavel.stehule@gmail.com>, "laurenz.albe@cybertec.at" <laurenz.albe@cybertec.at>, "horikyota.ntt@gmail.com" <horikyota.ntt@gmail.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>, "peter.eisentraut@enterprisedb.com" <peter.eisentraut@enterprisedb.com>, "michael@paquier.xyz" <michael@paquier.xyz>
Date: 2021-08-26T15:25:22Z
Lists: pgsql-hackers 
On Wed, 2021-08-25 at 16:15 -0400, John Naylor wrote:
> On Tue, Aug 24, 2021 at 1:50 PM Jacob Champion <pchampion@vmware.com> wrote:
> >
> > Does there need to be any sanity check for overlapping ranges between
> > the combining and fullwidth sets? The Unicode data on a dev's machine
> > would have to be broken somehow for that to happen, but it could
> > potentially go undetected for a while if it did.
> 
> It turns out I should have done that to begin with. In the Unicode
> data, it apparently happens that a character can be both combining
> and wide, and that will cause ranges to overlap in my scheme:

I was looking for overlaps in my review, but I skipped right over that,
sorry...

On Thu, 2021-08-26 at 11:12 -0400, John Naylor wrote:
> I pushed Jacob's patch with the addendum I shared upthread, plus a
> comment about search order. I also took the liberty of changing the
> author in the CF app to Jacob. Later I'll push detecting non-spacing
> characters beyond the BMP.

Thanks!

--Jacob

Commits

  1. Extend collection of Unicode combining characters to beyond the BMP

  2. Update display widths as part of updating Unicode

  3. Revert "Rename unicode_combining_table to unicode_width_table"

  4. Change mbbisearch to return the character range