unaccent fails when datlocprovider=i and datctype=C

Jeff Davis <pgsql@j-davis.com>

From: Jeff Davis <pgsql@j-davis.com>
To: pgsql-bugs@postgresql.org
Date: 2023-03-08T05:49:12Z
Lists: pgsql-bugs

Attachments

Repro:

$ initdb -D data -N --locale-provider=icu --icu-locale=en --locale=C

=# create extension unaccent;
ERROR:  invalid multibyte character for locale
HINT:  The server's LC_CTYPE locale is probably incompatible with the
database encoding.
CONTEXT:  line 1 of configuration file
".../share/tsearch_data/unaccent.rules": "¡  !
"

Cause: t_isspace() implementation is incomplete (notice "TODO"
comments):

    Oid         collation = DEFAULT_COLLATION_OID;  /* TODO */
    pg_locale_t mylocale = 0;   /* TODO */

    if (clen == 1 || lc_ctype_is_c(collation))
        return isspace(TOUCHAR(ptr));

    char2wchar(character, WC_BUF_LEN, ptr, clen, mylocale);

    return iswspace((wint_t) character[0]);

If using datlocprovider=c, then the earlier branch goes straight to
isspace(). But if datlocprovider=i, then
lc_ctype_is_c(DEFAULT_COLLATION_OID) returns false, and it goes into
char2wchar(). char2wchar() is essentially a wrapper around mbstowcs(),
which does not work on multibyte input when LC_CTYPE=C. 

Quick fix (attached): check whether datctype is C rather than the
default collation.

Eventually this should be fixed by doing character classification in
ICU when the provider is ICU.

-- 
Jeff Davis
PostgreSQL Contributor Team - AWS


Commits

  1. Fix t_isspace(), etc., when datlocprovider=i and datctype=C.