Horton

Re: Collation & ctype method table, and extension hooks

Jeff Davis <pgsql@j-davis.com>

View thread View on postgr.es
From: Jeff Davis <pgsql@j-davis.com>
To: Andreas Karlsson <andreas@proxel.se>, pgsql-hackers@postgresql.org
Date: 2024-11-19T21:32:47Z
Lists: pgsql-hackers

Attachments

On Fri, 2024-11-01 at 14:08 +0100, Andreas Karlsson wrote:
> I think adding such a small file would make life easier for people
> new 
> to the collation part of the code base. It would be a nice symmetry 
> between collation providers and where code for them can be found.

Done.

> > 
> For me combining them would make the intention of the code easier to 
> understand since aren't the casemap functions just a set of
> "ctype_methods"?

Done.

There is a bit of weirdness in libc because:

* Single byte encodings use the single-byte isupper(), toupper(), etc.
* UTF8 encoding uses wide character iswupper(), towupper(), etc.
* Non-UTF8 multibyte encodings use isupper() for pattern matching but
towupper() for case mapping

that weirdness existed before, but it's a bit more obvious what's
happening now.

> > > This commit makes me tempted to handle the ctype_is_c logic for
> > > character classes also in callbacks and remove the if in
> > > functions
> > > like
> > > pg_wc_ispunct(). But this si something that would need to be
> > > benchmarked.

I like this idea, but it can be a follow up.


Attached new patchset.

I also tried some performance tests again. I used smalltext (a table of
10M ~30-character strings) and bigtext (a table of 32768 rows, each
containing the 100KiB source of https://en.wikipedia.org/wiki/Diacritic
). And I then ran the following regex on each:

    select count(*) from thetable
      where t ~
'[[:digit:]][[:space:]][[:punct:]][[:alpha:]][[:lower:]][[:upper:]]';

for "C", "en_US", and "en-US-x-icu". The timings for smalltext were
indistinguishable between master and the patched version. The timings
for bigtext were pretty noisy so it's hard to tell if there was a
regression or not, but I saw some evidence in the profile that
char_properties has a cost (~1%). I'm not sure if that's a significant
concern or not.

Which API do you think is the right one? Individual functions testing
individual properties, or something like char_properties() that can
test several at once?

Regards,
	Jeff Davis

Commits

  1. Remove provider field from pg_locale_t.

  2. Control ctype behavior internally with a method table.

  3. Control collation behavior with a method table.

  4. Move code for collation version into provider-specific files.

  5. Refactor string case conversion into provider-specific files.

  6. Move check for ucol_strcollUTF8 to pg_locale_icu.c

  7. Perform provider-specific initialization in new functions.

  8. Refactor the code to create a pg_locale_t into new function.