Horton

Re: GB18030-2022 Support in PostgreSQL

Chao Li <li.evan.chao@gmail.com>

View thread
From: Chao Li <li.evan.chao@gmail.com>
To: John Naylor <johncnaylorls@gmail.com>
Cc: Peter Eisentraut <peter@eisentraut.org>, pgsql-hackers@lists.postgresql.org, Tom Lane <tgl@sss.pgh.pa.us>, Andrew Dunstan <andrew@dunslane.net>
Date: 2025-09-24T09:18:40Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Generate EUC_CN mappings from gb18030-2022.ucm

  2. Update GB18030 encoding from version 2000 to 2022

  3. Generate GB18030 mappings from the Unicode Consortium's UCM file

Attachments

On Sep 24, 2025, at 15:04, Chao Li <li.evan.chao@gmail.com> wrote:

On Sep 24, 2025, at 14:42, John Naylor <johncnaylorls@gmail.com> wrote:

Sounds good. Were you also interested in seeing if EUC_CN can use the
same UCM file? That would allow us to get rid of the XML file.


Sure, let me take a look.


I found that both EUC_CN and UHC use the same XML file, so I updated both.

I didn’t delete gb-18030-2000.xml in this patch, because it would make the
patch file very large, you can just add the deletion to the commit when you
push it.

Basically, the changes are all borrowed from the previous commit. With this
patch, regenerating the maps file lead to no map file change, which is
expected:

```
% make utf8_to_uhc.map utf8_to_euc_cn.map
'/usr/bin/perl' -I . UCS_to_UHC.pl
- Writing UTF8=>UHC conversion table: utf8_to_uhc.map
- Writing UHC=>UTF8 conversion table: uhc_to_utf8.map
'/usr/bin/perl' -I . UCS_to_EUC_CN.pl
- Writing UTF8=>EUC_CN conversion table: utf8_to_euc_cn.map
- Writing EUC_CN=>UTF8 conversion table: euc_cn_to_utf8.map

% git diff # no map file change
%
```

I am not sure if you should also upgrade the UCM file to 2022 version, but
if we need, let’s do it with a separate commit.

Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/