Horton

Re: GB18030-2022 Support in PostgreSQL

Chao Li <li.evan.chao@gmail.com>

View thread
From: Chao Li <li.evan.chao@gmail.com>
To: John Naylor <johncnaylorls@gmail.com>
Cc: Peter Eisentraut <peter@eisentraut.org>, pgsql-hackers@lists.postgresql.org, Tom Lane <tgl@sss.pgh.pa.us>, Andrew Dunstan <andrew@dunslane.net>
Date: 2025-09-24T09:31:39Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Generate EUC_CN mappings from gb18030-2022.ucm

  2. Update GB18030 encoding from version 2000 to 2022

  3. Generate GB18030 mappings from the Unicode Consortium's UCM file

Attachments

On Wed, Sep 24, 2025 at 5:18 PM Chao Li <li.evan.chao@gmail.com> wrote:

>
> On Sep 24, 2025, at 15:04, Chao Li <li.evan.chao@gmail.com> wrote:
>
> On Sep 24, 2025, at 14:42, John Naylor <johncnaylorls@gmail.com> wrote:
>
> Sounds good. Were you also interested in seeing if EUC_CN can use the
> same UCM file? That would allow us to get rid of the XML file.
>
>
> Sure, let me take a look.
>
>
> I found that both EUC_CN and UHC use the same XML file, so I updated both.
>
> I didn’t delete gb-18030-2000.xml in this patch, because it would make the
> patch file very large, you can just add the deletion to the commit when you
> push it.
>
> Basically, the changes are all borrowed from the previous commit. With
> this patch, regenerating the maps file lead to no map file change, which is
> expected:
>
> ```
> % make utf8_to_uhc.map utf8_to_euc_cn.map
> '/usr/bin/perl' -I . UCS_to_UHC.pl
> - Writing UTF8=>UHC conversion table: utf8_to_uhc.map
> - Writing UHC=>UTF8 conversion table: uhc_to_utf8.map
> '/usr/bin/perl' -I . UCS_to_EUC_CN.pl
> - Writing UTF8=>EUC_CN conversion table: utf8_to_euc_cn.map
> - Writing EUC_CN=>UTF8 conversion table: euc_cn_to_utf8.map
>
> % git diff # no map file change
> %
> ```
>
> I am not sure if you should also upgrade the UCM file to 2022 version, but
> if we need, let’s do it with a separate commit.
>
>
I included deletion of the xml file in v2, which will help confirm that
build will pass clearly. I realized that the patch files were huge because
of the map file changes.

Best regards,
Chao Li (Evan)
---------------------
HighGo Software Co., Ltd.
https://www.highgo.com/