Horton

Re: Supporting SJIS as a database encoding

Tatsuo Ishii <ishii@sraoss.co.jp>

View thread View on postgr.es
From: Tatsuo Ishii <ishii@sraoss.co.jp>
To: tsunakawa.takay@jp.fujitsu.com
Cc: pgsql-hackers@postgresql.org
Date: 2016-09-05T07:38:54Z
Lists: pgsql-hackers 
> But what I'm wondering is why PostgreSQL doesn't support SJIS.  Was there any technical difficulty?  Is there anything you are worried about if adding SJIS?

Yes, there's a technical difficulty with backend code. In many places
it is assumed that any string is "ASCII compatible", which means no
ASCII character is used as a part of multi byte string. Here is such a
random example from src/backend/util/adt/varlena.c:

	/* Else, it's the traditional escaped style */
	for (bc = 0, tp = inputText; *tp != '\0'; bc++)
	{
		if (tp[0] != '\\')
			tp++;

Sometimes SJIS uses '\' as the second byte of it.

Best regards,
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese:http://www.sraoss.co.jp

Commits

  1. Use radix tree for character encoding conversions.

  2. Small fixes to the Perl scripts to create unicode conversion tables.

  3. Rewrite the perl scripts to produce our Unicode conversion tables.

  4. Remove leading zeros, for consistency with other map files.

  5. Remove code points < 0x80 from character conversion tables.