Re: UTF8 with BOM support in psql

Tom Lane <tgl@sss.pgh.pa.us>

From: Tom Lane <tgl@sss.pgh.pa.us>
To: Andrew Dunstan <andrew@dunslane.net>
Cc: Peter Eisentraut <peter_e@gmx.net>, Chuck McDevitt <cmcdevitt@greenplum.com>, Itagaki Takahiro <itagaki.takahiro@oss.ntt.co.jp>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2009-11-17T18:01:52Z
Lists: pgsql-hackers
Andrew Dunstan <andrew@dunslane.net> writes:
> Peter Eisentraut wrote:
>> Well, someone could implement UTF-16 or UTF-whatever as client encoding.
>> But I have not heard of any concrete proposals about that.

> Doesn't the nul byte problem make that seriously hard?

Just about impossible.  It would require a protocol bump, and removal of
C-style string usage *everywhere* on the client side.

Again, this is something that might be more feasible with encoding
conversion inside psql --- translating UTF16 to UTF8 immediately upon
reading it from any external file would confine the problem to possibly
manageable bounds.

			regards, tom lane