Re: plperlu problem with utf8

On Fri, Dec 17, 2010 at 08:30, David Fetter <david@fetter.org> wrote:
> On Thu, Dec 16, 2010 at 07:24:46PM -0800, David Wheeler wrote:
>> On Dec 16, 2010, at 6:39 PM, Alex Hunsaker wrote:
>>
>> > Grr that should error out with "Invalid server encoding", or worst
>> > case should return a length of 3 (it utf8 encoded 128 into 2 bytes
>> > instead of leaving it as 1).  In this case the 333 causes perl
>> > store it internally as utf8.
>>
>> Well with SQL_ASCII anything goes, no?
>
> Anything except a byte that's all 0s :(

Keyword being byte, right?  Last time I checked 333 wont fit in a byte
:P.  In this case perl stores "333" as the utf8 that represents the
unicode code point 333.  Postgres uses whatever that internal
representation is, so in our SQL_ASCII database we actually end up
getting back utf8 which _is_ valid SQL_ASCII, but I wouldn't call it
"right". The behavior im aiming for is similar to what the built-in
chr does:
# SELECT chr(333);
ERROR:  requested character too large for encoding: 333

Also note this is just a simple test case, perl *could* elect to store
completely ascii strings internally as utf8.  In those cases we still
"do the wrong thing", that is we get back utf8ified bytes :(  Although
obviously from the lack of bug reports its quite uncommon.