Re: invalid UTF-8 via pl/perl

Andrew Dunstan <andrew@dunslane.net>

From: Andrew Dunstan <andrew@dunslane.net>
To: Hannu Krosing <hannu@2ndquadrant.com>
Cc: pgsql-hackers@postgresql.org
Date: 2010-01-03T19:54:53Z
Lists: pgsql-hackers

I wrote:
>
> I think the attached patch plugs the direct SPI holes as well.

There are two issues with this patch. First, how far if at all should it 
be backpatched? All the way, or 8.3, where we tightened the encoding 
rules, or not at all?

Second, It produces errors like this:

    andrew=# select 'a' || invalid_utf_seq() || 'b';
    ERROR:  invalid byte sequence for encoding "UTF8": 0xd0
    HINT:  This error can also happen if the byte sequence does not
    match the encoding expected by the server, which is controlled by
    "client_encoding".
    CONTEXT:  PL/Perl function "invalid_utf_seq"
    andrew=#


That hint seems rather misleading. I'm not sure what we can do about it 
though. If we set the noError param on pg_verifymbstr() we would miss 
the error message that actually identified the bad data, so that doesn't 
seem like a good plan.

cheers

andrew