Re: BUG #17501: COPY is failing with "ERROR: invalid byte sequence for encoding "UTF8": 0xe5"

Heikki Linnakangas <hlinnaka@iki.fi>

From: Heikki Linnakangas <hlinnaka@iki.fi>
To: wizard_1024@tut.by
Cc: pgsql-bugs@lists.postgresql.org
Date: 2022-05-29T21:09:38Z
Lists: pgsql-bugs
On 29/05/2022 13:39, Heikki Linnakangas wrote:
> On 28/05/2022 23:52, PG Bug reporting form wrote:
>> The following bug has been logged on the website:
>>
>> Bug reference:      17501
>> Logged by:          Vitaly Voronov
>> Email address:      wizard_1024@tut.by
>> PostgreSQL version: 14.3
>> Operating system:   CentOS Linux release 7.9.2009 (Core)
>> Description:
>>
>> Hello,
>>
>> We've seen a such bug: COPY command shows error "ERROR:  invalid byte
>> sequence for encoding "UTF8": 0xe5" on file.
>> The same file with small amount of lines is imported without any errors.
> 
> I was able to reproduce this with your instructions. Thank you!
> 
> The bug was indeed introduced by commit f82de5c46b, in v14. Attached
> patch fixes it. Barring any objections, I'll commit and backpatch this.
> 
> I also noticed that there are some outdated comments in copyfromparse.c
> that still talk about the old protocol version 2 and COPY_OLD_FE. I'll
> clean that up as a separate commit.

Committed. And I included those comment changes in the same commit, 
after all.

Thanks for the report!

- Heikki



Commits

  1. Fix COPY FROM when database encoding is SQL_ASCII.

  2. Do COPY FROM encoding conversion/verification in larger chunks.