Horton

Re: Perform COPY FROM encoding conversions in larger chunks

Heikki Linnakangas <hlinnaka@iki.fi>

View thread View on postgr.es
From: Heikki Linnakangas <hlinnaka@iki.fi>
To: John Naylor <john.naylor@enterprisedb.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2021-02-09T17:44:46Z
Lists: pgsql-hackers

Attachments

On 09/02/2021 19:36, Heikki Linnakangas wrote:
> On 09/02/2021 15:40, John Naylor wrote:
>> On Sun, Feb 7, 2021 at 2:13 PM Heikki Linnakangas <hlinnaka@iki.fi
>> <mailto:hlinnaka@iki.fi>> wrote:
>>   >
>>   > On 02/02/2021 23:42, John Naylor wrote:
>>   > >
>>   > > In copyfromparse.c, this is now out of date:
>>   > >
>>   > >   * Read the next input line and stash it in line_buf, with
>> conversion to
>>   > >   * server encoding.
>>
>> This comment for CopyReadLine() is still there. Conversion already
>> happened by now, so I think this comment is outdated.
>>
>> Other than that, I think this is ready for commit.
> 
> Fixed. And also fixed one more bug in allocating raw_buf_size, the "+ 1"
> somehow went missing again. That was causing a failure on Windows at
> cfbot.cputube.org.
> 
> I'll read through this one more time with fresh eyes tomorrow or the day
> after, and push. Thanks for all the review!

Forgot attachment..

- Heikki

Commits

  1. Do COPY FROM encoding conversion/verification in larger chunks.

  2. Add 'noError' argument to encoding conversion functions.

  3. Add direct conversion routines between EUC_TW and Big5.

  4. Add mbverifystr() functions specific to each encoding.