Re: confusing / inefficient "need_transcoding" handling in copy

Tom Lane <tgl@sss.pgh.pa.us>

From: Tom Lane <tgl@sss.pgh.pa.us>
To: Michael Paquier <michael@paquier.xyz>
Cc: Andres Freund <andres@anarazel.de>, pgsql-hackers@postgresql.org, Sutou Kouhei <kou@clear-code.com>, Tatsuo Ishii <ishii@sraoss.co.jp>
Date: 2024-02-06T17:51:48Z
Lists: pgsql-hackers
Michael Paquier <michael@paquier.xyz> writes:
> On Mon, Feb 05, 2024 at 06:05:04PM -0800, Andres Freund wrote:
>> I haven't yet dug into the code history. One guess is that this should only
>> have been set this way for COPY FROM.

> Looking the git history, this looks like an oversight of c61a2f58418e
> that has added the condition on pg_database_encoding_max_length(), no?
> Adding Tom and Ishii-san, even if this comes from 2005.

Yeah, back in 8.1 that code was applied for both directions, but
probably it should have enforced validation for same-encoding
cases only for COPY FROM.

It looks like now we have a mess, because the condition was copied
verbatim into copyto.c but not copyfrom.c.  Aren't we failing to
validate encoding in this case in COPY FROM, which is where we
actually need to?

			regards, tom lane



Commits

  1. Add some tests for encoding conversion in COPY TO/FROM

  2. Improve COPY TO performance when server and client encodings match