Re: Add new COPY option REJECT_LIMIT

torikoshia <torikoshia@oss.nttdata.com>

From: torikoshia <torikoshia@oss.nttdata.com>
To: Kirill Reshke <reshkekirill@gmail.com>
Cc: Fujii Masao <masao.fujii@oss.nttdata.com>, zhjwpku@gmail.com, pgsql-hackers@lists.postgresql.org
Date: 2024-07-23T14:10:24Z
Lists: pgsql-hackers
On 2024-07-23 02:06, Kirill Reshke wrote:

Thanks for your review.

> Few comments:
> 
>> +      When a positive integer value is specified, 
>> <command>COPY</command> limits
>> +      the maximum tolerable number of errors while converting a 
>> column's input
>> +      value into its data type.
> 
> If nothing is specified, then the maximum tolerable number of errors
> is one, right? Should we state  this explicitly in the documentation?

REJECT_LIMIT now can be used wonly when on_error=ignore, I think the 
default(when only on_error=ignore is specified) is unlimited.
Anyway, I'm going to add a description about the default.

>> +COPY x from stdin with (on_error ignore, reject_limit 0);
> How about a test where reject_limit is a string, but not
> (case-intensively) 'infinity'?

Considering the discussion in[1], I'm now going to remove 'infinity'.

>> + CopyRejectLimits reject_limits; /* thresholds of reject_limit */
> 
> Why are there multiple thresholds? Can we have only one?

This is because I thought it'd be more convenient to support both the 
number and the ratio of error, but I'm also beginning to think that it 
might be better to support only the number of cases, as discussed in 
[1].

[1]https://www.postgresql.org/message-id/5f807fcf3a36df7ba41464ab40b5c37d%40oss.nttdata.com

-- 
Regards,

--
Atsushi Torikoshi
NTT DATA Group Corporation



Commits

  1. Move check for binary mode and on_error option to the appropriate location.

  2. Add REJECT_LIMIT option to the COPY command.