Re: Add new COPY option REJECT_LIMIT

On Fri, Jan 26, 2024 at 2:49 AM torikoshia <torikoshia@oss.nttdata.com>
wrote:

> Hi,
>
> 9e2d870 enabled the COPY command to skip soft error, and I think we can
> add another option which specifies the maximum tolerable number of soft
> errors.
>
> I remember this was discussed in [1], and feel it would be useful when
> loading 'dirty' data but there is a limit to how dirty it can be.
>
> Attached a patch for this.
>
> What do you think?
>
>
I'm opposed to adding this particular feature.

When implementing this kind of business rule I'd need the option to specify
a percentage, not just an absolute value.

I would focus on trying to put the data required to make this kind of
determination into a place where applications implementing such business
rules and monitoring can readily get at it.  The "ERRORS TO" and maybe a
corresponding "STATS TO" option where a table can be specified for the
system to place the problematic data and stats about the copy itself.

David J.