Re: Change COPY ... ON_ERROR ignore to ON_ERROR ignore_row

Kirill Reshke <reshkekirill@gmail.com>

From: Kirill Reshke <reshkekirill@gmail.com>
To: jian he <jian.universality@gmail.com>
Cc: Fujii Masao <masao.fujii@oss.nttdata.com>, Jim Jones <jim.jones@uni-muenster.de>, "David G. Johnston" <david.g.johnston@gmail.com>, Yugo NAGATA <nagata@sraoss.co.jp>, torikoshia <torikoshia@oss.nttdata.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2025-01-08T07:05:01Z
Lists: pgsql-hackers
Thank you for your update.

On Fri, 13 Dec 2024 at 08:15, jian he <jian.universality@gmail.com> wrote:
>
> + /*
> + * Here we end processing of current COPY row.
> + * Update copy state counter for number of erroneous rows.
> + */
> + cstate->num_errors++;
> + cstate->escontext->error_occurred = true;
> +
> + /* Only print this NOTICE message, if it will not be followed by ERROR */
> + if (cstate->opts.log_verbosity == COPY_LOG_VERBOSITY_VERBOSE &&
> + (
> + (cstate->opts.on_error == COPY_ON_ERROR_NULL &&
> cstate->opts.reject_limit > 0 && cstate->num_errors <=
> cstate->opts.reject_limit) ||
> + (cstate->opts.on_error == COPY_ON_ERROR_IGNORE &&
> (cstate->opts.reject_limit ==  0 || cstate->num_errors <=
> cstate->opts.reject_limit))
> + ))
>   {
> this is kind of hard to comprehend.
> so attached is a simple version of it based on v8.

+1

So, in this version you essentially removed support for REJECT_LIMIT +
SET_TO_NULL feature? Looks like a promising change. It is more likely
to see this committed.
So, +1 on that too.

However, v9 lacks tests for REJECT_LIMIT vs erroneous rows tests.
In short, we need  this message somewhere in a regression test.
```
ERROR:  skipped more than REJECT_LIMIT (xxx) rows due to data type
incompatibility
```

Also, please update commit msg with all authors and reviewers. This
will make committer job a little bit easier

--
Best regards,
Kirill Reshke



Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →
  1. file_fdw: Add regression test for file_fdw with ON_ERROR='set_null'

  2. Add COPY (on_error set_null) option

  3. Add REJECT_LIMIT option to the COPY command.

  4. Add log_verbosity = 'silent' support to COPY command.

  5. Add new COPY option LOG_VERBOSITY.