Re: Change COPY ... ON_ERROR ignore to ON_ERROR ignore_row

jian he <jian.universality@gmail.com>

From: jian he <jian.universality@gmail.com>
To: Kirill Reshke <reshkekirill@gmail.com>
Cc: Fujii Masao <masao.fujii@oss.nttdata.com>, Jim Jones <jim.jones@uni-muenster.de>, "David G. Johnston" <david.g.johnston@gmail.com>, Yugo NAGATA <nagata@sraoss.co.jp>, torikoshia <torikoshia@oss.nttdata.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2025-01-10T06:38:20Z
Lists: pgsql-hackers

Attachments

On Wed, Jan 8, 2025 at 3:05 PM Kirill Reshke <reshkekirill@gmail.com> wrote:
>
> So, in this version you essentially removed support for REJECT_LIMIT +
> SET_TO_NULL feature? Looks like a promising change. It is more likely
> to see this committed.
> So, +1 on that too.
>
> However, v9 lacks tests for REJECT_LIMIT vs erroneous rows tests.
> In short, we need  this message somewhere in a regression test.
> ```
> ERROR:  skipped more than REJECT_LIMIT (xxx) rows due to data type
> incompatibility
> ```
>

hi.
you already answered this question.
since we do not support REJECT_LIMIT+SET_TO_NULL,
so these code path would not be reachable.

> Also, please update commit msg with all authors and reviewers. This
> will make committer job a little bit easier
>
commit message polished.
here and there cosmetic changes.

I think there are three remaining issues that may need more attention
1.
Table 27.42. pg_stat_progress_copy View
(<structname>pg_stat_progress_copy</structname>)
column pg_stat_progress_copy.tuples_skipped now the description is
""
When the ON_ERROR option is set to ignore, this value shows the number of tuples
skipped due to malformed data. When the ON_ERROR option is set to set_to_null,
this value shows the number of tuples where malformed data was converted to
NULL.
"""
now the column name tuples_skipped would not be that suitable for
(on_error set_to_null).
since now it is not tuple skipped, it is in a tuple some value was set to null.
Or
we can skip progress reports for (on_error set_to_null) case.

2. The doc is not very great, I guess.
3. do we settled (on_error set_to_null) syntax.

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →
  1. file_fdw: Add regression test for file_fdw with ON_ERROR='set_null'

  2. Add COPY (on_error set_null) option

  3. Add REJECT_LIMIT option to the COPY command.

  4. Add log_verbosity = 'silent' support to COPY command.

  5. Add new COPY option LOG_VERBOSITY.