Re: Change COPY ... ON_ERROR ignore to ON_ERROR ignore_row

jian he <jian.universality@gmail.com>

From: jian he <jian.universality@gmail.com>
To: Kirill Reshke <reshkekirill@gmail.com>
Cc: Fujii Masao <masao.fujii@oss.nttdata.com>, Jim Jones <jim.jones@uni-muenster.de>, "David G. Johnston" <david.g.johnston@gmail.com>, Yugo NAGATA <nagata@sraoss.co.jp>, torikoshia <torikoshia@oss.nttdata.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2025-01-14T05:51:20Z
Lists: pgsql-hackers

Attachments

On Sat, Jan 11, 2025 at 5:54 PM Kirill Reshke <reshkekirill@gmail.com> wrote:
>
> On Fri, 10 Jan 2025 at 11:38, jian he <jian.universality@gmail.com> wrote:
> > I think there are three remaining issues that may need more attention
> > 1.
> > Table 27.42. pg_stat_progress_copy View
> > (<structname>pg_stat_progress_copy</structname>)
> > column pg_stat_progress_copy.tuples_skipped now the description is
> > ""
> > When the ON_ERROR option is set to ignore, this value shows the number of tuples
> > skipped due to malformed data. When the ON_ERROR option is set to set_to_null,
> > this value shows the number of tuples where malformed data was converted to
> > NULL.
> > """
> > now the column name tuples_skipped would not be that suitable for
> > (on_error set_to_null).
> > since now it is not tuple skipped, it is in a tuple some value was set to null.
>
> Indeed this is something we need to fix.
>
> > Or
> > we can skip progress reports for (on_error set_to_null) case.
>
> Maybe we can add a `malformed_tuples` column to this view?
>
we can do this later.
so for on_error set_to_null, i've removed pgstat_progress_update_param
related code.

the attached patch also did some doc enhancement, error message enhancement.

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →
  1. file_fdw: Add regression test for file_fdw with ON_ERROR='set_null'

  2. Add COPY (on_error set_null) option

  3. Add REJECT_LIMIT option to the COPY command.

  4. Add log_verbosity = 'silent' support to COPY command.

  5. Add new COPY option LOG_VERBOSITY.