Re: Add new error_action COPY ON_ERROR "log"

Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>

From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
To: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: Michael Paquier <michael@paquier.xyz>, torikoshia <torikoshia@oss.nttdata.com>, "David G. Johnston" <david.g.johnston@gmail.com>, jian.universality@gmail.com, Pgsql Hackers <pgsql-hackers@postgresql.org>, Alexander Korotkov <aekorotkov@gmail.com>
Date: 2024-03-26T06:04:11Z
Lists: pgsql-hackers

Attachments

On Tue, Mar 26, 2024 at 9:56 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
>
> > > errmsg("data type incompatibility at line %llu for column %s: \"%s\"",
>
> > > I guess it would be better to make the log message clearer to convey
> > > what we did for the malformed row. For example, how about something
> > > like "skipping row due to data type incompatibility at line %llu for
> > > column %s: \"s\""?
> >
> > The summary message which gets printed at the end says that "NOTICE:
> > 6 rows were skipped due to data type incompatibility". Isn't this
> > enough? If someone is using ON_ERROR 'ignore', it's quite natural that
> > such rows get skipped softly and the summary message can help them,
> > no?
>
> I think that in the main log message we should mention what happened
> (or is happening) or what we did (or are doing). If the message "data
> type incompatibility ..." was in the DETAIL message with the main
> message saying something like "skipping row at line %llu for column
> %s: ...", it would make sense to me. But the current message seems not
> to be clear to me and consistent with other NOTICE messages. Also, the
> last summary line would not be written if the user cancelled, and
> someone other than person who used ON_ERROR 'ignore' might check the
> server logs later.

Agree. I changed the NOTICE message to what you've suggested. Thanks.

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Commits

  1. Add new COPY option LOG_VERBOSITY.

  2. Add tests for more row patterns with COPY FROM .. (ON_ERROR ignore)

  3. Support writing "CREATE/ALTER TABLE ... SET STORAGE DEFAULT".