Re: Add new error_action COPY ON_ERROR "log"

torikoshia <torikoshia@oss.nttdata.com>

From: torikoshia <torikoshia@oss.nttdata.com>
To: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Cc: "David G. Johnston" <david.g.johnston@gmail.com>, jian.universality@gmail.com, Pgsql Hackers <pgsql-hackers@postgresql.org>
Date: 2024-02-20T08:22:58Z
Lists: pgsql-hackers
On 2024-02-17 15:00, Bharath Rupireddy wrote:
> On Fri, Feb 16, 2024 at 8:17 PM torikoshia <torikoshia@oss.nttdata.com> 
> wrote:
>> 
>> I may be wrong since I seldom do data loading tasks, but I greed with
>> you.
>> 
>> I also a little concerned about the case where there are many 
>> malformed
>> data and it causes lots of messages, but the information is usually
>> valuable and if users don't need it, they can suppress it by changing
>> client_min_messages.
>> 
>> Currently both summary of failures and individual information is 
>> logged
>> in NOTICE level.
>> If we should assume that there are cases where only summary 
>> information
>> is required, it'd be useful to set lower log level, i.e. LOG to the
>> individual information.
> 
> How about we emit the summary at INFO level and individual information
> at NOTICE level? With this, the summary is given a different priority
> than the individual info. With SET client_min_messages = WARNING; one
> can still get the summary but not the individual info. Also, to get
> all of these into server log, one can SET log_min_messages = INFO; or
> SET log_min_messages = NOTICE;.
> 
> Thoughts?

It looks good to me.

-- 
Regards,

--
Atsushi Torikoshi
NTT DATA Group Corporation



Commits

  1. Add new COPY option LOG_VERBOSITY.

  2. Add tests for more row patterns with COPY FROM .. (ON_ERROR ignore)

  3. Support writing "CREATE/ALTER TABLE ... SET STORAGE DEFAULT".