Re: Add new error_action COPY ON_ERROR "log"

Masahiko Sawada <sawada.mshk@gmail.com>

From: Masahiko Sawada <sawada.mshk@gmail.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>, torikoshia <torikoshia@oss.nttdata.com>, "David G. Johnston" <david.g.johnston@gmail.com>, jian.universality@gmail.com, Pgsql Hackers <pgsql-hackers@postgresql.org>, Alexander Korotkov <aekorotkov@gmail.com>
Date: 2024-03-07T06:52:41Z
Lists: pgsql-hackers
On Thu, Mar 7, 2024 at 1:00 PM Michael Paquier <michael@paquier.xyz> wrote:
>
> On Wed, Mar 06, 2024 at 07:32:28PM +0530, Bharath Rupireddy wrote:
> > Please see the attached v4 patch. If it looks good, I can pull
> > LOG_VERBOSITY changes out into 0001 and with 0002 containing the
> > detailed messages for discarded rows.
>
> The approach looks sensible seen from here.

Looks like a good approach to me.

>
> +    LOG_VERBOSITY [ <replaceable class="parameter">boolean</replaceable> ]
> [...]
> +    <term><literal>LOG_VERBOSITY</literal></term>
> +    <listitem>
> +     <para>
> +      Sets the verbosity of logged messages by <command>COPY</command>
> +      command. As an example, see its usage for
> +      <command>COPY FROM</command> command's <literal>ON_ERROR</literal>
> +      clause with <literal>ignore</literal> option.
> +     </para>
> +    </listitem>
>
> Is a boolean the best interface for the end-user, though?  Maybe
> something like a "mode" value would speak more than a yes/no from the
> start, say a "default" mode to emit only the last LOG and a "verbose"
> for the whole set in the case of ON_ERROR?  That could use an enum
> from the start internally, but that's an implementation detail.

+1 for making it an enum, so that we will be able to have multiple
levels for example to get actual soft error contents.

One question I have is; do we want to write multiple NOTICE messages
for one row if the row has malformed data on some columns?

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com



Commits

  1. Add new COPY option LOG_VERBOSITY.

  2. Add tests for more row patterns with COPY FROM .. (ON_ERROR ignore)

  3. Support writing "CREATE/ALTER TABLE ... SET STORAGE DEFAULT".