Add tuples_skipped to pg_stat_progress_copy
torikoshia <torikoshia@oss.nttdata.com>
From: torikoshia <torikoshia@oss.nttdata.com>
To: Pgsql Hackers <pgsql-hackers@postgresql.org>
Date: 2024-01-17T05:22:03Z
Lists: pgsql-hackers
Attachments
- v1-0001-Add-tuples_skipped-to-pg_stat_progress_copy.patch (text/x-diff) patch v1-0001
Hi, 132de9968840c introduced SAVE_ERROR_TO option to COPY and enabled to skip malformed data, but there is no way to watch the number of skipped rows during COPY. Attached patch adds tuples_skipped to pg_stat_progress_copy, which counts the number of skipped tuples because source data is malformed. If SAVE_ERROR_TO is not specified, this column remains zero. The advantage would be that users can quickly notice and stop COPYing when there is a larger amount of skipped data than expected, for example. As described in commit log, it is expected to add more choices for SAVE_ERROR_TO like 'log' and using such options may enable us to know the number of skipped tuples during COPY, but exposed in pg_stat_progress_copy would be easier to monitor. What do you think? -- Regards, -- Atsushi Torikoshi NTT DATA Group Corporation
Commits
-
Add progress reporting of skipped tuples during COPY FROM.
- 729439607ad2 17.0 landed