Re: Enhance file_fdw to report processed and skipped tuples in COPY progress

On Thu, Oct 3, 2024 at 6:23 AM Fujii Masao <masao.fujii@oss.nttdata.com>
wrote:
>
> Hi,
>
> Currently, file_fdw updates several columns in the pg_stat_progress_copy
view,
> like relid and bytes_processed, but it doesn't track tuples_processed or
> tuples_skipped. Monitoring these would be particularly useful when
handling
> large data sets via file_fdw, as it helps track the progress of scan.
>
> The attached patch updates file_fdw to add support for reporting
> the number of tuples processed and skipped (due to on_error = 'ignore')
> in the pg_stat_progress_copy view. What are your thoughts?
>

Awesome... no failures building and it worked as expected:

-[ RECORD 1 ]----+-----------
pid              | 102570
datid            | 16388
datname          | fabrizio
relid            | 16398
command          | COPY FROM
type             | FILE
bytes_processed  | 1359675392
bytes_total      | 2921579128
tuples_processed | 18605306
tuples_excluded  | 0
tuples_skipped   | 0

Regards,

--
Fabrízio de Royes Mello