Re: Improvements and additions to COPY progress reporting

Matthias van de Meent <boekewurm+postgres@gmail.com>

From: Matthias van de Meent <boekewurm+postgres@gmail.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: Matthias van de Meent <boekewurm+postgres@gmail.com>, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>, Justin Pryzby <pryzby@telsasoft.com>, Tomas Vondra <tomas.vondra@enterprisedb.com>, Josef Šimánek <josef.simanek@gmail.com>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Date: 2021-03-15T11:43:40Z
Lists: pgsql-hackers
On Mon, 15 Mar 2021 at 05:53, Michael Paquier <michael@paquier.xyz> wrote:
>
> On Wed, Mar 10, 2021 at 09:35:10AM +0100, Matthias van de Meent wrote:
> > There are examples in which pg_stat_progress_* -views report
> > inaccurate data. I think it is fairly reasonable to at least validate
> > some part of the progress reporting, as it is one of the few methods
> > for administrators to look at the state of currently running
> > administrative tasks, and as such, this user-visible api should be
> > validated.
>
> Looking closer at 0002, the size numbers are actually incorrect on
> Windows for the second query.  The CRLFs at the end of each line of
> emp.data add three bytes to the report of COPY FROM, so this finishes
> with 82 bytes for bytes_total and bytes_processed instead of 79.

Hmm, does CFBot not run checkout on windows with crlf line endings? I
had expected it to do as such.

> Let's make this useful but simpler here, so I propose to check that
> the counters are higher than zero instead of an exact number.  Let's
> also add the relation name relid::regclass while on it.

+1, I hadn't thought of casting relid to its regclass to get a stable
identifier.

> The tests introduced are rather limited, but you are right that
> something is better than nothing here, and I have slightly updated
> what the tests sent previously as per the attached.  What do you
> think?

That seems great, thanks for picking this up.


With regards,

Matthias van de Meent



Commits

  1. Add some basic tests for progress reporting of COPY

  2. Add support for more progress reporting in COPY

  3. doc: Add backlinks to progress reporting documentation