Re: pgbench logging broken by time logic changes

Hello Thomas,

> I committed the code change without the new TAP tests, because I
> didn't want to leave the open item hanging any longer.

Ok. Good.

>  As for the test, ... [...]

Argh, so there are no tests that would have caught the regressions:-(

> ... I know it can fail, and your v18 didn't fix that, because...
>
> +check_pgbench_logs($bdir, '001_pgbench_log_1', $nthreads, 1, 3,
>                                              ... this range can be exceeded.

Indeed. I meant to move that one in the TODO section as well, not just the 
previous call, so that all time-sensitive tests are fully ignored but 
reported, which would be enough for me.

> I suspect the number of aggregates could be made deterministic, as I
> described in an earlier message.  What do you think about doing
> something like that first for the next release, before trying to add
> assertions about the number of aggregates?

I think that last time I did something to get more deterministic results 
in pgbench, which involved a few lines of hocus-pocus in pgbench, the 
patch got rejected:-)

An "ignored" tests looked like a good compromise to check how things are 
going in the farm and to be able to check for more non regressions when 
developing pgbench, without introducing behavioral changes.

> I'm with you on the importance of testing, but it seems better to start 
> by making the thing more testable.

I'm used to my test patches being rejected, including modifying pgbench 
behavior to make it more testable. Am I mad enough to retry? Maybe, maybe 
not.

Attached the fully "ignored" version of the time features test as a patch.

-- 
Fabien.