Re: pgbench logging broken by time logic changes

Fabien COELHO <coelho@cri.ensmp.fr>

From: Fabien COELHO <coelho@cri.ensmp.fr>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: Michael Paquier <michael@paquier.xyz>, Andrew Dunstan <andrew@dunslane.net>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Gregory Smith <gregsmithpgsql@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>, david.christensen@crunchydata.com
Date: 2021-07-10T08:25:22Z
Lists: pgsql-hackers
>> Works for me: patch applies, global and local check ok. I'm fine with it.
>
> I hoped we were done here but I realised that your check for 1-3 log
> lines will not survive the harsh environment of the build farm.
> Adding sleep(2) before the final doLog() confirms that.  I had two
> ideas:
>
> 1.  Give up and expect 1-180 lines.  (180s is the current timeout
> tolerance used elsewhere to deal with
> swamped/valgrind/swapping/microsd computers, after a few rounds of
> inflation, so if you need an arbitrary large number to avoid buildfarm
> measles that's my suggestion....)
> 2.  Change doLog() to give up after end_time.  But then I think you'd
> need to make it 0-3 :-(
>
> I think the logging could be tightened up to work the way you expected
> in future work, though.  Perhaps we could change all logging to work
> with transaction start time instead of log-writing time, which doLog()
> should receive.  If you never start a transaction after end_time, then
> there can never be an aggregate that begins after that, and the whole
> thing becomes more deterministic.  That kind of alignment of aggregate
> timing with whole-run timing could also get rid of those partial
> aggregates completely.  But that's an idea for 15.
>
> So I think we should do 1 for now.  Objections or better ideas?

At least, we now that it is too much.

What about moving the test as is in the TODO section with a comment, next 
to the other one, for now?

I hesitated to suggest that before for the above risks, but I was very 
naively optimistic that it may pass because the test is not that too 
demanding.

Someone suggested to have a "real-time" configure switch to enable/disable 
time-sensitive tests.

-- 
Fabien.



Commits

  1. Fix pgbench timestamp bugs.

  2. Fix pattern matching logic for logs in TAP tests of pgbench

  3. pgbench: Improve time logic.

  4. pgbench: Synchronize client threads.

  5. pgbench: refactor handling of stats tracking