Horton

Re: reducing isolation tests runtime

Alvaro Herrera <alvherre@alvh.no-ip.org>

View thread View on postgr.es
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Thomas Munro <thomas.munro@enterprisedb.com>, Andres Freund <andres@anarazel.de>, pgsql-hackers@postgresql.org
Date: 2018-01-25T14:37:26Z
Lists: pgsql-hackers 
Tom Lane wrote:
> Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
> > On the subject of test total time, we could paralelize isolation tests.
> > Right now "make check" in src/test/isolation takes 1:16 on my machine.
> > Test "timeouts" takes full 40s of that, with nothing running in parallel
> > -- the machine is completely idle.
> 
> BTW, one small issue there is that the reason the timeouts test is so
> slow is that we have to use multi-second timeouts to be sure slower
> buildfarm critters (eg valgrind animals) will get the expected results.
> So I'm worried that if the machine isn't otherwise idle, we will get
> random failures.

I think we could solve this by putting in the same parallel group only
slow tests that mostly sleeps, ie. nothing that would monopolize CPU for
long enough to cause a problem.  Concretely:

test: timeouts tuplelock-update deadlock-hard deadlock-soft-2

all of these tests have lots of sleeps and don't go through a lot of
data.  (Compared to the previous patch, I removed alter-table-1, which
uses a thousand tuples, and multiple-row-versions, which uses 100k; also
removed receipt-report which uses a large number of permutations.)

Timings:
	timeouts		40.3s
	tuplelock-update	10.5s
	deadlock-hard		10.9s
	deadlock-soft-2		5.4s

alter-table-1 takes 1.5s, receipt-report 1.2s and there's nothing else
that takes above 1s, so I think this is good enough -- we can still have
the whole thing run in ~45 seconds without the hazard you describe.

-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Commits

  1. Update obsolete sentence in README.parallel.

  2. Rewrite ConditionVariableBroadcast() to avoid live-lock.

  3. Tweak parallel hash join test case in hopes of improving stability.

  4. Rename pg_rewind's copy_file_range() to avoid conflict with new linux syscall.

  5. Fix some minor errors in new PHJ code.

  6. Fix EXPLAIN ANALYZE output for Parallel Hash.

  7. Fix rare assertion failure in parallel hash join.

  8. Cancel CV sleep during subtransaction abort.

  9. Add parallel-aware hash joins.

  10. Fix EXPLAIN ANALYZE of hash join when the leader doesn't participate.

  11. Add some regression tests that exercise hash join code.