Re: Need a builtin way to run all tests faster manner

On 2017-03-11 11:48:31 -0800, Andres Freund wrote:
> On 2017-03-11 12:05:23 -0500, Tom Lane wrote:
> > I wrote:
> > > I believe the core problem is that contrib/test_decoding's regresscheck
> > > and isolationcheck targets each want to use ./tmp_check as their
> > > --temp-instance.  make has no reason to believe it shouldn't run those
> > > two sub-jobs in parallel, but when it does, we get two postmasters trying
> > > to share the same directory.  This looks reasonably straightforward to
> > > solve, but I'm not entirely familiar with the code here, and am not
> > > sure what is the least ugly way to fix it.
> > 
> > Enlarging on that: if I cd into contrib/test_decoding and do
> > "make check -j4" or so, it reliably fails.
> 
> Yep, can reproduce here as well. Interesting that, with -j16, I could
> survive several dozen runs from the toplevel locally.
> 
> 
> > This is a localized patch that only fixes things for
> > contrib/test_decoding; what I'm wondering is if it would be better to
> > establish a more widespread convention that
> > $(pg_isolation_regress_check) should use a different --temp-instance
> > directory than $(pg_regress_check) does.
> 
> I think that'd be a good plan.  We probably should also keep --outputdir
> seperate (which test_decoding/Makefile does, but
> pg_isolation_regress_check doesn't)?

Here's a patch doing that (based on yours).  I'd be kind of inclined to
set --outputdir for !isolation tests too; possibly even move tmp_check
below output_iso/ output_regress/ or such - but that seems like it
potentially could cause some disagreement...

- Andres