stats_temp_directory conflicts

On Sun, Dec 30, 2018 at 10:32:31AM -0500, Andrew Dunstan wrote:
> On 12/30/18 12:53 AM, Noah Misch wrote:
> >  stats_temp_directory = '/cygdrive/w/lorikeet/HEAD'

> > 2. stats_temp_directory is incompatible with TAP suites that start more than
> >    one node simultaneously.

> > It's already a problem for "make -j check-world".

> lorikeet is putting the stats_temp directory on a ramdisk. This is worth
> testing in any case, but in lorikeet's case was done to help speed up
> the tests. When I had a Raspberry Pi instance I did something similar,
> for the same reason.

I, too, value the ability to override stats_temp_directory for test runs.  (I
get stats.sql failures at high load, even on a high-performance machine.
Using stats_temp_directory may fix that.)

> The obvious quick fix would be to have PostgresNode.pm set this to the
> default after inserting the TEMP_CONFIG file.

True.

> There are a couple of problems here that bear further consideration.
> First, that the stats_temp_directory has to exist, and second that there
> is no convenient way to make it unique. It would be nice if a) the
> directory could be created if it didn't exist and b) some place-holder
> in the name could be replaced by a unique identifier such as the node
> id. If there is interest I'll work on these. One problem I foresee is
> that it might lead to a plethora of stats temp directories being left
> around. Still thinking about how we should deal with that. In the
> buildfarm client I'd be tempted to create a directory to hold all the
> run's stats_temp_directories and then clean it up at the end of the run.

I'm thinking the server should manage this; during startup, create
$stats_temp_directory/PostgreSQL.$postmaster_pid and store each stats file
therein.  Just before creating that directory, scan $stats_temp_directory and
delete subdirectories that no longer correspond to live PIDs.  Subdirectories
would not build up over time, even if one deletes a test data directory while
its subdirectory of stats_temp_directory still exists.  For non-test
applications, this makes stats_temp_directory safer to use.  Today, we don't
detect two clusters using the same stats_temp_directory.  We don't even
document that it's unacceptable.  This makes it acceptable.

Thanks,
nm