initdb caching during tests

Andres Freund <andres@anarazel.de>

From: Andres Freund <andres@anarazel.de>
To: Tom Lane <tgl@sss.pgh.pa.us>, Thomas Munro <thomas.munro@gmail.com>
Cc: Robert Haas <robertmhaas@gmail.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2023-08-05T19:56:56Z
Lists: pgsql-hackers

Attachments

Hi,

We have some issues with CI on macos and windows being too expensive (more on
that soon in a separate email), which reminded me of this thread (with
original title: [1])

I've attached a somewhat cleaned up version of the patch to cache initdb
across runs.  The results are still fairly impressive in my opinion.


One thing I do not like, but don't have a good idea for how to improve, is
that there's a bunch of duplicated logic in pg_regress.c and Cluster.pm. I've
tried to move that into initdb.c itself, but that ends up pretty ugly, because
we need to be a lot more careful about checking whether options are compatible
etc. I've also thought about just putting this into a separate perl script,
but right now we still allow basic regression tests without perl being
available.  So I concluded that for now just having the copies is the best
answer.


Times for running all tests under meson, on my workstation (20 cores / 40
threads):

cassert build -O2:

Before:
real	0m44.638s
user	7m58.780s
sys	2m48.773s

After:
real	0m38.938s
user	2m37.615s
sys	2m0.570s


cassert build -O0:

Before:
real	1m11.290s
user	13m9.817s
sys	2m54.946s

After:
real	1m2.959s
user	3m5.835s
sys	1m59.887s


non-cassert build:

Before:
real	0m34.579s
user	5m30.418s
sys	2m40.507s

After:
real	0m27.710s
user	2m20.644s
sys	1m55.770s


On CI this reduces the test times substantially:
Freebsd                           8:51 -> 5:35
Debian w/ asan, autoconf          6:43 -> 4:55
Debian w/ alignmentsan, ubsan     4:02 -> 2:33
macos                             5:07 -> 4:29
windows                          10:21 -> 9:49

This is ignoring a bit of run-to-run variance, but the trend is obvious enough
that it's not worth worrying about that.

Greetings,

Andres Freund

[1] https://postgr.es/m/20220120021859.3zpsfqn4z7ob7afz%40alap3.anarazel.de

Commits

  1. Don't clean initdb files on template creation failure

  2. Avoid non-POSIX cp flags

  3. Use "template" data directory in tests

  4. tests: Consistently use pg_basebackup -cfast --no-sync to accelerate tests.