Re: make installcheck-world in a clean environment

07.05.2018 20:07, Tom Lane wrote:
> Robert Haas <robertmhaas@gmail.com> writes:
>> After thinking about this some more, I think the question here is
>> definitional.  A first attempt at defining 'make installcheck' is to
>> say that it runs the tests from the build tree against the running
>> server.  Certainly, we intend to use the SQL files that are present in
>> the build tree, not the ones that were present in the build tree where
>> the running server was built.  But what about client programs that we
>> use to connect to the server?  You're suggesting that we use the
>> pre-installed ones, but that is actually pretty problematic because
>> the ones we see as installed might correspond neither to the contents
>> of the build tree nor to the running server.
If the contents of the source tree doesn't correspond to the running 
server, then I'm afraid, we can't installcheck it for sure.
I think it's supposed that we use for installcheck exactly the same 
source files that was used to build the server.
Regarding clients program, if we will not use/check it while 
installchecking, then by what means they can be tested when installed?
I think, the pgsql-packagers would like to check whether the whole 
installation of PostgreSQL is working, not only the server.
For me, very realistic and most useful scenario of installcheck is:
install postgresql-x.rpm
install postgresql-x.src.rpm
./configure --prefix=$target_path --enable-tap-tests, etc...
make installcheck(-world)
>>    Conceivably we could end
>> up having a mix of assets from three different places: (1) the running
>> server, (2) the build tree, (3) whatever is in our path at the moment.
>> That seems very confusing.  So now I think it's probably right to
>> define 'make installcheck' as using the assets from the build tree to
>> test the running server.  Under that definition, we're missing some
>> dependencies, but USE_INSTALLED_ASSETS isn't a thing we need.
I see another scenario, that was discussed a month ago - remote (or 
server-only) installcheck.
( 
https://www.postgresql.org/message-id/flat/CAM0nTJ6iorX_tmg5MX0mQU3z3w%3D9wk%2BpGK8zrvn7DNWqnyv%2BsQ%40mail.gmail.com 
)
It can be useful too and if it will be supported, then 
USE_INSTALLED_ASSETS usage should be extended to psql, etc.
> Nah, I disagree with this.  To me, the purpose of "make installcheck"
> is to verify the correctness of an installation, which I take to include
> the client programs as well as the server.  I think that "make
> installcheck" ought to use the installed version of any file that we
> actually install, and go to the build tree only for things we don't
> install (e.g. SQL test scripts).
Yes, that's the proposed patches intended for.  I didn't encounter any 
problems with it during internal testing with Linux and mingw-builds.
> If the user has screwed up his PATH or other environmental aspects so that
> what he's testing isn't a single installation, that's his error, not
> something that "make installcheck" ought to work around.  Indeed, maybe
> such aspects of his setup are intentional, and second-guessing them would
> completely defeat his purpose.  In any case, if you want to test the
> build-tree assets, that's what "make check" is for.
Even modified configs could lead to test failures (for example, 
lc_messages can break server log checking), so the installcheck should 
be performed against some clean and determinated installation anyway.

Best regards,
------
Alexander Lakhin
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company