Horton

Thread

Commits

  1. Doc: improve explanation of how to use our code coverage infrastructure.

  1. Question about make coverage-html

    Peter Smith <smithpb2250@gmail.com> — 2020-10-27T08:09:30Z

    Hi hackers.

The example of test coverage in the documentation [1] works as advertised.

But I wanted to generate test coverage results only of some TAP tests
in src/test/subscription.

The documentation [1] also says "The make commands also work in
subdirectories." so I tried running them all in that folder.

However, when I run "make coverage-html" in that subdirectory
src/test/subscription it does not work:

=====
[postgres@CentOS7-x64 subscription]$ make coverage-html
/usr/local/bin/lcov --gcov-tool /usr/bin/gcov -q --no-external -c -i
-d . -d . -o lcov_base.info
geninfo: WARNING: no .gcno files found in . - skipping!
geninfo: WARNING: no .gcno files found in . - skipping!
/usr/local/bin/lcov --gcov-tool /usr/bin/gcov -q --no-external -c -d .
-d . -o lcov_test.info
geninfo: WARNING: no .gcda files found in . - skipping!
geninfo: WARNING: no .gcda files found in . - skipping!
rm -rf coverage
/usr/local/bin/genhtml -q --legend -o coverage --title='PostgreSQL
14devel' --num-spaces=4 --prefix='/home/postgres/oss_postgres_2PC'
lcov_base.info lcov_test.info
genhtml: ERROR: no valid records found in tracefile lcov_base.info
make: *** [coverage-html-stamp] Error 255
[postgres@CentOS7-x64 subscription]$
=====

OTOH, running the "make coverage-html" at the top folder after running
my TAP tests does give the desired coverage results.

~

QUESTION:

Was that documentation [1] just being misleading by saying it can work
in the subdirectories?
e.g. Are you only supposed to run "make coverage-html" from the top folder?

Or is it supposed to work but I did something wrong?

--

[1] https://www.postgresql.org/docs/13/regress-coverage.html

Kind Regards.
Peter Smith
Fujitsu Australia.

  2. Re: Question about make coverage-html

    Heikki Linnakangas <hlinnaka@iki.fi> — 2020-10-27T09:17:19Z

    On 27/10/2020 10:09, Peter Smith wrote:
> Hi hackers.
> 
> The example of test coverage in the documentation [1] works as advertised.
> 
> But I wanted to generate test coverage results only of some TAP tests
> in src/test/subscription.
> 
> The documentation [1] also says "The make commands also work in
> subdirectories." so I tried running them all in that folder.
> 
> However, when I run "make coverage-html" in that subdirectory
> src/test/subscription it does not work:
> 
> =====
> [postgres@CentOS7-x64 subscription]$ make coverage-html
> /usr/local/bin/lcov --gcov-tool /usr/bin/gcov -q --no-external -c -i
> -d . -d . -o lcov_base.info
> geninfo: WARNING: no .gcno files found in . - skipping!
> geninfo: WARNING: no .gcno files found in . - skipping!
> /usr/local/bin/lcov --gcov-tool /usr/bin/gcov -q --no-external -c -d .
> -d . -o lcov_test.info
> geninfo: WARNING: no .gcda files found in . - skipping!
> geninfo: WARNING: no .gcda files found in . - skipping!
> rm -rf coverage
> /usr/local/bin/genhtml -q --legend -o coverage --title='PostgreSQL
> 14devel' --num-spaces=4 --prefix='/home/postgres/oss_postgres_2PC'
> lcov_base.info lcov_test.info
> genhtml: ERROR: no valid records found in tracefile lcov_base.info
> make: *** [coverage-html-stamp] Error 255
> [postgres@CentOS7-x64 subscription]$
> =====
> 
> OTOH, running the "make coverage-html" at the top folder after running
> my TAP tests does give the desired coverage results.
> 
> ~
> 
> QUESTION:
> 
> Was that documentation [1] just being misleading by saying it can work
> in the subdirectories?
> e.g. Are you only supposed to run "make coverage-html" from the top folder?
> 
> Or is it supposed to work but I did something wrong?

Running "make coverage-html" in src/test/subscription doesn't work, 
because there is no C code in that directory.

Creating a coverage report is a two-step process. First, you run the 
test you're interested in, with "make check" or similar. Then you create 
a report for the source files you're interested in, with "make 
coverage-html". You can run these commands in different subdirectories.

In this case, you want to do "cd src/test/subscription; make check", to 
run those TAP tests, and then run "make coverage-html" from the top 
folder. Or if you wanted to create coverage report that covers only 
replication-related source code, for example, you could run it in the 
src/backend/replication directory ("cd src/backend/replication; make 
coverage-html").

- Heikki

  3. Re: Question about make coverage-html

    Tom Lane <tgl@sss.pgh.pa.us> — 2020-10-27T14:19:34Z

    Heikki Linnakangas <hlinnaka@iki.fi> writes:
> On 27/10/2020 10:09, Peter Smith wrote:
>> The documentation [1] also says "The make commands also work in
>> subdirectories." so I tried running them all in that folder.
>> However, when I run "make coverage-html" in that subdirectory
>> src/test/subscription it does not work:

> Creating a coverage report is a two-step process. First, you run the 
> test you're interested in, with "make check" or similar. Then you create 
> a report for the source files you're interested in, with "make 
> coverage-html". You can run these commands in different subdirectories.

> In this case, you want to do "cd src/test/subscription; make check", to 
> run those TAP tests, and then run "make coverage-html" from the top 
> folder. Or if you wanted to create coverage report that covers only 
> replication-related source code, for example, you could run it in the 
> src/backend/replication directory ("cd src/backend/replication; make 
> coverage-html").

I agree with the OP that the documentation is a bit vague here.
I think (maybe I'm wrong) that it's clear enough that you can run
whichever test case(s) you want, but this behavior of generating a
partial coverage report is less clear.  Maybe instead of

	The "make" commands also work in subdirectories.

we could say

	You can run the "make coverage-html" command in a subdirectory
	if you want a coverage report for only a portion of the code tree.

			regards, tom lane

  4. Re: Question about make coverage-html

    Peter Smith <smithpb2250@gmail.com> — 2020-10-27T23:04:20Z

    > > Creating a coverage report is a two-step process. First, you run the
> > test you're interested in, with "make check" or similar. Then you create
> > a report for the source files you're interested in, with "make
> > coverage-html". You can run these commands in different subdirectories.
>
> > In this case, you want to do "cd src/test/subscription; make check", to
> > run those TAP tests, and then run "make coverage-html" from the top
> > folder. Or if you wanted to create coverage report that covers only
> > replication-related source code, for example, you could run it in the
> > src/backend/replication directory ("cd src/backend/replication; make
> > coverage-html").
>
> I agree with the OP that the documentation is a bit vague here.
> I think (maybe I'm wrong) that it's clear enough that you can run
> whichever test case(s) you want, but this behavior of generating a
> partial coverage report is less clear.  Maybe instead of
>
>         The "make" commands also work in subdirectories.
>
> we could say
>
>         You can run the "make coverage-html" command in a subdirectory
>         if you want a coverage report for only a portion of the code tree.

Thank you for the clarifications and the updated documentation.

Kind Regards,
Peter Smith
Fujitsu Australia