Horton
Menu

Thread

  1. Having problems generating a code coverage report

    Aleksander Alekseev <aleksander@timescale.com> — 2024-10-30T21:21:10Z

    Hi hackers,

Recently I tried to build a code coverage report according to the
documentation [1]. When using the following command:

```
time sh -c 'git clean -dfx && meson setup --buildtype debug
-DPG_TEST_EXTRA="kerberos ldap ssl" -Db_coverage=true -Dldap=disabled
-Dssl=openssl -Dcassert=true -Dtap_tests=enabled
-Dprefix=/home/eax/pginstall build && ninja -C build &&
PG_TEST_EXTRA=1 meson test -C build && ninja -C build coverage-html'
```

... I get:

```
geninfo: ERROR: Unexpected negative count '-3' for
/home/eax/projects/c/postgresql/src/port/snprintf.c:532.
    Perhaps you need to compile with '-fprofile-update=atomic
    (use "geninfo --ignore-errors negative ..." to bypass this error)
Traceback (most recent call last):
  File "/usr/bin/meson", line 40, in <module>
    sys.exit(mesonmain.main())
             ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/mesonbuild/mesonmain.py", line
294, in main
    return run(sys.argv[1:], launcher)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/mesonbuild/mesonmain.py", line
282, in run
    return run_script_command(args[1], args[2:])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/mesonbuild/mesonmain.py", line
223, in run_script_command
    return module.run(script_args)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/mesonbuild/scripts/coverage.py",
line 206, in run
    return coverage(options.outputs, options.source_root,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/mesonbuild/scripts/coverage.py",
line 123, in coverage
    subprocess.check_call([lcov_exe,
  File "/usr/lib/python3.12/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['lcov', '--directory',
'/home/eax/projects/c/postgresql/build', '--capture', '--output-file',
'/home/eax/projects/c/postgresql/build/meson-logs/coverage.info.run',
'--no-checksum', '--rc', 'branch_coverage=1']' returned non-zero exit
status 1.
ninja: build stopped: subcommand failed.
```

If I add -fprofile-update=atomic as suggested:

```
time CFLAGS="-fprofile-update=atomic"
CXXFLAGS="-fprofile-update=atomic" sh -c 'git clean -dfx && meson
setup --buildtype debug -DPG_TEST_EXTRA="kerberos ldap ssl"
-Db_coverage=true -Dldap=disabled -Dssl=openssl -Dcassert=true
-Dtap_tests=enabled -Dprefix=/home/eax/pginstall build && ninja -C
build && PG_TEST_EXTRA=1 meson test -C build && ninja -C build
coverage-html'
```

... I get:

```
genhtml: ERROR: duplicate merge record src/include/catalog
Traceback (most recent call last):
  File "/usr/bin/meson", line 40, in <module>
    sys.exit(mesonmain.main())
             ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/mesonbuild/mesonmain.py", line
294, in main
    return run(sys.argv[1:], launcher)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/mesonbuild/mesonmain.py", line
282, in run
    return run_script_command(args[1], args[2:])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/mesonbuild/mesonmain.py", line
223, in run_script_command
    return module.run(script_args)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/mesonbuild/scripts/coverage.py",
line 206, in run
    return coverage(options.outputs, options.source_root,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/mesonbuild/scripts/coverage.py",
line 150, in coverage
    subprocess.check_call([genhtml_exe,
  File "/usr/lib/python3.12/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['genhtml', '--prefix',
'/home/eax/projects/c/postgresql/build', '--prefix',
'/home/eax/projects/c/postgresql', '--output-directory',
'/home/eax/projects/c/postgresql/build/meson-logs/coveragereport',
'--title', 'Code coverage', '--legend', '--show-details',
'--branch-coverage',
'/home/eax/projects/c/postgresql/build/meson-logs/coverage.info']'
returned non-zero exit status 1.
ninja: build stopped: subcommand failed.
```

I'm using Xubuntu 24.04 LTS and my lcov version is:

```
$ lcov --version
lcov: LCOV version 2.0-1
```

I tried using Autotools with the same results. Pretty confident it
worked before. I'm wondering if anyone else experienced this lately
and/or knows a workaround.

[1]: https://www.postgresql.org/docs/current/regress-coverage.html

--
Best regards,
Aleksander Alekseev

  2. Re: Having problems generating a code coverage report

    Peter Geoghegan <pg@bowt.ie> — 2024-10-30T21:26:49Z

    On Wed, Oct 30, 2024 at 5:21 PM Aleksander Alekseev
<aleksander@timescale.com> wrote:
> I'm using Xubuntu 24.04 LTS and my lcov version is:
>
> ```
> $ lcov --version
> lcov: LCOV version 2.0-1
> ```

I use Debian unstable for most of my day to day work. Apparently
Debian unstable has exactly the same version of lcov as Ubuntu 24.04.

I've also been unable to generate coverage reports for some time (at
least on Debian, with LCOV version 2.0-1).

--
Peter Geoghegan

  3. Re: Having problems generating a code coverage report

    Michael Paquier <michael@paquier.xyz> — 2024-10-31T00:13:40Z

    On Wed, Oct 30, 2024 at 05:26:49PM -0400, Peter Geoghegan wrote:
> I use Debian unstable for most of my day to day work. Apparently
> Debian unstable has exactly the same version of lcov as Ubuntu 24.04.
>
> I've also been unable to generate coverage reports for some time (at
> least on Debian, with LCOV version 2.0-1).

So do I, for both the debian SID and the lcov parts.
--
Michael

  4. Re: Having problems generating a code coverage report

    Aleksander Alekseev <aleksander@timescale.com> — 2024-10-31T14:02:23Z

    Hi,

> On Wed, Oct 30, 2024 at 05:26:49PM -0400, Peter Geoghegan wrote:
> > I use Debian unstable for most of my day to day work. Apparently
> > Debian unstable has exactly the same version of lcov as Ubuntu 24.04.
> >
> > I've also been unable to generate coverage reports for some time (at
> > least on Debian, with LCOV version 2.0-1).
>
> So do I, for both the debian SID and the lcov parts.

I downgraded to lcov 1.16 [1] and it helped. This is merely a
workaround of course, not a long-time solution.

[1]: https://github.com/linux-test-project/lcov/releases/tag/v1.16

-- 
Best regards,
Aleksander Alekseev

  5. Re: Having problems generating a code coverage report

    jian he <jian.universality@gmail.com> — 2024-10-31T15:36:21Z

    On Thu, Oct 31, 2024 at 10:02 PM Aleksander Alekseev
<aleksander@timescale.com> wrote:
>
> Hi,
>
> > On Wed, Oct 30, 2024 at 05:26:49PM -0400, Peter Geoghegan wrote:
> > > I use Debian unstable for most of my day to day work. Apparently
> > > Debian unstable has exactly the same version of lcov as Ubuntu 24.04.
> > >
> > > I've also been unable to generate coverage reports for some time (at
> > > least on Debian, with LCOV version 2.0-1).
> >
> > So do I, for both the debian SID and the lcov parts.
>
> I downgraded to lcov 1.16 [1] and it helped. This is merely a
> workaround of course, not a long-time solution.
>
> [1]: https://github.com/linux-test-project/lcov/releases/tag/v1.16
>
> --

my ubuntu meson version: 0.61.2, which also fails.
with similar errors you've posted.


from these two posts,
https://github.com/mesonbuild/meson/pull/12345
https://github.com/mesonbuild/meson/issues/11995

Maybe upgrading meson can solve this problem.

  6. Re: Having problems generating a code coverage report

    Aleksander Alekseev <aleksander@timescale.com> — 2024-10-31T16:37:43Z

    Hi Jian,

> > I downgraded to lcov 1.16 [1] and it helped. This is merely a
> > workaround of course, not a long-time solution.
> >
> > [1]: https://github.com/linux-test-project/lcov/releases/tag/v1.16
> >
> > --
>
> my ubuntu meson version: 0.61.2, which also fails.
> with similar errors you've posted.
>
>
> from these two posts,
> https://github.com/mesonbuild/meson/pull/12345
> https://github.com/mesonbuild/meson/issues/11995
>
> Maybe upgrading meson can solve this problem.

Thanks for the hint. I'm using Meson 1.3.2. Although it's not ancient
(Feb 2024) there is 1.6.0 available. I'll try upgrading and let you
know the results.

-- 
Best regards,
Aleksander Alekseev

  7. Re: Having problems generating a code coverage report

    Aleksander Alekseev <aleksander@timescale.com> — 2024-11-01T14:22:17Z

    Hi,

> Thanks for the hint. I'm using Meson 1.3.2. Although it's not ancient
> (Feb 2024) there is 1.6.0 available. I'll try upgrading and let you
> know the results.

I upgraded to Meson 1.6.0 and Lcov 2.0-1. Unfortunately it doesn't work:

```
genhtml: ERROR: no data for line:864, TLA:UNC,
file:/home/eax/projects/c/postgresql/src/bin/psql/psqlscanslash.l
    (use "genhtml --ignore-errors unmapped ..." to bypass this error)
Traceback (most recent call last):
  File "/home/eax/.venv/bin/meson", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/eax/.venv/lib/python3.12/site-packages/mesonbuild/mesonmain.py",
line 293, in main
    return run(sys.argv[1:], launcher)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eax/.venv/lib/python3.12/site-packages/mesonbuild/mesonmain.py",
line 281, in run
    return run_script_command(args[1], args[2:])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eax/.venv/lib/python3.12/site-packages/mesonbuild/mesonmain.py",
line 222, in run_script_command
    return module.run(script_args)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eax/.venv/lib/python3.12/site-packages/mesonbuild/scripts/coverage.py",
line 208, in run
    return coverage(options.outputs, options.source_root,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eax/.venv/lib/python3.12/site-packages/mesonbuild/scripts/coverage.py",
line 148, in coverage
    subprocess.check_call([genhtml_exe,
  File "/usr/lib/python3.12/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['genhtml', '--prefix',
'/home/eax/projects/c/postgresql/build', '--prefix',
'/home/eax/projects/c/postgresql', '--output-directory',
'/home/eax/projects/c/postgresql/build/meson-logs/coveragereport',
'--title', 'Code coverage', '--legend', '--show-details',
'--branch-coverage',
'/home/eax/projects/c/postgresql/build/meson-logs/coverage.info']'
returned non-zero exit status 1.
ninja: build stopped: subcommand failed.
```

If I add `--ignore-errors unmapped` as suggested and execute:

```
genhtml --ignore-errors unmapped --prefix
/home/eax/projects/c/postgresql/build --prefix
/home/eax/projects/c/postgresql --output-directory
/home/eax/projects/c/postgresql/build/meson-logs/coveragereport
--title 'Code coverage' --legend --show-details --branch-coverage
/home/eax/projects/c/postgresql/build/meson-logs/coverage.info
```

... I get:

```
...
Processing file src/pl/plpython/plpy_util.c
  lines=33 hit=22 functions=4 hit=4 branches=14 hit=5
Processing file src/include/catalog/index.h
  lines=10 hit=10 functions=2 hit=2
genhtml: ERROR: duplicate merge record src/include/catalog
```

I didn't investigate further.

-- 
Best regards,
Aleksander Alekseev

  8. Re: Having problems generating a code coverage report

    Aleksander Alekseev <aleksander@timescale.com> — 2024-11-01T15:42:37Z

    Hi everyone,

> I upgraded to Meson 1.6.0 and Lcov 2.0-1. Unfortunately it doesn't work:
> [...]
> I didn't investigate further.

Lcov 2.1 and 2.2beta don't work either.

-- 
Best regards,
Aleksander Alekseev

  9. Re: Having problems generating a code coverage report

    Peter Geoghegan <pg@bowt.ie> — 2025-01-15T18:48:03Z

    On Wed, Oct 30, 2024 at 5:26 PM Peter Geoghegan <pg@bowt.ie> wrote:
> I use Debian unstable for most of my day to day work. Apparently
> Debian unstable has exactly the same version of lcov as Ubuntu 24.04.
>
> I've also been unable to generate coverage reports for some time (at
> least on Debian, with LCOV version 2.0-1).

I found a temporary workaround. I'm now once again able to produce
html coverage reports on my Debian unstable workstation. Here's what I
did:

I found that a local installation of lcov 1.16 worked. I cloned the
lcov git repo, and checked out the tag 'v1.16'. I then installed this
older lcov version to a directory under my home directory.

It was also necessary to convince the build system to use my
known-good version of lcov. I had to use autoconf here, since (if I'm
not mistaken) only autoconf will accept specific instructions as to
which gcov, lcov, and genhtml to use (via environment variables). I
also used a relatively old version of GCC, taken from the "gcc-11"
package (not sure if that part was really necessary).

I adding something along these lines to the zsh function that runs
"configure" for me, to automate the process going forward:

  export CC="gcc-11"  # Provided by gcc-11 package
  export GCOV="gcov-11" # Provided by gcc-11 package
  export LCOV="/home/pg/.../lcov"  # Provided by local lcov 1.16 installation
  export GENHTML="/home/pg/.../genhtml"  # Provided by local lcov 1.16
installation

From here I ran the standard, documented procedure: I ran configure
(through the zsh function), built Postgres in the usual way, ran the
tests in the usual way, and finally ran "make coverage-html". The
final html report looks very similar to the one from
coverage.postgresql.org.

(There were some geninfo warnings about certain files not having any
coverage whatsoever, but I'm pretty sure that that's normal.)

-- 
Peter Geoghegan

  10. Re: Having problems generating a code coverage report

    Tom Lane <tgl@sss.pgh.pa.us> — 2025-01-15T21:44:10Z

    Peter Geoghegan <pg@bowt.ie> writes:
> On Wed, Oct 30, 2024 at 5:26 PM Peter Geoghegan <pg@bowt.ie> wrote:
>> I've also been unable to generate coverage reports for some time (at
>> least on Debian, with LCOV version 2.0-1).

> I found a temporary workaround. I'm now once again able to produce
> html coverage reports on my Debian unstable workstation.

I got around to poking at this on a shiny new Fedora 41 image,
with gcc-14.2.1 and lcov-2.0-4.  I see errors and warnings aplenty
there too, but after reading the suppression hints that appear in
the error messages I was able to make them all go away:

$ make -s coverage-html GENHTML_FLAGS="-q --legend --ignore-errors unmapped,unmapped"
Overall coverage rate:
  lines......: 60.7% (282865 of 466310 lines)
  functions......: 67.4% (16114 of 23895 functions)
$

The "-q --legend" part is our default value of GENHTML_FLAGS,
the new magic is the --ignore-errors bit.  As far as I can tell,
specifying "unmapped" once reduces that class of errors to
warnings and specifying it again silences them entirely.

With this, I got plausible-looking html output (I didn't vet it
in detail, but a couple of spot checks looked sane).

I'm curious whether a similar workaround will help with the
Debian toolchain.

			regards, tom lane

  11. Re: Having problems generating a code coverage report

    Peter Geoghegan <pg@bowt.ie> — 2025-01-16T00:51:39Z

    On Wed, Jan 15, 2025 at 4:44 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> With this, I got plausible-looking html output (I didn't vet it
> in detail, but a couple of spot checks looked sane).
>
> I'm curious whether a similar workaround will help with the
> Debian toolchain.

Looks like it will.

I was able to get the lcov git master branch's tip to produce a
Postgres coverage report (same compiler version and OS as before).
Unfortunately, I had to use an even grottier incantation to get this
to work:

make -s coverage-html GENHTML_FLAGS="-q --legend --ignore-errors
unmapped,unmapped,empty,empty,inconsistent,inconsistent,corrupt,corrupt,range,range"
LCOVFLAGS="--ignore-errors
empty,empty,negative,negative,inconsistent,inconsistent"

(As you can imagine, I came up with this through pure trial and error.)

The final report seems fine, though -- at least on a quick look. The
number of times that we actually relied on the suppressions seems to
have been fairly limited, in the end:

Message summary:
  11 ignore messages:
    empty: 1
    inconsistent: 3
    negative: 7
Overall coverage rate:
  source files: 1527
  lines.......: 78.7% (396815 of 504471 lines)
  functions...: 86.3% (23710 of 27474 functions)

Note also: I had to install gcov to /usr/local this time. I couldn't
get away with installing it to my home directory (something about Perl
not being able to see lcovutil.pm in @INC).

--
Peter Geoghegan

  12. Re: Having problems generating a code coverage report

    Michael Paquier <michael@paquier.xyz> — 2025-01-29T01:00:38Z

    On Wed, Jan 15, 2025 at 07:51:39PM -0500, Peter Geoghegan wrote:
> I was able to get the lcov git master branch's tip to produce a
> Postgres coverage report (same compiler version and OS as before).
> Unfortunately, I had to use an even grottier incantation to get this
> to work:
> 
> make -s coverage-html GENHTML_FLAGS="-q --legend --ignore-errors
> unmapped,unmapped,empty,empty,inconsistent,inconsistent,corrupt,corrupt,range,range"
> LCOVFLAGS="--ignore-errors
> empty,empty,negative,negative,inconsistent,inconsistent"
> 
> (As you can imagine, I came up with this through pure trial and error.)

Interesting.  These tricks do not seem to work here with a Debian GID
host, unfortunately.  I'll try to dig a bit more, this is getting very
annoying :(
--
Michael

  13. Re: Having problems generating a code coverage report

    Andres Freund <andres@anarazel.de> — 2026-01-21T18:57:58Z

    Hi,

On 2024-11-01 18:42:37 +0300, Aleksander Alekseev wrote:
> Hi everyone,
> 
> > I upgraded to Meson 1.6.0 and Lcov 2.0-1. Unfortunately it doesn't work:
> > [...]
> > I didn't investigate further.
> 
> Lcov 2.1 and 2.2beta don't work either.

I just encountered this. The problem seems to only happen when the build
directory is inside the source directory.

Not that that's a great solution, but just having the build dir be in a
separate directory fixed the issue for me.

Greetings,

Andres Freund

  14. Re: Having problems generating a code coverage report

    Michael Paquier <michael@paquier.xyz> — 2026-01-22T00:48:35Z

    On Wed, Jan 21, 2026 at 01:57:58PM -0500, Andres Freund wrote:
> I just encountered this. The problem seems to only happen when the build
> directory is inside the source directory.
> 
> Not that that's a great solution, but just having the build dir be in a
> separate directory fixed the issue for me.

Even with a VPATH build on Debien GID?  FWIW, I am unable to run a
coverage-html even in this case from the root of a build, finishing
with stuff like that:
/usr/bin/lcov --gcov-tool /usr/bin/gcov -q --no-external -c -i -d . -d
/home/user/git/postgres/ -o lcov_base.info
lcov: ERROR: (empty) no .gcno files found in /home/user/git/postgres
(use "lcov --ignore-errors empty ..." to bypass this error)

Passing "--ignore-errors empty" to LCOVFLAGS bypasses the first step
but genhtml complains, like:
genhtml: ERROR: (corrupt) unable to read trace file 'lcov_base.info':

meson is no better as far as I've tried, so I have pretty much given
up with coverage reports in this environment at this stage.  If there
is a magic recipe, I am not really aware of one..
--
Michael

  15. Re: Having problems generating a code coverage report

    Andres Freund <andres@anarazel.de> — 2026-01-22T01:59:45Z

    Hi,

On 2026-01-22 09:48:35 +0900, Michael Paquier wrote:
> On Wed, Jan 21, 2026 at 01:57:58PM -0500, Andres Freund wrote:
> > I just encountered this. The problem seems to only happen when the build
> > directory is inside the source directory.
> > 
> > Not that that's a great solution, but just having the build dir be in a
> > separate directory fixed the issue for me.
> 
> Even with a VPATH build on Debien GID?

Well, mine was a meson build.


> meson is no better as far as I've tried, so I have pretty much given
> up with coverage reports in this environment at this stage.  If there
> is a magic recipe, I am not really aware of one..

I have the following .lcovrc in my postgres source dir:

ignore_errors=inconsistent,gcov,range
check_data_consistency=0
stop_on_error=0
genhtml_hierarchical=1
genhtml_show_navigation=1
parallel=16
geninfo_gcov_tool=/usr/bin/gcov-15

(assuming you build with gcc 15)

With that meson coverage information works (i.e. ninja coverage-html) for both
lcov 2.0 and 2.4 as long as the build directory is outside of the source
directory. Both on Debian Sid and macos.

Greetings,

Andres Freund

  16. Re: Having problems generating a code coverage report

    Michael Paquier <michael@paquier.xyz> — 2026-01-22T05:35:40Z

    On Wed, Jan 21, 2026 at 08:59:45PM -0500, Andres Freund wrote:
> I have the following .lcovrc in my postgres source dir:
> 
> ignore_errors=inconsistent,gcov,range
> check_data_consistency=0
> stop_on_error=0
> genhtml_hierarchical=1
> genhtml_show_navigation=1
> parallel=16
> geninfo_gcov_tool=/usr/bin/gcov-15
> 
> (assuming you build with gcc 15)
> 
> With that meson coverage information works (i.e. ninja coverage-html) for both
> lcov 2.0 and 2.4 as long as the build directory is outside of the source
> directory. Both on Debian Sid and macos.

Thanks for sharing.  Yes, I have a matching gcc version at 15.2
currently.  I have been playing with this area of the configuration
for some time without finding the correct combination before giving
up, and these suggestions are actually helping for meson.  So now I
can get a report offline.  Yeah at least it's one thing working.

Hmm.  configure with and without VPATH still burp at me on HEAD.  If I
tweak the lcov commands in src/Makefile.global.in like in the attached 
patch then the non-VPATH build can work, where I remove the srcdir
part (aka reverting c3d9a66024a9).  For the VPATH build, this patch
does not work at all, it generates an incomplete report, ignoring
entirely contrib/, with its base index.html starting inside src/,
missing a bunch of subdirectories.  I can live with this custom patch
for now, just wondering if anybody else sees what I am seeing for a
few months now.

Thoughts?  At least two out of the three methods are working with the
attached instead of one out of three.  Not sure what to do with VPATH,
but well.
--
Michael

  17. Re: Having problems generating a code coverage report

    Andres Freund <andres@anarazel.de> — 2026-01-22T18:19:43Z

    Hi,

On 2026-01-22 14:35:40 +0900, Michael Paquier wrote:
> On Wed, Jan 21, 2026 at 08:59:45PM -0500, Andres Freund wrote:
> > With that meson coverage information works (i.e. ninja coverage-html) for both
> > lcov 2.0 and 2.4 as long as the build directory is outside of the source
> > directory. Both on Debian Sid and macos.
>
> Thanks for sharing.  Yes, I have a matching gcc version at 15.2
> currently.  I have been playing with this area of the configuration
> for some time without finding the correct combination before giving
> up, and these suggestions are actually helping for meson.  So now I
> can get a report offline.  Yeah at least it's one thing working.

Cool.


> Hmm.  configure with and without VPATH still burp at me on HEAD.  If I
> tweak the lcov commands in src/Makefile.global.in like in the attached
> patch then the non-VPATH build can work, where I remove the srcdir
> part (aka reverting c3d9a66024a9).  For the VPATH build, this patch
> does not work at all, it generates an incomplete report, ignoring
> entirely contrib/, with its base index.html starting inside src/,
> missing a bunch of subdirectories.  I can live with this custom patch
> for now, just wondering if anybody else sees what I am seeing for a
> few months now.

If I suppress the empty errors & warnings with --ignore-errors empty,empty the
current "-d . -d $(srcdir)" kinda works.  Unfortunately, in VPATH builds, the
directory structure is quite messed up. It seems to get a bit better if I add
-p $(srcdir) to the genhtml invocation, but a few generated files still show
up bogus.

The bogus build files unfortunately aren't trivial to fix, if I additionally
add a "-p $(pwd)" (i.e. add the build dir as a prefix), I get errors about
"duplicate merge record".   But it's not too bad to see that stupid path for
the generated files.


Greetings,

Andres Freund

  18. Re: Having problems generating a code coverage report

    Michael Paquier <michael@paquier.xyz> — 2026-01-22T22:11:11Z

    On Thu, Jan 22, 2026 at 01:19:43PM -0500, Andres Freund wrote:
> On 2026-01-22 14:35:40 +0900, Michael Paquier wrote:
> If I suppress the empty errors & warnings with --ignore-errors empty,empty the
> current "-d . -d $(srcdir)" kinda works.  Unfortunately, in VPATH builds, the
> directory structure is quite messed up. It seems to get a bit better if I add
> -p $(srcdir) to the genhtml invocation, but a few generated files still show
> up bogus.

With a bit more information to skip more error patterns, I actually
got an unpatched HEAD build able to work properly as well, with:
ignore_errors=inconsistent,gcov,range,empty,usage,usage,empty,empty,path

The VPATH build is indeed looking pretty broken here, with the
top-level including the contents of src/ and /home/, /home/ pointing
to contrib/.  I'll live with that, that's so much better than nothing
working locally.  Thanks for all the pointers, Andres.
--
Michael

  19. Re: Having problems generating a code coverage report

    Stefan Kaltenbrunner <stefan@kaltenbrunner.cc> — 2026-02-14T08:04:44Z

    On 1/29/25 02:00, Michael Paquier wrote:
> On Wed, Jan 15, 2025 at 07:51:39PM -0500, Peter Geoghegan wrote:
>> I was able to get the lcov git master branch's tip to produce a
>> Postgres coverage report (same compiler version and OS as before).
>> Unfortunately, I had to use an even grottier incantation to get this
>> to work:
>>
>> make -s coverage-html GENHTML_FLAGS="-q --legend --ignore-errors
>> unmapped,unmapped,empty,empty,inconsistent,inconsistent,corrupt,corrupt,range,range"
>> LCOVFLAGS="--ignore-errors
>> empty,empty,negative,negative,inconsistent,inconsistent"
>>
>> (As you can imagine, I came up with this through pure trial and error.)
> 
> Interesting.  These tricks do not seem to work here with a Debian GID
> host, unfortunately.  I'll try to dig a bit more, this is getting very
> annoying :(

hmm this is now also breaking coverage.postgresql.org (after an upgrade 
to the current debian trixie):


make: Leaving directory 
'/home/coverage/pgsrc/pgsql/src/interfaces/ecpg/test'
/usr/bin/lcov --gcov-tool /usr/bin/gcov -q --no-external -c -i -d . -d . 
-o lcov_base.info
lcov: WARNING: (deprecated) RC option 'lcov_branch_coverage' is 
deprecated.  Consider using 'branch_coverage'. instead. 
(Backward-compatible support will be removed in the future.)
         (use "lcov --ignore-errors deprecated,deprecated ..." to 
suppress this warning)
Message summary:
   1 error message:
     usage: 1
   1 warning message:
     deprecated: 1
lcov: ERROR: (usage) duplicate file ./src/fe_utils/astreamer_gzip.gcno 
in both . and .
         (use "lcov --ignore-errors usage ..." to bypass this error)
make: *** [src/Makefile.global:1064: lcov_base.info] Error 1


This seems to be basically the same issue that Andres patch in this 
thread tries to address by removing one of the -d - applying a similar 
hack locally seems to help a bit until we are running into one of the 
other reported issues:

lcov: WARNING: (inconsistent) 
/home/coverage/pgsrc/pgsql/contrib/bloom/blinsert.c:274: unexecuted 
block on non-branch line with non-zero hit count.  Use "geninfo --rc 
geninfo_unexecuted_blocks=1 to set count to zero.
	(use "lcov --ignore-errors inconsistent,inconsistent ..." to suppress 
this warning)
lcov: ERROR: (negative) Unexpected negative hit count '-5' for line 
/home/coverage/pgsrc/pgsql/src/port/snprintf.c:529" while capturing from 
./src/port/snprintf_shlib.gcda.
	(use "lcov --ignore-errors negative ..." to bypass this error)
Message summary:
   1 error message:
     negative: 1
   2 warning messages:
     deprecated: 1
     inconsistent: 1
make: *** [src/Makefile.global:1069: lcov_test.info] Error 1


While I certainly can try to locally hack around that further to get it 
build it feels wrong to post process/patch our own tree to get 
"official" coverage reporting...



Stefan

  20. Re: Having problems generating a code coverage report

    Michael Paquier <michael@paquier.xyz> — 2026-02-16T01:49:14Z

    On Sat, Feb 14, 2026 at 09:04:44AM +0100, Stefan Kaltenbrunner wrote:
> While I certainly can try to locally hack around that further to get it
> build it feels wrong to post process/patch our own tree to get "official"
> coverage reporting...

Are you using a VPATH build?  That was the only build option that was
causing me problems on Debian GID even with the tweaks proposed by
Andres in a .lcovrc.  Non-vpath under configure and meson worked
unpatched.  With vpath under configure, that worked but the
directories were in a weird state (contrib/ missing from the root
patch, $HOME included).  meson was much slower than the two others.

I have also tried a few more experiments on Fedora 34 lately, that
points to even newer versions of all these tools.  Things are even
more broken moving forward for meson, vpath and non-vpath.  This is
testing things with an unpatched PG tree, just mentioning that it
does not seem to get better as time goes..
--
Michael

  21. Re: Having problems generating a code coverage report

    Álvaro Herrera <alvherre@kurilemu.de> — 2026-02-17T17:10:46Z

    On 2026-Feb-16, Michael Paquier wrote:

> On Sat, Feb 14, 2026 at 09:04:44AM +0100, Stefan Kaltenbrunner wrote:
> > While I certainly can try to locally hack around that further to get it
> > build it feels wrong to post process/patch our own tree to get "official"
> > coverage reporting...
> 
> Are you using a VPATH build?

Nope, it's a straight in-tree build.  I made it work after a few more
failed tries with a call like

GENHTML_FLAGS="-q --legend --ignore-errors unmapped,corrupt,inconsistent,range"
LCOVFLAGS="-q --legend --ignore-errors usage"
make coverage-html

... and we have a nice (??) trace of what errors are being ignored.  I
think this is absolutely bonkers, but who knows when or if lcov and its
tools are going to be fixed in a way that allows our tree to be
processed correctly.

However, it appears that there are problems with the CSS, because the
lines are not colored anymore.  I have no idea how to get this to work,
but you can see the result running lcov 2.0 in
https://coverage.postgresql.org/

-- 
Álvaro Herrera        Breisgau, Deutschland  —  https://www.EnterpriseDB.com/
"You're _really_ hosed if the person doing the hiring doesn't understand
relational systems: you end up with a whole raft of programmers, none of
whom has had a Date with the clue stick."              (Andrew Sullivan)
https://postgr.es/m/20050809113420.GD2768@phlogiston.dyndns.org

  22. Re: Having problems generating a code coverage report

    Álvaro Herrera <alvherre@kurilemu.de> — 2026-02-17T17:28:24Z

    On 2026-Feb-17, Álvaro Herrera wrote:

> Nope, it's a straight in-tree build.  I made it work after a few more
> failed tries with a call like
> 
> GENHTML_FLAGS="-q --legend --ignore-errors unmapped,corrupt,inconsistent,range"
> LCOVFLAGS="-q --legend --ignore-errors usage"
> make coverage-html

Correction.  These variables are not taken into account if given as
environment variables, but they are if given as make arguments.
Moreover, --legend makes lcov die.  So, apparently this one works:

make coverage-html LCOVFLAGS="-q --ignore-errors usage" GENHTML_FLAGS="-q --legend --ignore-errors unmapped,corrupt,inconsistent,range"

I also removed a file called .lcovrc that only had this line
  lcov_branch_coverage = 0
though that doesn't appear to have automatically added branch coverage
to the report.  However, I'll wait until the next run to see if anything
changes.

The CSS problem appears to have been a caching glitch, and now it works,
so I guess the cached file must have expired.  It looks Christmas-treeish
again now.

-- 
Álvaro Herrera               48°01'N 7°57'E  —  https://www.EnterpriseDB.com/
"No renuncies a nada. No te aferres a nada."

  23. Re: Having problems generating a code coverage report

    Michael Paquier <michael@paquier.xyz> — 2026-02-18T06:21:21Z

    On Tue, Feb 17, 2026 at 06:28:24PM +0100, Alvaro Herrera wrote:
> Correction.  These variables are not taken into account if given as
> environment variables, but they are if given as make arguments.
> Moreover, --legend makes lcov die.  So, apparently this one works:
> 
> make coverage-html LCOVFLAGS="-q --ignore-errors usage" GENHTML_FLAGS="-q --legend --ignore-errors unmapped,corrupt,inconsistent,range"
> 
> I also removed a file called .lcovrc that only had this line
>   lcov_branch_coverage = 0
> though that doesn't appear to have automatically added branch coverage
> to the report.  However, I'll wait until the next run to see if anything
> changes.

Based on your ideas, I have been able to get my way through today with
this modified artistic flavor, for a non-VPATH build, in a Fedora
environment:
make coverage-html \
    LCOVFLAGS="-q --ignore-errors gcov,gcov" \
    GENHTML_FLAGS="-q --legend --ignore-errors unmapped,corrupt,unmapped,inconsistent"
--
Michael

  24. Re: Having problems generating a code coverage report

    Álvaro Herrera <alvherre@kurilemu.de> — 2026-02-26T12:35:35Z

    On 2026-Feb-18, Michael Paquier wrote:

> Based on your ideas, I have been able to get my way through today with
> this modified artistic flavor, for a non-VPATH build, in a Fedora
> environment:
> make coverage-html \
>     LCOVFLAGS="-q --ignore-errors gcov,gcov" \
>     GENHTML_FLAGS="-q --legend --ignore-errors unmapped,corrupt,unmapped,inconsistent"

FWIW this has been failing all along (last report dates back Feb 17th),
now because of this:

lcov: ERROR: (negative) Unexpected negative hit count '-9' for line /home/coverage/pgsrc/pgsql/src/port/snprintf.c:529" while capturing from ./src/port/snprintf_shlib.gcda.
        (use "lcov --ignore-errors negative ..." to bypass this error)

I added this flag as well, and the report is now running; we should have
results soon.

I hope the lcov people find time to fix all these problems :-(

-- 
Álvaro Herrera         PostgreSQL Developer  —  https://www.EnterpriseDB.com/
"Pero la cosa no es muy grave ..."     (le petit Nicolas -- René Goscinny)

  25. Re: Having problems generating a code coverage report

    Tom Lane <tgl@sss.pgh.pa.us> — 2026-04-30T22:45:39Z

    =?utf-8?Q?=C3=81lvaro?= Herrera <alvherre@kurilemu.de> writes:
> On 2026-Feb-18, Michael Paquier wrote:
>> Based on your ideas, I have been able to get my way through today with
>> this modified artistic flavor, for a non-VPATH build, in a Fedora
>> environment:
>> make coverage-html \
>> LCOVFLAGS="-q --ignore-errors gcov,gcov" \
>> GENHTML_FLAGS="-q --legend --ignore-errors unmapped,corrupt,unmapped,inconsistent"

> FWIW this has been failing all along (last report dates back Feb 17th),
> now because of this:
> lcov: ERROR: (negative) Unexpected negative hit count '-9' for line /home/coverage/pgsrc/pgsql/src/port/snprintf.c:529" while capturing from ./src/port/snprintf_shlib.gcda.
>         (use "lcov --ignore-errors negative ..." to bypass this error)
> I added this flag as well, and the report is now running; we should have
> results soon.

Another data point: I got around to updating my dev machine to RHEL
10.1 (with lcov version 2.0-1), and unsurprisingly coverage-html
stopped working for me too.  But here, it seems sufficient to use

    LCOVFLAGS="-q --ignore-errors gcov,gcov"
    GENHTML_FLAGS="-q --legend --ignore-errors unmapped,unmapped"

That still leaves me with a "no functions found" warning for each file
that compiles to empty with my build options.  I saw those on my old
installation too, but for some reason it seems like there are more
now.  I didn't find a way to suppress those, so that what my script
actually does now is

make -j16 -s coverage-html \
    LCOVFLAGS="-q --ignore-errors gcov,gcov" \
    GENHTML_FLAGS="-q --legend --ignore-errors unmapped,unmapped" \
    2>&1 | grep -v -e 'gcno:no functions found$' -e '^stderr:$' \
    -e 'WARNING: Duplicate specification "base-directory' \
    -e 'WARNING: Duplicate specification "build-directory'

Silence is golden ;-)

			regards, tom lane

  26. Re: Having problems generating a code coverage report

    Michael Paquier <michael@paquier.xyz> — 2026-05-01T04:30:29Z

    On Thu, Apr 30, 2026 at 06:45:39PM -0400, Tom Lane wrote:
> That still leaves me with a "no functions found" warning for each file
> that compiles to empty with my build options.  I saw those on my old
> installation too, but for some reason it seems like there are more
> now.  I didn't find a way to suppress those, so that what my script
> actually does now is

FWIW, I have just upgraded to Fedora 44 (lcov and genhtml at 2.0-1),
and neither am I able to find a way to remove these warnings.  A
coverage report that just works happens to be good enough for me these
days.  At least it works.
--
Michael

  27. Re: Having problems generating a code coverage report

    Jacob Champion <jacob.champion@enterprisedb.com> — 2026-06-11T00:49:47Z

    On Wed, Jan 21, 2026 at 5:59 PM Andres Freund <andres@anarazel.de> wrote:
> I have the following .lcovrc in my postgres source dir:
>
> ignore_errors=inconsistent,gcov,range
> check_data_consistency=0
> stop_on_error=0
> genhtml_hierarchical=1
> genhtml_show_navigation=1
> parallel=16
> geninfo_gcov_tool=/usr/bin/gcov-15
>
> (assuming you build with gcc 15)
>
> With that meson coverage information works (i.e. ninja coverage-html) for both
> lcov 2.0 and 2.4 as long as the build directory is outside of the source
> directory. Both on Debian Sid and macos.

Setting `genhtml_no_prefix = 1` fixes the directory name collision for
me (clang 22, lcov 2.4) without rehoming the build directory. (As does
hacking meson to pass the source root --prefix before the build root,
which makes for prettier output. Apparently lcov doesn't handle the
case where one prefix is a substring of the other very well.)

Also, in case it helps anyone: I found that clang-15 generates
outright buggy .gcno information for some of our #included C snippets,
such as `#include "equalfuncs.switch.c"`, and lcov-2's stricter
processing correctly complains about it. Apparently that was fixed at
some point with newer clangs.

--Jacob