[gcov] No data gathered for child processes

[Didier "Ptitjes"]

Hello list,

I'm using lcov to analyze the coverage of libgee's test suite. Libgee is
based on GLib and as thus I use GTest framework of GLib.

GTest forks the test process in order to test for asserting tests.
Looking at the reports of lcov, it looks like there is absolutely no
information gathered about the child processes.

Indeed my tests pass so I'm sure I do branch by those code parts
reported as never branched by lcov.

I already have seen this question asked two times on gcc-help but none
got a real answer:

http://gcc.gnu.org/ml/gcc-help/2009-07/msg00334.html

and

http://gcc.gnu.org/ml/gcc/2008-09/msg00490.html

For information, I'm running Gentoo Linux with 2.6.28 kernel in a
virtual machine (Parallels/MacOS X 10.5). My hardware is Core 2 Duo 2.6GHz.

Thanks a lot for your help.
Didier.

[Holger Blasum]

Hello Didier,

Post by Didier "Ptitjes"
GTest forks the test process in order to test for asserting tests.
Looking at the reports of lcov, it looks like there is absolutely no
information gathered about the child processes.

...

Post by Didier "Ptitjes"
I already have seen this question asked two times on gcc-help but none
http://gcc.gnu.org/ml/gcc-help/2009-07/msg00334.html
http://gcc.gnu.org/ml/gcc/2008-09/msg00490.html

Hmm, I have no quick idea how to fix your problem, but it might be
helpful to understand where it occurs (then one can think of a
work-around): basically the gcov tools work in a way
(1) that enabling -ftest-coverage will result in a different binary
than the binary without -ftest-coverage (1.4.1 in [1]), this gives you
*.gcno files
(2) when the instrumented binary it is run you have a data
collection (1.4.2 in [1]) phase where the binary adds up
counters in memory whenever passing through C sequence points
and then you have a reporting phase where the memory data is dumped
into *.gcda files.
(3) then there is a reporting phase (1.4.3 in [1]) where that *.gcda
and *.gcno files are converted into *.gcov files

The first thing to divide and conquer this would be to test and report
* whether after the phase (1) all *.gcno files are built for both
parent and child,
* whether after the phase (2) all *.gcda files are built/modified
for parent and child.

[1] http://sysrun.haifa.il.ibm.com/hrl/greps2007/papers/gcov-on-an-embedded-system.pdf

HTH,

[Didier 'Ptitjes']

Hi Holger,

First thanks for your support.

Post by Holger Blasum
Hello Didier,

Post by Didier "Ptitjes"
GTest forks the test process in order to test for asserting tests.
Looking at the reports of lcov, it looks like there is absolutely no
information gathered about the child processes.

...

Post by Didier "Ptitjes"
I already have seen this question asked two times on gcc-help but none
http://gcc.gnu.org/ml/gcc-help/2009-07/msg00334.html
http://gcc.gnu.org/ml/gcc/2008-09/msg00490.html

Hmm, I have no quick idea how to fix your problem, but it might be
helpful to understand where it occurs (then one can think of a
work-around): basically the gcov tools work in a way
(1) that enabling -ftest-coverage will result in a different binary
than the binary without -ftest-coverage (1.4.1 in [1]), this gives you
*.gcno files
(2) when the instrumented binary it is run you have a data
collection (1.4.2 in [1]) phase where the binary adds up
counters in memory whenever passing through C sequence points
and then you have a reporting phase where the memory data is dumped
into *.gcda files.
(3) then there is a reporting phase (1.4.3 in [1]) where that *.gcda
and *.gcno files are converted into *.gcov files
The first thing to divide and conquer this would be to test and report
* whether after the phase (1) all *.gcno files are built for both
parent and child,
* whether after the phase (2) all *.gcda files are built/modified
for parent and child.

OK, I can confirm that I actually have *.gcno and *.gcda files for every
of my library .c files. (I tried to build them for test cases themselves
without any change).

You must note that the parent process and the child processes have the
exact same code. In fact, the parent process is just forked.

For an example of the forking code you can look at:
http://git.gnome.org/cgit/libgee/tree/tests/testreadonlycollection.vala#n121

This is Vala code but it is just transformed into the following C code:

<code>
if (g_test_trap_fork ((guint64) 0, G_TEST_TRAP_SILENCE_STDOUT |
G_TEST_TRAP_SILENCE_STDERR)) {
g_assert (gee_collection_add (self->ro_collection, "two"));
_g_object_unref0 (dummy);
return;
}
g_test_trap_assert_failed ();
g_assert (gee_collection_get_size (self->ro_collection) == 1);
g_assert (gee_collection_contains (self->ro_collection, "one"));
</code>

g_test_trap_fork uses fork() to fork the process.

Here is the g_test_trap_fork code if this can enlighten the problem:
http://git.gnome.org/cgit/glib/tree/glib/gtestutils.c#n1582

I did not yet resolve the problem. I hope I will be able with your help.

Best regards, Didier.

[Holger Blasum]

Hi Didier,

Post by Didier 'Ptitjes'
You must note that the parent process and the child processes have the
exact same code. In fact, the parent process is just forked.

Ah now I understand. Well then one has a classical race condition:
the same infrastructure for storing coverage data (*.gcda files) is
used by both parent and child, and who terminates last will get the
overwrite I guess.

Post by Didier 'Ptitjes'
I did not yet resolve the problem. I hope I will be able with your help.

Fixing ideas:
(1) try renaming functions in your child (ugly, I agree, but could be automated).
Maybe first try out whether this works with a very small example.
(2) modify gcov so that that the gcda files are written in a way that if
a child with say process ID (pid) 2 has a main.c
and a parent with say pid 1 has main.c, then the data is dumped into
main.1.gcda, main.2.gcda, and modify gcov.c to sum up both

BTW, anyone else is encouraged to comment on too (I had just been a user
of gcov too - no contributions to development). There is also a list
with gcov knowledge at the lcov project
http://ltp.sourceforge.net/coverage/lcov.php

[Holger Blasum]

Post by Holger Blasum

Post by Didier 'Ptitjes'
You must note that the parent process and the child processes have the
exact same code. In fact, the parent process is just forked.

the same infrastructure for storing coverage data (*.gcda files) is
used by both parent and child, and who terminates last will get the
overwrite I guess.

Let me stress: I'm not sure at the moment whether that race condition
is necessary or just accidental (and no-one has ever bothered to fix
it). Study the gcov sources (not long and documented in the paper I
linked to in yesterday's mail) - don't have patience atm to do that myself.
Special things to look at is e.g. atexit() and how this is used in
a forking context.

Post by Holger Blasum

Post by Didier 'Ptitjes'
I did not yet resolve the problem. I hope I will be able with your help.

(1) try renaming functions in your child (ugly, I agree, but could be automated).
Maybe first try out whether this works with a very small example.
(2) modify gcov so that that the gcda files are written in a way that if
a child with say process ID (pid) 2 has a main.c
and a parent with say pid 1 has main.c, then the data is dumped into
main.1.gcda, main.2.gcda, and modify gcov.c to sum up both
BTW, anyone else is encouraged to comment on too (I had just been a user
of gcov too - no contributions to development). There is also a list
with gcov knowledge at the lcov project
http://ltp.sourceforge.net/coverage/lcov.php

Oops, https://lists.sourceforge.net/lists/listinfo/ltp-coverage to give
a more useful pointer.

[Scott Spivak]

Post by Didier 'Ptitjes'
Hi Holger,
First thanks for your support.

Post by Holger Blasum
Hello Didier,

Post by Didier "Ptitjes"
GTest forks the test process in order to test for asserting tests.
Looking at the reports of lcov, it looks like there is absolutely no
information gathered about the child processes.

...

Post by Didier "Ptitjes"
I already have seen this question asked two times on gcc-help but none
http://gcc.gnu.org/ml/gcc-help/2009-07/msg00334.html
http://gcc.gnu.org/ml/gcc/2008-09/msg00490.html

Hmm, I have no quick idea how to fix your problem, but it might be
helpful to understand where it occurs (then one can think of a
work-around): basically the gcov tools work in a way
(1) that enabling -ftest-coverage will result in a different binary
than the binary without -ftest-coverage (1.4.1 in [1]), this gives you
*.gcno files
(2) when the instrumented binary it is run you have a data
collection (1.4.2 in [1]) phase where the binary adds up
counters in memory whenever passing through C sequence points
and then you have a reporting phase where the memory data is dumped
into *.gcda files.
(3) then there is a reporting phase (1.4.3 in [1]) where that *.gcda
and *.gcno files are converted into *.gcov files
The first thing to divide and conquer this would be to test and report
* whether after the phase (1) all *.gcno files are built for both
parent and child,
* whether after the phase (2) all *.gcda files are built/modified
for parent and child.

OK, I can confirm that I actually have *.gcno and *.gcda files for every
of my library .c files. (I tried to build them for test cases themselves
without any change).
You must note that the parent process and the child processes have the
exact same code. In fact, the parent process is just forked.

http://git.gnome.org/cgit/libgee/tree/tests/testreadonlycollection.vala#n121

Post by Didier 'Ptitjes'
<code>
if (g_test_trap_fork ((guint64) 0, G_TEST_TRAP_SILENCE_STDOUT |
G_TEST_TRAP_SILENCE_STDERR)) {
g_assert (gee_collection_add (self->ro_collection, "two"));
_g_object_unref0 (dummy);
return;
}
g_test_trap_assert_failed ();
g_assert (gee_collection_get_size (self->ro_collection) == 1);
g_assert (gee_collection_contains (self->ro_collection, "one"));
</code>
g_test_trap_fork uses fork() to fork the process.
http://git.gnome.org/cgit/glib/tree/glib/gtestutils.c#n1582
I did not yet resolve the problem. I hope I will be able with your help.
Best regards, Didier.

FWIW, I ran into the same issue and, since I only really care whether an
assert (and thus an abort()) is getting called during a given test, I came
up with the following. This approach doesn't require the forking of a child
process as ASSERT_DEATH does, and so I was able to capture gcov data for the
assert() statement. Note that simply catching SIGABRT wasn't enough,
because abort() still terminates the program after the signal handler
returns (which I don't want), hence the setjmp and longjmp.

static jmp_buf jmpBuf;

void abortHandler(int param)
{
abortCalled = true;
void (*prev_handler)(int);

prev_handler = signal(SIGABRT, abortHandler);

longjmp(jmpBuf, 1);
}

#define ASSERT_ABORT(statement) \
abortCalled = false; \
\
signal(SIGABRT, abortHandler); \
\
if (!setjmp(jmpBuf)) \
{ \
statement; \
} \
\
ASSERT_TRUE(abortCalled);

You can use it it like any other gtest macro:

ASSERT_ABORT(myObject.doSomething());


--Scott