Maven – How to generate Jacoco integration test report using offline instrumentation

code-coverageintegration-testingjacocomavenunit testing

Build tool: Maven

Reason to use offline instrumentation: Removing Powermock is not an option

Problem:
Both failsafe and surefire are run and reports are generated. However, jacoco.exec is generate but jacoco-it.exec is not. Apart from IT , offline instrumentation, coverage and reporting works fine.

This is the maven plugin configuration I use:

<plugin>
    <groupId>org.jacoco</groupId>
    <artifactId>jacoco-maven-plugin</artifactId>
    <executions>
        <execution>
            <id>default-instrument</id>
            <goals>
                <goal>instrument</goal>
            </goals>
        </execution>
        <execution>
            <id>default-restore-instrumented-classes</id>
            <goals>
                <goal>restore-instrumented-classes</goal>
            </goals>
        </execution>
        <execution>
            <id>default-report</id>
            <goals>
                <goal>report</goal>
            </goals>
        </execution>
        <execution>
            <id>default-report-integration</id>
            <goals>
                <goal>report-integration</goal>
            </goals>
        </execution>
    </executions>
</plugin>
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.16</version>
    <configuration>
        <systemPropertyVariables>
            <jacoco-agent.destfile>target/jacoco.exec</jacoco-agent.destfile>
        </systemPropertyVariables>
    </configuration>
</plugin>
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>2.15</version>
    <executions>
        <execution>
            <id>integration-tests</id>
            <goals>
                <goal>integration-test</goal>
                <goal>verify</goal>
            </goals>
            <configuration>
                <skipTests>${skip.integration.tests}</skipTests>
            </configuration>
        </execution>
    </executions>
</plugin>

To run the test i use maven clean install.

At the end of the test execution I get the following output:

[INFO] 
[INFO] --- maven-failsafe-plugin:2.15:integration-test (integration-tests) @ elune ---
[INFO] Skipping execution of surefire because it has already been run for this configuration
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.8:report (default-report) @ elune ---
[INFO] Loading execution data file C:\Projects\elune\target\jacoco.exec
[INFO] Analyzed bundle 'elune' with 4 classes
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.8:report-integration (default-report-integration) @ elune ---
[INFO] Skipping JaCoCo execution due to missing execution data file.

Another possible pointer might be that the de-instrumentation of the classes happens after the unit tests are run but before the integration tests. But I don't know if this is right or wrong:

[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.8:restore-instrumented-classes (default-restore-instrumented-classes) @ elune ---
[INFO] 
[INFO] --- maven-jar-plugin:2.6:jar (default-jar) @ elune ---
[INFO] Building jar: C:\Projects\elune\target\elune-0.0.1-SNAPSHOT.jar
[INFO] 
[INFO] --- spring-boot-maven-plugin:1.4.2.RELEASE:repackage (default) @ elune ---
[INFO] 
[INFO] --- maven-failsafe-plugin:2.15:integration-test (default) @ elune ---
[INFO] Failsafe report directory: C:\Projects\elune\target\failsafe-reports

Any idea why the jacoco-it.exec is not showing up?

Best Answer

I think the instrumentation of Integration Tests with the failsafe plugin are just not baked in. The restore-instrumented-classes goal defaults to the prepare-package phase which runs before the integration test phase: http://maven.apache.org/ref/3.3.9/maven-core/lifecycles.html - so it may be enough to move that goal into the post-integration-test phase:

<execution>
  <id>default-restore-instrumented-classes</id>
  <phase>post-integration-test</phase>
  <goals>
    <goal>restore-instrumented-classes</goal>
  </goals>
</execution>

that may be enough. Otherwise you may be able to change the includes pattern of the surefire plugin to also include the integration tests (if that seems suitable in your case).

Related Solutions

Java – How to test a class that has private methods, fields or inner classes

Update:

Some 10 years later perhaps the best way to test a private method, or any inaccessible member, is via @Jailbreak from the Manifold framework.
@Jailbreak Foo foo = new Foo();
// Direct, *type-safe* access to *all* foo's members
foo.privateMethod(x, y, z);
foo.privateField = value;
This way your code remains type-safe and readable. No design compromises, no overexposing methods and fields for the sake of tests.

If you have somewhat of a legacy Java application, and you're not allowed to change the visibility of your methods, the best way to test private methods is to use reflection.

Internally we're using helpers to get/set private and private static variables as well as invoke private and private static methods. The following patterns will let you do pretty much anything related to the private methods and fields. Of course, you can't change private static final variables through reflection.

Method method = TargetClass.getDeclaredMethod(methodName, argClasses);
method.setAccessible(true);
return method.invoke(targetObject, argObjects);

And for fields:

Field field = TargetClass.getDeclaredField(fieldName);
field.setAccessible(true);
field.set(object, value);

Notes:
1. TargetClass.getDeclaredMethod(methodName, argClasses) lets you look into private methods. The same thing applies for getDeclaredField.
2. The setAccessible(true) is required to play around with privates.

Maven – JaCoCo not generating jacoco.exec until after skipping JaCoCo execution

Just an addition to the answers already given. It could happen that in your maven-surefire-plugin configuration you already use the argLine configuration to override something like the memory used. If you do so the argline set by jacoco-maven-plugin will not be used failing to generate the jacoco report. In this case assign a property name to the jacoco-maven-plugin config and then reference it in your maven-surefire-plugin argLine parameter.

        <plugin>
            <groupId>org.jacoco</groupId>
            <artifactId>jacoco-maven-plugin</artifactId>
            <version>0.7.9</version>
            <executions>
                <!-- prepare agent for measuring unit tests -->
                <execution>
                    <id>prepare-unit-tests</id>
                    <goals>
                        <goal>prepare-agent</goal>
                    </goals>
                    <configuration>
                        <append>true</append>
                        <destFile>${sonar.jacoco.reportPath}</destFile>
                        <!-- Sets the VM argument line used when unit tests are run. -->
                        <propertyName>surefireArgLine</propertyName>
                    </configuration>
                </execution>
            </executions>
        </plugin>

[...]

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.19.1</version>
            <configuration>
                <printSummary>false</printSummary>
                <redirectTestOutputToFile>true</redirectTestOutputToFile>
                <forkCount>3</forkCount>
                <reuseForks>true</reuseForks>
                <argLine>${surefireArgLine} -Xmx1024m -noverify</argLine>                  
            </configuration>
        </plugin>

Best Answer

Related Solutions

Java – How to test a class that has private methods, fields or inner classes

Maven – JaCoCo not generating jacoco.exec until after skipping JaCoCo execution

Related Topic