ChatGPT解决这个技术问题 Extra ChatGPT

How do I get my Maven Integration tests to run

I have a maven2 multi-module project and in each of my child modules I have JUnit tests that are named Test.java and Integration.java for unit tests and integration tests respectively. When I execute:

mvn test

all of the JUnit tests *Test.java within the child modules are executed. When I execute

mvn test -Dtest=**/*Integration

none of the Integration.java tests get execute within the child modules.

These seem like the exact same command to me but the one with the -Dtest=/*Integration** does not work it displays 0 tests being run at the parent level, which there are not any tests

Kief's answer should be the accepted one, as it is the current standard for defining integration tests in Maven.

K
Kief

The Maven build lifecycle now includes the "integration-test" phase for running integration tests, which are run separately from the unit tests run during the "test" phase. It runs after "package", so if you run "mvn verify", "mvn install", or "mvn deploy", integration tests will be run along the way.

By default, integration-test runs test classes named **/IT*.java, **/*IT.java, and **/*ITCase.java, but this can be configured.

For details on how to wire this all up, see the Failsafe plugin, the Failsafe usage page (not correctly linked from the previous page as I write this), and also check out this Sonatype blog post.


@WillV Correct. The Codehaus' graveyard. And the maven-failsafe-plugin is in Apache now. Sorry. :)
By default mvn integration-test also runs unit tests (using via surefire) but mvn failsafe:integration-test runs only the failsafe integration tests.
Incredibly, the Failsafe plugin documentation, usage page and FAQ do not mention that it runs test classes named */IT.java, **/*IT.java, and **/*ITCase.java...
If it runs after package phase, that means I should put all my IT java source code under src/main/java instead of src/test/java right?
@HennoVermeulen I was confused about what to name tests as well. It's described in Inclusions and Exclusions of Tests. It's nice that the defaults can be overridden, but it'd be nice if they'd mention the defaults earlier on.
O
Olathe

You can set up Maven's Surefire to run unit tests and integration tests separately. In the standard unit test phase you run everything that does not pattern match an integration test. You then create a second test phase that runs just the integration tests.

Here is an example:

    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-surefire-plugin</artifactId>
      <configuration>
        <excludes>
          <exclude>**/*IntegrationTest.java</exclude>
        </excludes>
      </configuration>
      <executions>
        <execution>
          <id>integration-test</id>
          <goals>
            <goal>test</goal>
          </goals>
          <phase>integration-test</phase>
          <configuration>
            <excludes>
              <exclude>none</exclude>
            </excludes>
            <includes>
              <include>**/*IntegrationTest.java</include>
            </includes>
          </configuration>
        </execution>
      </executions>
    </plugin>

I configured this as you said and only the *Test not the *Integration.java files will run when executing: mvn install I need to run my *Test.java as the default, but for my nightlty build I need to run both *Test.java and *Integration.java. I have to execute mvn install then cd to each sub-child directory and execute mvn -Dtest=**/*Integration test
You should use the Fail-safe plugin for integration testing, not the sure-fire plugin. It won't fail the build until after the post-integration phase is complete; allowing you to tear down test resources (a web server, for example) before the build is failed. Hence, fail-safe.
for me, as part of pre-integration phase, jetty server starts. Last log line is : [INFO] Started Jetty Server. After that, nothing happens. It gets stuck. maven surefire failsafe plugin doesn't execute tests nor jetty server stops. Any idea what's wrong? I am using same configuration as specified by you.
This answer is very out of date and should be updated or removed.
J
Joshua Taylor

I have done EXACTLY what you want to do and it works great. Unit tests "*Tests" always run, and "*IntegrationTests" only run when you do a mvn verify or mvn install. Here it the snippet from my POM. serg10 almost had it right....but not quite.

  <plugin>
    <!-- Separates the unit tests from the integration tests. -->
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <configuration>
       <!-- Skip the default running of this plug-in (or everything is run twice...see below) -->
       <skip>true</skip>
       <!-- Show 100% of the lines from the stack trace (doesn't work) -->
       <trimStackTrace>false</trimStackTrace>
    </configuration>
    <executions>
       <execution>
          <id>unit-tests</id>
          <phase>test</phase>
          <goals>
             <goal>test</goal>
          </goals>
          <configuration>
                <!-- Never skip running the tests when the test phase is invoked -->
                <skip>false</skip>
             <includes>
                   <!-- Include unit tests within integration-test phase. -->
                <include>**/*Tests.java</include>
             </includes>
             <excludes>
               <!-- Exclude integration tests within (unit) test phase. -->
                <exclude>**/*IntegrationTests.java</exclude>
            </excludes>
          </configuration>
       </execution>
       <execution>
          <id>integration-tests</id>
          <phase>integration-test</phase>
          <goals>
             <goal>test</goal>
          </goals>
          <configuration>
            <!-- Never skip running the tests when the integration-test phase is invoked -->
             <skip>false</skip>
             <includes>
               <!-- Include integration tests within integration-test phase. -->
               <include>**/*IntegrationTests.java</include>
             </includes>
          </configuration>
       </execution>
    </executions>
  </plugin>

Good luck!


Exactly what I tried to do, but my integration tests kept running during mvn test phase, as I did NOT SKIP THE DEFAULT. I thought configuring the test execution would override it. As you explained, it just adds a new execution (thus everything would run twice). So for me the skip was missing piece. +1 As this configuration answers the question to 100%
Then feel free to check the box for this response being the answer!
for me, as part of pre-integration phase, jetty server starts. Last log line is : [INFO] Started Jetty Server. After that, nothing happens. It gets stuck. maven surefire failsafe plugin doesn't execute tests nor jetty server stops. Any idea what's wrong? I am using same configuration as specified by you.
@Tarun - ask a new question re your issue
This should be the accepted answer. The associated maven goal is: clean compile integration-test -Dmaven.test.failure.ignore=false
J
Jacob van Lingen

You can split them very easily using JUnit categories and Maven. This is shown very, very briefly below by splitting unit and integration tests.

Define A Marker Interface

public interface IntegrationTest {}

Mark your test classes

Add the category annotation to the top of your test class. It takes the name of your new interface.

import org.junit.experimental.categories.Category;

@Category(IntegrationTest.class)
public class ExampleIntegrationTest{

    @Test
    public void longRunningServiceTest() throws Exception {
    }

}

Configure Maven Unit Tests

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.11</version>
    <configuration>
        <includes>
            <include>**/*.class</include>
        </includes>
        <excludedGroups>
            com.test.annotation.type.IntegrationTest
        </excludedGroups>
    </configuration>
</plugin>

When you do a mvn clean test, only your unmarked unit tests will run.

Configure Maven Integration Tests

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>2.19.1</version>
    <configuration>
        <includes>
            <include>**/*.class</include>
        </includes>
        <groups>
            com.test.annotation.type.IntegrationTest
        </groups>
    </configuration>
</plugin>

The configuration uses a standard execution goal to run the failsafe plugin during the integration-test phase of the build.

You can now do a mvn clean install.
This time as well as the unit tests running, the integration tests are run during the integration-test phase.


I thought JUnit has no more secrets to me. Good spot!
This only works if the marker interface already exists somewhere available to Maven. It does not work if your marker interface exists in another module of the same multi-module build.
@EngineerBetter_DJ what do you mean by that ? You can't do that if you have a multi project based maven config ?
L
Liquidpie

You should use maven surefire plugin to run unit tests and maven failsafe plugin to run integration tests.

Please follow below if you wish to toggle the execution of these tests using flags.

Maven Configuration

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <configuration>
                <skipTests>${skipUnitTests}</skipTests>
            </configuration>
        </plugin>

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-failsafe-plugin</artifactId>
            <configuration>
                <includes>
                    <include>**/*IT.java</include>
                </includes>
                <skipTests>${skipIntegrationTests}</skipTests>
            </configuration>
            <executions>
                <execution>
                    <goals>
                        <goal>integration-test</goal>
                        <goal>verify</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>

        <properties>
            <skipTests>false</skipTests>
            <skipUnitTests>${skipTests}</skipUnitTests>
            <skipIntegrationTests>${skipTests}</skipIntegrationTests>
        </properties>

So, tests will be skipped or switched according to below flag rules:

Tests can be skipped by below flags:

-DskipTests skips both unit and integration tests

-DskipUnitTests skips unit tests but executes integration tests

-DskipIntegrationTests skips integration tests but executes unit tests

Running Tests

Run below to execute only Unit Tests

mvn clean test

You can execute below command to run the tests (both unit and integration)

mvn clean verify

In order to run only Integration Tests, follow

mvn failsafe:integration-test

Or skip unit tests

mvn clean install -DskipUnitTests

Also, in order to skip integration tests during mvn install, follow

mvn clean install -DskipIntegrationTests

You can skip all tests using

mvn clean install -DskipTests

Exactly the solution that I have been searching for may hours. Works fine!, Thankyou Liquidpie.
UPDATE: the skipXXTests trick doesn't work anymore. As of version 3.0.0 of failsafe, the <skipTests> option has been removed. Now, we need to use -DskipITs to skip integration tests, -DskipTests to skip all tests, and mvn integration-test to only run its (no unit tests).
J
James Kingsbery

You should try using maven failsafe plugin. You can tell it to include a certain set of tests.


+1 This is what I use. Works well, and allows you to do pre/post setup, such as starting and shutting down a local servlet container.
maven-failsafe-plugin has gone to Plugin Graveyard
The graveyard page just says the failsafe plugin has been moved to maven-failsafe-plugin. It looks like the maven-failsafe-plugin is still active (docs were last pushed March 2014).
D
Dov Benyomin Sohacheski

By default, Maven only runs tests that have Test somewhere in the class name.

Rename to IntegrationTest and it'll probably work.

Alternatively you can change the Maven config to include that file but it's probably easier and better just to name your tests SomethingTest.

From Inclusions and Exclusions of Tests:

By default, the Surefire Plugin will automatically include all test classes with the following wildcard patterns: \*\*/Test\*.java - includes all of its subdirectory and all java filenames that start with "Test". \*\*/\*Test.java - includes all of its subdirectory and all java filenames that end with "Test". \*\*/\*TestCase.java - includes all of its subdirectory and all java filenames that end with "TestCase". If the test classes does not go with the naming convention, then configure Surefire Plugin and specify the tests you want to include.


Hi and thanks I have two kinds of tests normal POJO Junit tests called SomethingTest.java which get fired. I also have integration tests called SomethingIntegration.java which do not get fired. The SomethingTest.java get fired via mvn test or mvn install. The second tests do not get fired. mvn test -Dtest=**/*Integration
.. and by "Maven only runs tests that have Test somewhere in the class name" you mean "the Maven surefire plugin only runs tests that have Test somewhere in the class name".
Its not "somewhere in the classname" its "the class name ends with Test", for example MyTest works but MyTests doesn't
S
Stephan

Another way of running integration tests with Maven is to make use of the profile feature:

...
<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <configuration>
                <includes>
                    <include>**/*Test.java</include>
                </includes>
                <excludes>
                    <exclude>**/*IntegrationTest.java</exclude>
                </excludes>
            </configuration>
        </plugin>
    </plugins>
</build>

<profiles>
    <profile>
        <id>integration-tests</id>
        <build>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <configuration>
                        <includes>
                            <include>**/*IntegrationTest.java</include>
                        </includes>
                        <excludes>
                            <exclude>**/*StagingIntegrationTest.java</exclude>
                        </excludes>
                    </configuration>
                </plugin>
            </plugins>
        </build>
    </profile>
</profiles>
...

Running 'mvn clean install' will run the default build. As specified above integration tests will be ignored. Running 'mvn clean install -P integration-tests' will include the integration tests (I also ignore my staging integration tests). Furthermore, I have a CI server that runs my integration tests every night and for that I issue the command 'mvn test -P integration-tests'.


Why wouldn't you use the integration-test phase? Profiles can then be used for things like integration-testing against various app servers, etc. like Arquillian does. I'm not a Maven expert, but I think the experts might say this is not very 'Maven-y'.
@Joshua I guess I do it this way because my integration tests take at least 5 minutes to run and I issue 'mvn clean install' many times a day because I need to update my artifacts in my local maven repo. According to what people are saying above, running 'install' will cause the integration-test phase to run causing me to lose precious developer time.
Hmm... not sure about 'install' running integration-test. In any case I'd still use the phase instead of a profile. Profiles are better off used for things like supporting different app servers, etc.
I'll go ahead and play around with that then. Thanks for the advice!
@jorge I guess the correct goal to use here would be verify, right?
D
Dherik

You can follow the maven documentation to run the unit tests with the build and run the integration tests separately.

<project>
    <properties>
        <skipTests>true</skipTests>
    </properties>
    [...]
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-failsafe-plugin</artifactId>
                <version>2.20.1</version>
                <configuration>
                    <skipITs>${skipTests}</skipITs>
                </configuration>
            </plugin>
        </plugins>
    </build>
    [...]
</project>

This will allow you to run with all integration tests disabled by default. To run them, you use this command:

mvn install -DskipTests=false

B
Bruno Negrão Zica

I didn't need to do all those setups suggested in other answers and I can run unit tests or integration tests separately.

What I did was:

add the config bellow to pom.xml in order to instruct maven where are the integration tests files (the java code and the resources files):

<plugins>
    <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>build-helper-maven-plugin</artifactId>
        <executions>
            <execution>
                <id>add-integration-test-source</id>
                <phase>generate-test-sources</phase>
                <goals>
                    <goal>add-test-source</goal>
                </goals>
                <configuration>
                    <sources>
                        <source>src/integration-test/java</source>
                    </sources>
                </configuration>
            </execution>
            <execution>
                <id>add-integration-test-resource</id>
                <phase>generate-test-resources</phase>
                <goals>
                    <goal>add-test-resource</goal>
                </goals>
                <configuration>
                    <resources>
                        <resource>
                            <directory>src/integration-test/resources</directory>
                        </resource>
                    </resources>
                </configuration>
            </execution>
        </executions>
    </plugin>
</plugins>

Name the integration test files with the "IT" suffix, like "MyTestIT.java" Execute only the integration tests with this command: mvn failsafe:integration-test failsafe:verify

In the command above "failsafe:integration-test" will execute the *IT.java files and "failsafe:verify" will tell maven to check if any of the integration tests ran previously have failed. If you don't use 'failsafe:verify' the mvn command will exit with success even if one test has failed.

To run only the unit tests, run: mvn test


Am I really MAD if I find it UNBELIEVABLE when such an ESSENTIAL functionality like this SIMPLE separation of running Unit-tests or Integration-tests still isn't working RELIABLY in Maven ? Do I really want too much ?