Java JUnit: programmatically setup a test retry mechanism
In a Java CI pipeline, sometimes it is useful to retry tests, to handle false positive errors (such as network errors).
To do so, we can use the org.junit-pioneer library. Also, we want to be able to change the number of retry attempts programmatically,
and disable the retry mechanism when the tests are run from an IDE like IntelliJ.
First let’s add the lib-custom, junit and junit-pioneer dependencies to our pom.xml:
<dependency>
<groupId>io.github.jleblanc64</groupId>
<artifactId>lib-custom</artifactId>
<version>1.0.7</version>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<version>5.9.3</version>
</dependency>
<dependency>
<groupId>org.junit-pioneer</groupId>
<artifactId>junit-pioneer</artifactId>
<version>2.2.0</version>
</dependency>Then, let’s create a class that wil extend io.github.jleblanc64.libcustom.custom.test.TestRetryDefault,
and will load some custom code that dynamically passes our parameters to junit-pioneer:
import io.github.jleblanc64.libcustom.custom.test.TestRetryDefault;
public class TestRetry extends TestRetryDefault {
static {
MAX_ATTEMPTS = 5;
DISABLE = TestRetry::isIntelliJ;
retryTests();
}
static boolean isIntelliJ() {
var javaCmd = System.getProperty("sun.java.command", "").toLowerCase();
return javaCmd.contains("intellij");
}
}Now, we can extend all of our existing test classes with this new TestRetry class.
All test methods that have the annotation org.junit.jupiter.api.Test will be retried on failure, using the parameters from above.
For instance, the following test will be retried 4 times (if not run from IntelliJ):
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals;
public class TestRetryTests extends TestRetry {
@Test
public void test() {
assertEquals(1, 2);
}
}You can see the full working code in my Github repo. You can ask a question in the issues if you need.