Integration testing in JIRA Server

Hello,

We are developing a wide range of plugins, using different technologies (Kotlin, Groovy and Java).

At the moment we are trying to add our first integration tests, but to no avail. We couldn’t find any piece of documentation that actually works when updating dependencies to current versions. Creating a new plugin with atlas-create-jira-plugin works, ands it runs the tests (both unit and integration), but after tweaking the dependencies (testkit, JIRA, atlassian-plugins-osgi-testrunner) it breaks. We also found that adding kotlin-test dependency completely breaks the build (even when not using actual kotlin sources at this point!).

What is the Atlassian-recommended/correct way of doing integration testing? Are there any resources we can leverage to learn how to do these tests with for a modern-ish JIRA version?

We are using JIRA 7.11.2, and will be updating to 8.x this year, so we are looking for a solution that works in both of them, using OSGi, and is actually maintained by Atlassian. :slightly_smiling_face:

An example of one of the thrown errors:

[INFO] <?xml version="1.0" encoding="UTF-8"?>
<configuration>
  <includes>
    <include>%regex[it[/\\].*]</include>
  </includes>
  <excludes>
    <exclude>**/*$*</exclude>
    <exclude>**/Abstract*</exclude>
  </excludes>
  <systemPropertyVariables>
    <http.port>2990</http.port>
    <reportsDirectory>/tmp/jira-testing/target/group-__no_test_group__/tomcat85x/surefire-reports</reportsDirectory>
    <baseurl.jira>...:2990/jira</baseurl.jira>
    <http.jira.port>2990</http.jira.port>
    <product.7.11.2.version>7.11.2</product.7.11.2.version>
    <product.jira.id>jira</product.jira.id>
    <homedir>/tmp/jira-testing/target/jira/home</homedir>
    <homedir.jira>/tmp/jira-testing/target/jira/home</homedir.jira>
    <baseurl>...:2990/jira</baseurl>
    <testGroup>__no_test_group__</testGroup>
    <context.path>/jira</context.path>
    <context.jira.path>/jira</context.jira.path>
    <http.jira.url>...:2990/jira</http.jira.url>
    <plugin.jar>/tmp/jira-testing/target/mypackage-1.0.0-SNAPSHOT.jar</plugin.jar>
    <http.jira.protocol>http</http.jira.protocol>
  </systemPropertyVariables>
  <reportsDirectory>/tmp/jira-testing/target/group-__no_test_group__/tomcat85x/surefire-reports</reportsDirectory>
</configuration>
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running it.com.mypackage.MyComponentWiredTest
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[INFO] [talledLocalContainer] Mar 28, 2019 12:52:53 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
[INFO] [talledLocalContainer] INFO: Initiating Jersey application, version 'Jersey: 1.19 02/11/2015 03:25 AM'
[INFO] [talledLocalContainer] Mar 28, 2019 12:52:54 PM com.sun.jersey.api.wadl.config.WadlGeneratorLoader loadWadlGenerator
[INFO] [talledLocalContainer] INFO: Loading wadlGenerator com.sun.jersey.server.wadl.generators.WadlGeneratorApplicationDoc
[INFO] [talledLocalContainer] Mar 28, 2019 12:52:54 PM com.sun.jersey.api.wadl.config.WadlGeneratorLoader loadWadlGenerator
[INFO] [talledLocalContainer] INFO: Loading wadlGenerator com.atlassian.plugins.rest.doclet.generators.grammars.WadlGrammarsAdaptor
[INFO] [talledLocalContainer] Mar 28, 2019 12:52:54 PM com.sun.jersey.api.wadl.config.WadlGeneratorLoader loadWadlGenerator
[INFO] [talledLocalContainer] INFO: Loading wadlGenerator com.atlassian.plugins.rest.doclet.generators.resourcedoc.AtlassianWadlGeneratorResourceDocSupport
[INFO] [talledLocalContainer] 2019-03-28 12:52:55,022 http-nio-2990-exec-6 ERROR anonymous 772x2x2 - 127.0.0.1 /rest/atlassiantestrunner/1.0/runtest/it.com.mypackage.MyComponentWiredTest [c.a.p.r.c.error.jersey.ThrowableExceptionMapper] Uncaught exception thrown by REST service: null                                                                                                                                    
[INFO] [talledLocalContainer] java.lang.NullPointerException
[INFO] [talledLocalContainer]   at com.google.gson.internal.bind.TypeAdapters$1.write(TypeAdapters.java:64)
[INFO] [talledLocalContainer]   at com.google.gson.internal.bind.TypeAdapters$1.write(TypeAdapters.java:61)
[INFO] [talledLocalContainer]   at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68)
[INFO] [talledLocalContainer]   at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.write(ReflectiveTypeAdapterFactory.java:89)
[INFO] [talledLocalContainer]   at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.write(ReflectiveTypeAdapterFactory.java:195)
[INFO] [talledLocalContainer]   at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68)
[INFO] [talledLocalContainer]   at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.write(ReflectiveTypeAdapterFactory.java:89)
[INFO] [talledLocalContainer]   at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.write(ReflectiveTypeAdapterFactory.java:195)
[INFO] [talledLocalContainer]   at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68)
[INFO] [talledLocalContainer]   at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.write(CollectionTypeAdapterFactory.java:96)
[INFO] [talledLocalContainer]   at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.write(CollectionTypeAdapterFactory.java:60)
[INFO] [talledLocalContainer]   at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68)
[INFO] [talledLocalContainer]   at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.write(ReflectiveTypeAdapterFactory.java:89)
[INFO] [talledLocalContainer]   at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.write(ReflectiveTypeAdapterFactory.java:195)
[INFO] [talledLocalContainer]   at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68)
[INFO] [talledLocalContainer]   at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.write(ReflectiveTypeAdapterFactory.java:89)
[INFO] [talledLocalContainer]   at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.write(ReflectiveTypeAdapterFactory.java:195)
[INFO] [talledLocalContainer]   at com.google.gson.Gson.toJson(Gson.java:581)
[INFO] [talledLocalContainer]   at com.google.gson.Gson.toJson(Gson.java:560)
[INFO] [talledLocalContainer]   at com.google.gson.Gson.toJson(Gson.java:515)
[INFO] [talledLocalContainer]   at com.google.gson.Gson.toJson(Gson.java:495)
[INFO] [talledLocalContainer]   at com.atlassian.plugins.osgi.test.rest.TestRunnerResource.runTest(TestRunnerResource.java:57)
[INFO] [talledLocalContainer]   ... 3 filtered
[INFO] [talledLocalContainer]   at java.lang.reflect.Method.invoke(Method.java:498)
[INFO] [talledLocalContainer]   ... 19 filtered
[INFO] [talledLocalContainer]   at com.atlassian.plugins.rest.module.RestDelegatingServletFilter$JerseyOsgiServletContainer.doFilter(RestDelegatingServletFilter.java:154)
[INFO] [talledLocalContainer]   ... 1 filtered
[INFO] [talledLocalContainer]   at com.atlassian.plugins.rest.module.RestDelegatingServletFilter.doFilter(RestDelegatingServletFilter.java:68)
[INFO] [talledLocalContainer]   ... 41 filtered
[INFO] [talledLocalContainer]   at com.atlassian.web.servlet.plugin.request.RedirectInterceptingFilter.doFilter(RedirectInterceptingFilter.java:21)
[INFO] [talledLocalContainer]   ... 24 filtered
[INFO] [talledLocalContainer]   at com.atlassian.labs.httpservice.resource.ResourceFilter.doFilter(ResourceFilter.java:59)
[INFO] [talledLocalContainer]   ... 32 filtered
[INFO] [talledLocalContainer]   at com.atlassian.jira.security.JiraSecurityFilter.lambda$doFilter$0(JiraSecurityFilter.java:66)
[INFO] [talledLocalContainer]   ... 1 filtered
[INFO] [talledLocalContainer]   at com.atlassian.jira.security.JiraSecurityFilter.doFilter(JiraSecurityFilter.java:64)
[INFO] [talledLocalContainer]   ... 16 filtered
[INFO] [talledLocalContainer]   at com.atlassian.plugins.rest.module.servlet.RestSeraphFilter.doFilter(RestSeraphFilter.java:37)
[INFO] [talledLocalContainer]   ... 19 filtered
[INFO] [talledLocalContainer]   at com.atlassian.jira.servermetrics.CorrelationIdPopulatorFilter.doFilter(CorrelationIdPopulatorFilter.java:30)
[INFO] [talledLocalContainer]   ... 10 filtered
[INFO] [talledLocalContainer]   at com.atlassian.web.servlet.plugin.request.RedirectInterceptingFilter.doFilter(RedirectInterceptingFilter.java:21)
[INFO] [talledLocalContainer]   ... 4 filtered
[INFO] [talledLocalContainer]   at com.atlassian.web.servlet.plugin.LocationCleanerFilter.doFilter(LocationCleanerFilter.java:36)
[INFO] [talledLocalContainer]   ... 26 filtered
[INFO] [talledLocalContainer]   at com.atlassian.jira.servermetrics.MetricsCollectorFilter.doFilter(MetricsCollectorFilter.java:25)
[INFO] [talledLocalContainer]   ... 24 filtered
[INFO] [talledLocalContainer]   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[INFO] [talledLocalContainer]   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[INFO] [talledLocalContainer]   at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
[INFO] [talledLocalContainer]   at java.lang.Thread.run(Thread.java:748)
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.708 s <<< FAILURE! - in it.com.mypackage.MyComponentWiredTest
[ERROR] it.com.mypackage.MyComponentWiredTest  Time elapsed: 2.702 s  <<< ERROR!
java.lang.IllegalStateException: Could not find resource for test [.../jira/rest/atlassiantestrunner/1.0/runtest/it.com.mypackage.MyComponentWiredTest]. Status: 500 - null

[INFO] 
[INFO] Results:
[INFO] 
[ERROR] Errors: 
[ERROR]   MyComponentWiredTest.it.com.mypackage.MyComponentWiredTest ? IllegalState ...
[INFO] 
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
[INFO] 
[INFO] jira: Shutting down
[INFO] using codehaus cargo v1.6.10
[INFO] [talledLocalContainer] Tomcat 8.x is stopping...ing...

Adaptavist’s Arquillian integration for Atlassian products may be useful for you: atlassian-arquillian-containers

All of our integration testing is implemented with the above.