SoapUI Runner Gradle Plugin (5.3.0-5)

Introduction

This is a user manual of soapui-runner. soap-runner it’s a gradle plugin written in groovy. It simplified configuration of SoapUI test projects and provides theirs cross-platform execution (at the moment supports only testrunner and loadtestrunner, but it could be very easilly extend - feel free to contribute).

1. Why?

1.1. Officially, gradle plugin is not supported

Currently SmartBear is not supporting gradle plugin. At the moment only SoapUI Maven Plugin officially supported.

SmartBear provides platform dependant scripts for SoapUI test project execution, but it’s not so convenient. For example, to execute SoapUI tests using testrunner (version: 5.3.0) tool you can use these commands:

Example, how to execute tests on windows using SoapUI testrunner.bat

C:\path\to\SoapUI-5.3.0\bin\testrunner.bat C:\path\to\soapui-test-project.xml

Example, how to execute testrunner.sh on unix

/path/to/SoapUI-5.3.0.app/Contents/Resources/app/bin/testrunner.sh /path/to/soapui-test-project.xml

1.2. Running tests using soapui-runner gradle plugin

Using preconfigured build script you only need to execute a single command:

gradle testrunner # or:
gradle loadtestrunner

2. What’s inside?

There are few plugins and tasks you have to get all functionality you might need …almost :) Currently, in my daily work and for testing API of my projects, I was required to use only testrunner and loadtestrunner

Note

Anyway, All other existing runner in SoapUI could be easily added to soapui-runner plugin, really. Contributors are welcome!

2.1. Available plugins and tasks

2.1.1. Three plugins available on Gradle Portal

SoapUI Test Runner Plugin soapui-testrunner (io.github.daggerok.soapui-testrunner)
SoapUI Load Test Runner Plugin soapui-loadtestrunner (io.github.daggerok.soapui-loadtestrunner)
SoapUI Runner Plugin soapui-runner (io.github.daggerok.soapui-runner) - contains all tasks: testrunner and loadtestrunner

2.1.2. Three tasks available right now

extDir: helpful task for generating ext folder to be copied inside $SOAPUI_HOME/bin (see Generate directory for external libraries)
testrunner: task run SoapUI Test Suites and/or Test Cases (see Test (suites and cases) runner)
loadtestrunner: task run SoapUI Load Tests (see Load Tests runner)

3. Installation

There are few ways to integrate soapui-runner in your gradle project. We are recommending you usi that plugin in separate project. It’s better to not include soapui-runner gradle project as subproject of main project is going to be tested to avoid complexity of dependency resolution, but this is not required.

In general, it could be installed just like so:

Using plugins approach

buildscript {
  repositories {
    maven { url "https://plugins.gradle.org/m2/" }
    maven { url "http://smartbearsoftware.com/repository/maven2/" }
  }
}

plugins {
  id "io.github.daggerok.soapui-runner" version "5.3.0-5"
}

3.1. Multi-project build

If you are using multi-project build or for some reasons you don’t want apply runner-plugin immediately:

Do not apply plugin in plugins block

buildscript {
  repositories {
    maven { url "https://plugins.gradle.org/m2/" }
    maven { url "http://smartbearsoftware.com/repository/maven2/" }
  }
}

plugins {
  id "io.github.daggerok.soapui-runner" version "5.3.0-5" apply false
}

subprojects {
  apply plugin: "io.github.daggerok.soapui-runner"
  // ...
}

3.2. Gradle version less then 3.0

If you are using gradle ⇐ 3.0, then use apply plugin approach only:

Use apply plugin approach

buildscript {
  repositories {
    maven { url "https://plugins.gradle.org/m2/" }
    maven { url "http://smartbearsoftware.com/repository/maven2/" }
  }
  dependencies {
    classpath "gradle.plugin.io.github.daggerok:soapui-runner:5.3.0-5"
  }
}

apply plugin: "io.github.daggerok.soapui-runner"

Important

Make sure you have added required smartbear software maven repository in your buildscript for correct plugin dependencies resolution.

3.2.1. Many ways apply plugins…

You can apply FQDN class name for each plugin instead (see plugins main classes source code for details):

// turns on extDir and testrunner tasks:
apply plugin: io.github.daggerok.SoapUITestRunnerPlugin // or:
apply plugin: "io.github.daggerok.soapui-testrunner"

// turns on extDir and loadtestrunner tasks:
apply plugin: io.github.daggerok.SoapUILoadTestRunnerPlugin // or:
apply plugin: "io.github.daggerok.soapui-loadtestrunner"

// turns on all tasks: extDir, testrunner and loadtestrunner:
apply plugin: io.github.daggerok.SoapUIRunnerPlugin // or:
apply plugin: "io.github.daggerok.soapui-runner"

4. Configuration plugins tasks

You can pre-configure once all needed properties / configurations using runner tasks closures

Configuration example

testrunner {
  projectFile "src/test/resources/soapui-test-project.xml"
  outputFolder "out/tests"
  failOnError true

  projectProperties = [
    "apiBaseUrl=https://api.github.com"
  ]
}

loadtestrunner {
  projectFile = "$projectDir/soapui-load-tests.xml"
  outputFolder = "buildDir/soapui/load"
}

Feel free omit all runners configurations if default Runners properties are good enough for you:

/*
testrunner {
  projectFile = "soapui-test-project.xml"
  outputFolder = "build/soapui"
  // ...
}

loadtestrunner {
  projectFile = "soapui-test-project.xml"
  outputFolder = "build/soapui"
  // ...
}
*/

4.1. Test (suites and cases) runner

Very extendable and configurable SoapUI testrunner task. For details see: Test runner specific configuration properties

Example of testrunner task configuration

testrunner {
  projectFile = "soapui-test-project.xml"
  outputFolder = "build/soapui
}

Execution of test runner task

gradle testrunner

4.1.1. Test runner configuration customization

Using groovy inside gradle build files we can do pretty much whatever we want.

Look, how easy you can specify exact testSuites for run

Example of testrunner task configuration: how to execute only needed TestSuite list

apply plugin: io.github.daggerok.SoapUITestRunnerPlugin

task soapUITestSuites(dependsOn: [

  "TestSuite 1",
  "TestSuite 2",

].collect { suiteName ->
  tasks.create(name: suiteName, type: io.github.daggerok.tasks.SoapUITestRunnerTask) {
    testSuite = suiteName
    outputFolder = "$buildDir/soapui/$suiteName"
  }
})

Note

same approach can be used for testCases.

Look, how to execute only needed test cases

Example of testrunner execution for some concrete TestCases

apply plugin: "io.github.daggerok.soapui-testrunner"

import io.github.daggerok.tasks.SoapUITestRunnerTask

Task[] soapUITasks = [

    "TestCase 1",
    "TestCase 2",
    "TestCase 3",
    "TestCase 4",
    "TestCase 5",

].collect { testCaseName ->

  def noSpaceCase = testCaseName.replaceAll(/\s+$/, "").capitalize()

  tasks.create(name: noSpaceCase, type: SoapUITestRunnerTask) {
    testCase = noSpaceCase
    outputFolder = "$buildDir/soapui/testCases/$noSpaceCase"
    projectProperties = [
        "apiEndpoint=https://jsonplaceholder.typicode.com"
    ]
  }
}

task soapUITestCases(dependsOn: soapUITasks)

4.2. Load Tests runner

Expandable and configurable load test runner task. See Load test runner specific configuration properties for more details

Example of loadtestrunner task configuration

loadtestrunner {
  projectFile = "soapui-test-project.xml"
  outputFolder = "build/soapui"
}

Load test runner execution

gradle loadtestrunner

4.3. Generate directory for external libraries

Sometimes we need use some external packages, like jdbc drivers. for that purposes we have to add them inside $SOAPUI_HOME/bin/ext directory to simplify that process, we can generate needed ext folder to copy it later inside SoapUI ext dir

repositories {
  jcenter()
}

dependencies {
  extDir "org.postgresql:postgresql:9.4.1212.jre7"
}

gradle extDir
...
cp -Rf build/soapui/ext $SOAPUI_HOME/bin/ext

see SoapUI systemProperty: soapui.ext.libraries: testrunnert -Dsoapui.ext.libraries=…

4.4. Adding new and override generic (parent) properties

If you are using gradle multi-project build, you can define base configuration inside parent build:

in build.gradle file

allprojects {
  testrunner {
    projectProperties = [
        "os=base",
        "url=https://example.com"
    ]
    systemProperties = [
        "soapui.ext.libraries=$buildDir"
    ]
    // ...
  }
}

…and later you might need to override it inside some of your children builds:

in ./modules/windows/build.gradle file

testrunner {
  projectProperties = [
      // add new:
      "newPropjectProp=adding",
      // override existing:
      "os=windows",
      "url=https://microsoft.com"
  ]
  systemProperties = [
      // override existing:
      "soapui.ext.libraries=C:/path/to/SoapUI-5.3.0/bin/ext"
  ]
  // ...
}

5. Appendix

5.1. Runners properties

5.1.1. Base (allowed for testrunner and loadtestrunner tasks)

property	arg	default	description
failOnError	N/A	true	sets if gradle plugin execution should stop and fails on any plugin configuration errors occurs
projectFile	N/A	soapui-test-project.xml	sets the SoapUI project file containing the tests to run
outputFolder	-f	build/soapui	sets the output folder to export results to
projectProperties	-P		sets list of "key=value" project properties
globalProperties	-G		sets list of "key=value" global properties
systemProperties	-D		sets list of "key=value" system properties
settingsFile	-t		sets the SoapUI settings file
endpoint	-e		sets the endpoint to use for all test requests
domain	-d		sets the domain to use for any authentications
host	-h		sets the host to use by all test-requests, the existing endpoint port and path will be used
username	-u		sets the username to use for any authentications
password	-p		sets the password to use for any authentications
wssPasswordType	-w		sets the WSS password-type to use for any authentications. Setting this will result in the addition of WS-Security UsernamePassword tokens to any outgoing request containing the specified username and password
projectPassword	-x		sets SoapUI project password
soapUISettingsPassword	-v		sets SoapUI settings password
enableUI	-i	false	enables Swing UI components
testSuite	-s		sets the TestSuite to run. If not set all TestSuites in the specified project file are run
testCase	-c		sets the TestCase to run. If not set all TestCases in the specified project file are run
printReport	-r	true	a flag controlling if a summary should be printed
saveAfterRun	-S	false	saves the project after running the tests

property

arg

default

description

failOnError

N/A

true

sets if gradle plugin execution should stop and fails on any plugin configuration errors occurs

projectFile

N/A

soapui-test-project.xml

sets the SoapUI project file containing the tests to run

outputFolder

-f

build/soapui

sets the output folder to export results to

projectProperties

-P

sets list of "key=value" project properties

globalProperties

-G

sets list of "key=value" global properties

systemProperties

-D

sets list of "key=value" system properties

settingsFile

-t

sets the SoapUI settings file

endpoint

-e

sets the endpoint to use for all test requests

domain

-d

sets the domain to use for any authentications

host

-h

sets the host to use by all test-requests, the existing endpoint port and path will be used

username

-u

sets the username to use for any authentications

password

-p

sets the password to use for any authentications

wssPasswordType

-w

sets the WSS password-type to use for any authentications. Setting this will result in the addition of WS-Security UsernamePassword tokens to any outgoing request containing the specified username and password

projectPassword

-x

sets SoapUI project password

soapUISettingsPassword

-v

sets SoapUI settings password

enableUI

-i

false

enables Swing UI components

testSuite

-s

sets the TestSuite to run. If not set all TestSuites in the specified project file are run

testCase

-c

sets the TestCase to run. If not set all TestCases in the specified project file are run

printReport

-r

true

a flag controlling if a summary should be printed

saveAfterRun

-S

false

saves the project after running the tests

5.1.2. Test runner specific configuration properties

property	arg	default	description
exportAll	-a	true	adds console appender results to groovy log
junitReport	-j	true	сollects TestRun results and creates JUnitReports
junitReportWithProperties	-J	true	include JUnit XML reports adding test
ignoreErrors	-I	false	a flag controlling if errors are ignored
printAlertSiteReport	-M	true	creates a Test Run Log Report in XML format
maxErrors	-m	5	sets the maximum number of TestStep errors to save for each testcase

property

arg

default

description

exportAll

-a

true

adds console appender results to groovy log

junitReport

-j

true

сollects TestRun results and creates JUnitReports

junitReportWithProperties

-J

true

include JUnit XML reports adding test

ignoreErrors

-I

false

a flag controlling if errors are ignored

printAlertSiteReport

-M

true

creates a Test Run Log Report in XML format

maxErrors

-m

sets the maximum number of TestStep errors to save for each testcase

5.1.3. Load test runner specific configuration properties

property	arg	description
loadTest	-l	sets the Load Test to run. If not set all Load Tests in the specified project file are run
limit	-m	override limit property of Load Test
threadCount	-n	override thread count property of Load Test

property

arg

default

description

loadTest

-l

sets the Load Test to run. If not set all Load Tests in the specified project file are run

limit

-m

override limit property of Load Test

threadCount

-n

override thread count property of Load Test

6. Examples

See some usage examples here

7. Frequently asked questions

7.1. soapuios

If you faced with errors like

An error occurred [com.eviware.soapui.plugins.auto.factories.AutoDiscoveryMethodFactory], see error log for details
java.lang.ClassNotFoundException: com.eviware.soapui.plugins.auto.factories.AutoDiscoveryMethodFactory
...

From 5.2 Release Notes (2015-07-02):

SoapUI failed to load plugins when you run tests with testrunner.bat and the current directory differed from <SoapUI>\bin

Fix: rename $HOME/.soapuios folder to NOT.soapuios

mv -f $HOME/.soapuios $HOME/NOT.soapuios

8. Development and contribution

Feel free to contribute or open an issue

build and test

bash gradlew clean build

For testing locally published plugin into maven repo publishing use this example

publish locally for development purpose

bash gradlew install publish

note: Don’t try publish new version of plugin to gradle portal (it’s available only for owner)

publishing plugin

bash gradlew publishPlugins

publishing documentation

bash gradlew deploy

note: Don’t try publish new version of plugin documentation to GitHub pages (it’s available only for owner)

8.1. Supported gradle versions

plugin:
1. 4: 4.2.1, 4.2, 4.1, 4.0.2, 4.0.1, 4.0
client:
1. 4: 4.2.1, 4.2, 4.1, 4.0.2, 4.0.1, 4.0
2. 3: 3.5, 3.4.1, 3.4, 3.3, 3.2, 3.1, 3.0
3. 2 (apply plugin only): 2.14, 2.13, 2.12, 2.11, 2.10, 2.9, 2.8

8.1.1. Latest plugin release version

5.3.0 (plugin: 5.3.0-5)

8.1.2. Current plugin milestone version

5.3.1-RC (plugin: 5.3.1-RC-4)

8.1.3. Support SoapUI old versions

5.2: 5.2.1, 5.2.0
5.1: 5.1.3, 5.1.2-hotfix.2, 5.1.2
5.0: 5.0.1-hotfix.0 (5.0.0-beta), 5.0.1, 5.0.0
4.6: 4.6.4, 4.6.3, 4.6.2, 4.6.1

Note

Strike through version x.y.z cloud be buggy. You have to use next one instead.

Recomendations

We are recommend use latest stable release versions - it should be able support old SoapUI test project file format as well.

9. License

MIT

10. Enjoy

^_^

SoapUI Runner Gradle Plugin (5.3.0-5)

Introduction

1. Why?

1.1. Officially, gradle plugin is not supported

1.2. Running tests using soapui-runner gradle plugin

2. What’s inside?

2.1. Available plugins and tasks

2.1.1. Three plugins available on Gradle Portal

2.1.2. Three tasks available right now

3. Installation

3.1. Multi-project build

3.2. Gradle version less then 3.0

3.2.1. Many ways apply plugins…​

4. Configuration plugins tasks

4.1. Test (suites and cases) runner

4.1.1. Test runner configuration customization

4.2. Load Tests runner

4.3. Generate directory for external libraries

4.4. Adding new and override generic (parent) properties

5. Appendix

5.1. Runners properties

5.1.1. Base (allowed for testrunner and loadtestrunner tasks)

5.1.2. Test runner specific configuration properties

5.1.3. Load test runner specific configuration properties

6. Examples

7. Frequently asked questions

7.1. soapuios

8. Development and contribution

8.1. Supported gradle versions

8.1.1. Latest plugin release version

8.1.2. Current plugin milestone version

8.1.3. Support SoapUI old versions

9. License

10. Enjoy

3.2.1. Many ways apply plugins…