ReadyAPI Plugin for TestRail

SmartBear ReadyAPI (opens in a new tab) is one of the most popular commerical applications for testing API interfaces. Railfow ReadyAPI plugin is a native plugin that seamlessly integrates into ReadyAPI using ReadyAPI extensions framework and exposes context menus and workflows for integrating ReadyAPI with TestRail.

The ReadyAPI Plugin provides the following capabilities

• Project : Import and Export Projects
• Tests: Import and Export ReadyAPI tests to TestRail
• Test Mapping: Map existing TestRail tests to ReadyAPI tests
• Custom Fields : Configure TestRail custom fields
• Automation : Automatically create Tests, Milestone, Plans, Runs
• CICD Support : Run ReadyAPI and Railflow via ReadyAPI cmdline

Installation

Download (opens in a new tab) the latest version of the ReadyAPI plugin and use the installer's easy to follow instructions.

License

To update/configure the Railflow License file, copy the license file to ~/railflow folder (e.g. C:\Users\user\railflow on Windows or /home/user/railflow on Unix) on the machine where Railflow plugin is installed.

Project Configuration

Railflow plugin configurations are stored in ReadyAPI project as project level fields. This allows users to configure each ReadyAPI independently of other ReadyAPI projects.

Initialize

To start using Railflow plugin, first initialize the ReadyAPI project. This will inject TestRail specific properties at the Project, Suite, and Test Levels.

Custom Fields

TestRail users often have countless custom fields for Tests and Results entities to capture additional data. Railflow ReadyAPI plugin provides a wizard for configuring these custom fields.

To configure custom fields, just right-click on the project and select Railflow: Configure Custom Fields

The wizard contains two tabs: Test Case and Test Result for defining custom fields and their values.

The wizard has the following columns:

• TestRail Field Label - the label of the field in TestRail
• ReadyAPI Field - the name of the test case field which contains value for the corresponding TestRail field
• Default value - default value

Upon Save, Railflow Plugin stores the field names and values in the ReadyAPI Test Case Properies tab. These values can be adjusted in eact test case as needed for your workflow and custom field requirements.

Properties Reference

These properties allow you to specify TestRail connection details and control the test and results export behavior.

The following custom properties can be overridden on Test Suite level:

• TR_section_path
• TR_plan_name
• TR_run_name
• TR_milestone
• TR_case_type
• TR_case_priority
• TR_template_name

Project Export

In order to record testcase results, the ReadyAPI project should first be exported to TestRail. Export action creates Tests in TestRail and maps them to ReadyAPI tests.

To export ReadyAPI tests, right-click the Project's on the Test, Suite, or Project level and select Railflow: Export to TestRail menu. Railflow will export ReadyAPI tests AND test steps.

Exported test cases in TestRail:

Results Export

Railflow exports test results into TestRail automatically at the end of a test, suite, or project run.

Railflow creates a new runs in TestRail or adds runs to specified TR_run_name value.

Railflow exports both test, test steps, attaches the request and response of HTTP/SOAP/JDBC steps, and their statuses. Request/Response attachments can be disabled by TR_upload_attachments project field.

CICD Integration

Smartbear ReadyAPI allows two ways of running ReadyAPI tests from CI applications such as Jenkins, TeamCity, Gitlab, etc. Railflow can be configured at runtime to download the Railflow JAR and license for both of these options

• ReadyAPI TestRunner (opens in a new tab)
• SmartBear TestEngine (opens in a new tab)

ReadyAPI TestRunner

TestRunner is a shell/batch script which starts ReadyAPI and loads some project.

• Download Railflow JAR (opens in a new tab) using wget or something similar.
• COPY Railflow JAR into ~/.soapui/plugins folder (Example: C:\Users\user\.soapui\plugins on Windows or /home/user/.soapui/plugins on Unix) on the CI machine or in your Docker image
• Create a new ~/railflow directory
• COPY Railflow license file into ~/railflow directory

TestEngine Server

TestEngine server runs on the CI agent. To configure TestEngine,

• Download Railflow JAR (opens in a new tab) using wget or something similar.
• COPY Railflow JAR into ~/.soapui/plugins folder (Example: C:\Users\user\.soapui\plugins on Windows or /home/user/.soapui/plugins on Unix) on the CI machine or in your Docker image
• Create a new ~/railflow directory
• COPY Railflow license file into ~/railflow directory
• Restart TestEngine server

TestEngine Docker Container

To install Railflow plugin to TestEngine docker container, you need to put Railflow plugin JAR file and Railflow license into TestEngine container, to do so please follow the steps below:

• Download Railflow JAR (opens in a new tab) using wget or something similar.
• COPY/MOUNT Railflow JAR into /data/.soapui/plugins folder of TestEngine container
• COPY/MOUNT Railflow license file into /data/railflow directory of TestEngine container
• Restart TestEngine container if you copied files into running container

Runtime Configuration

Config Overide

All Railflow configurations can be passed to the ReadyAPI TestRunner CLI (opens in a new tab)

testrunner.sh -P TR_url=https://myserver.testrail.io readyapi-project.xml
 

Project Export

In some teams, the most accurate state of ReadyAPI may be in GIT. For such teams, it may be more appropriate to export ReadyAPI tests during the CI build. To export ReadyAPI tests within CI, pass the TR_update_cases option to ReadyAPI testrunner

testrunner.sh -P "TR_update_cases=enabled" readyapi-project.xml