Configuring Test Run Workflow

Polarion can support formal testing review and sign-off process for Test Runs, using secure electronic signatures compliant with U.S. 21 CFR Part 11. An administrator can map the organization's review and sign-off process to the Test Run Workflow configuration. Each step in the process can optionally be set up to require an electronic signature before a Test Run can move to the next stage in the workflow

Initial Setup for Existing Installations

If you have updated an existing Polarion system from a previous version that did not support workflow for Test Runs (version 2015 or earlier), you will need to enable the feature in Administration before it can be used with existing projects. This section provides details about the procedure.

You should enable Test Run workflow in project configurations first, then in the global scope. This ensures that existing projects do not inherit the global configuration incorrectly. For example, some projects might have Status definitions that are incompatible with the initial global workflow configuration. During the setup process, Polarion creates an initial workflow configuration for all Test Run types in the scope based on the current configuration of Test Run statuses.

Important

Polarion does not support having some Test Run types with workflow, and some types without workflow in the same project. Polarion sets up a default workflow configuration applicable to all Test Run types, which can be overridden by defining workflow for specific types. To avoid problems, do not remove the "All Types" workflow configuration in any scope. You can modify it to be as generic or a broad as you want, and then focus on type-specific configurations.

To enable workflow for Test Runs in a project:

  1. In project Administration's Navigation, expand Testing and select Test Run Statuses (only present if the workflow feature is not already set up).

  2. If you need to modify the Statuses configuration, do that now. You can still make changes after Test Run workflow is enabled. However, the initial setup is based on the current statuses, so it is preferable to fully define all the Test Run statuses the project needs before proceeding.

  3. On the toolbar of the Test Runs Statuses topic, click the Set Up Workflow button.

  4. Follow the messages in the dialog to complete the setup operation.

To enable workflow for Test Runs globally:

  1. Log in with administrator permissions for the Repository scope.

  2. Enter global Administration, and in Navigation expand Testing and select Test Run Statuses (only present if the workflow feature is not already set up).

  3. On the toolbar of the Test Runs Statuses topic, click the Set Up Workflow button.

  4. Follow the messages in the dialog to complete the setup operation.

After the setup is complete, the Test Run Statuses node in Navigation becomes hidden, and is replaced with the Test Run Workflow node, and the Test Run Workflow page opens. On this page you can view and edit the default workflow configuration that Polarion created for all Test Run types, and you can define type-specific workflow for any of the existing Test Run types, which appear listed in the table. If you need to define new types, you can do so.

Alternatives for Advanced Users

You will get a simple all-allowed workflow upon using the Set Up Workflow action described above. You can optionally install the new default from a project template instead, either by picking it up from a new project created from a corresponding template, or by mining it directly from the plugins and then uploading it to your own project's Subversion repository.

Using the Test Run Workflow Designer

If you have previously configured workflow for Work Items or Documents, you already know most of what you need to be able to configure it for Test Runs. As with these other workflows...

  • In the Statuses section you define every status a Test Run in the scope can potentially have during its lifecycle.

  • In the Actions section, you define the actions that users invoke to trigger the transition of a Test Run from one Status to a different Status. This is where you, in essence, map your specific process for running tests into Polarion's workflow control.

    Programmatic workflow functions can be specified and their input parameter values configured. The configured workflow functions are then executed along with the respective transition actions. The execution can be conditional, dependent upon one or more specified workflow Conditions. All conditions must be satisfied in order for the related function to be executed.

  • In the Transitions section, for each Status in the left column you specify to which other Statuses it can transition, and which of the defined Actions triggers each transition.

A brief example can help to grasp the concept. Suppose there are only 3 possible Statuses a Test Run can have in your process: Open, Passed, and Failed. The Actions to transition from one Status to another might be specified as: Mark Open, Mark Passed, and Mark Failed. With these defined, you only need to decide to which other Status(es) a Test Run can transition from any given Status.

A Test Run with the Open status cannot transition to that same status. Can it transition from Open to Passed? It obviously can, so in the cell where Open intersects Passed, you select the Mark Passed action. From Open status, a Test Run should also be able to transition to the Failed status, so you would select the Mark Failed action where Open intersects Failed.

Figure 6.1. Test Run Workflow Transition

Test Run Workflow Transition

Setting workflow transition for a Test Run

The end result for users is that they can only execute valid transitions, and they cannot bypass or skip any process steps. For example, you might decide a Test Run should never directly transition from Failed to Passed, but only from Failed to Open, you would set an action only in the Open column:

Figure 6.2. Transition Limited to a Single Status

Transition Limited to a Single Status

Transition from Failed is limited to the Open status

Users cannot then bypass the Open status and set Passed. There can be some important reasons for this beyond mere policy. Transitions can be configured so that certain things happen when they are invoked: field values are set or cleared, some programmatic function executes (such as prompting for electronic signature), and so on.

Defining Test Run Statuses

Testing managers should analyze their needs and process to determine the statuses a Test Run can have at the various stages of its lifecycle (e.g. "Open", "Planned", "Failed", etc.) Then the administrator can create a Status definition for each one in the Statuses section of the Workflow Designer. Keep in mind that you can have workflow configurations for different Test Run types, which means that different types can have different sets of statuses, if that is what makes sense for the context. For example, Test Runs for quick, experimental testing might have one set of status definitions, while Test Runs for formal regulatory documentation might have a quite different set.

Defining Workflow Actions

It is in the Actions section of the workflow configuration that you exert the greatest control over the workflow. You can:

  • Specify the role(s) that users must be assigned in order to be able to invoke the Action.

  • Require that some field(s) be filled before the Action can be executed by specifying field ID(s) in the Required Fields column.

  • Specify fields whose values is cleared when the Action executes, by specifying the relevant field ID(s) in the Cleared Fields column.

  • Mark an Action as the initial one in the process. Remember that the Initial action, will execute along with its required and cleared fields, workflow conditions, and workflow functions when a new Test Run is created.

  • Require an electronic signature from the user invoking the Action before the Action is executed.

  • Specify one or more workflow Functions to execute when the Action executes - to set a date or run some custom script, for example.

  • Specify one or more workflow Conditions to control whether or not the workflow action can be executed. If any of the conditions is not satisfied, users are informed that the corresponding action as unavailable, and they are blocked from executing it. For example, a Condition might check that a specific field is not empty, and the action is not available to users unless that Condition is satisfied.

For reference information about workflow Conditions and Functions, see the Administration Reference topic Workflow Conditions and Functions (Test Runs).

Workflow and Imported Test Results

After Test Run workflow is set up and configured, when test results generated by external testing tools are imported, the xUnitFileImport job processes the results according to the workflow configuration, including conditions and functions for workflow actions, if workflow is configured for the project and the Test Run type involved.