Test Management Reference

This section contains reference topics related to Test Case Management features.

Permissions and License

Your ability to access and execute Test Runs depends on the permissions assigned to your user account, which are normally tied to the user role(s) assigned to you by the system or project administrator. The license you are currently using may also affect your ability to use some test management features.

You must have the following permissions in order to execute manual Test Runs:

  • Permission to EXECUTE - You must have this permission for Test Runs in order to be able to execute a manual Test Run.

  • Permissions for Work Items - Because Test Runs can read, create and modify Work Items, you must have the following permissions for Work Items in addition to the EXECUTE permission for Test Runs:

    • Permission to READ

    • Permission to MODIFY

    • Permission to CREATE NEW

Test Step Attachments

When manually executing a Test Run, you can optionally add a file attachment to each individual test step, or to a test record. For example, you might attach a screenshot image illustrating the result of a failed test step. This section describes several important points you need to know about these attachments.

  • Click Add Attachment to attach a reference file.

  • Hover over an already attached file and click to remove it.

  • You can attach a file to a test step or a test record, or remove an attachment from these, while text execution is in progress.

  • Users can download attachments from Test Steps or the Test Records section of Test Case type Work Items in the Work Item Tracker.

  • Users can download files attached to a test step result or a test record if the test record is visible on the page of a Test Run.

  • ​Users can download files attached to a test record if the test record is visible in the Test Run Records view.

  • Attachments are displayed in any Defect items created as a result of failed test steps.

  • Attachments are not exported to, or imported from Excel during round-trip operations.

xUnitFileImport Job Parameters

This section documents the parameters of the xUnitFileImport job which enables importing results of externally executed tests into Polarion, creating a Test Run in the process. Parameters are optional unless otherwise specified. Parameters for importing test results during Maven builds are also documented, and an example of the job configuration is also provided.

Table 2.19. Parameters

NameDescription

path

Required. Path to xUnit file, or to a folder containing xUnit files. (All XML files are imported and then deleted.)

project

Required. ID of the project in which the job will create test case and defect Work Items as output of the job.

idPrefix

Prefix for the ID (composed of prefix+timestamp) of Test Run created as output of the job.

userAccountVaultKey

Reference to user authentication key contained in the User Account Vault in Administration > User Management > User Account Vault.

maxCreatedDefects

The number of failed tests before a summary Defect type Work Item will be created instead of individual Defect items for every failed test. Default: 20.

maxCreatedDefectsPercent

The percentage of tests that can fail before a summary Defect type Work Item will be created instead of individual Defect items for every failed test. Default: 10%.

templateTestRunId

ID of the template Test Run on which the Test Run created by the job is based. The template Test Run must exist in the same project specified in project.

templateTestCaseId

ID of template Test Case type Work Item used as basis for Test Case items created by the job. The template Test Case must exist in the same project specified in project.

templateDefectId

ID of template Defect type Work Item used as basis for Defect items created by the job. The template Defect must exist in the same project specified in project.

idRegex

Fills ID of the created Test Run with regex matching the xUnit filename.

groupIdRegex

Fills group.id of the created Test Run with regex matching the xUnit filename.

field[N]

Replace [N] with a number between 1 and 3. Specify up to 3 fields that can be additionally filled with regex matching xUnit filename.

fieldNRegex

Replace [N] with a number between 1 and 3. When field[N] is specified, use this parameter to fill Test Run field[N] with regex matching xUnit filename.

See also: User Guide: Monitor Topic and Administrator's Guide: Configuring the Scheduler.

Parameters for Import in Maven Builds

Test results may be imported to a created Test Run during Maven builds by configuration in .polarion/builder/build.properties. The parameters are identical to the parameters for the xUnitFileImport job, except that all must be prefixed with polarion.build.xUnit.import. Unlike the xUnitFileImport job, however, xUnit files are not deleted after import.

The following parameters must also be specified in the Maven build configuration.

Table 2.20. Additional Maven Build Parameters

ParameterDescription

polarion.build.xUnit.import.enabled

Set to "true" to enable import of test results during the Maven build process.

polarion.build.xUnit.import.path=custom/path

When import is enabled, specified the relative path to artifactId\target\surefire-reports in the job's working directory.

polarion.build.xUnit.import.singleTestRun

Set to "true" to import all xUnit files to a single Test Run. Set "false" to import different xUnit files to separate Test Runs.

Job Example

<job id="xUnitFileImport" name="Import Stress Tests Results" scope="system">
    <path>C:\Temp\xUnit</path>
    <project>PolarionSVN</project>
    <idPrefix>UT</idPrefix>
    <userAccountVaultKey>xUnitFileImportUser</userAccountVaultKey>
    <maxCreatedDefects>10</maxCreatedDefects>
    <maxCreatedDefectsPercent>5</maxCreatedDefectsPercent>
    <templateTestRunId>20110718-151106</templateTestRunId>
    <templateTestCaseId>WI-12</templateTestCaseId>
    <templateDefectId>WI-5</templateDefectId>
    <idRegex>(.*).xml</idRegex>
    <groupIdRegex>[^-]*-(.*).xml</groupIdRegex>
    <typeRegex>([^-]*)-.*.xml</typeRegex>
    <field1>fieldId</field1>
    <field1Regex>[^-]*-(.*).xml</field1Regex>
</job>
                

xUnit File Example

Here is an example of an xUnit file showing only the tags and attributes read by Polarion. There follows a screenshot of the result of importing the example file.

<?xml version="1.0" encoding="UTF-8"?>
<!-- This example shows only tags, attributes and content which is currently read by Polarion
     See http://windyroad.org/dl/Open%20Source/JUnit.xsd -->

<testsuites>
    <!--Name of the root tag does not matter, but it must not be same as the ones below -->
    
    <testsuite timestamp="2011-10-17T23:05:06">
        <!-- testsuite tags can be nested, timestamp is not required and format is "yyyy-MM-dd'T'HH:mm:ssZ" -->
        <testsuite>
            <testcase name="someMethod" classname="SomeClass" time="0.285"/>
            <!-- test case id is "SomeClass.someMethod", time is not required and is in seconds -->
        </testsuite>
        
        <testcase name="otherMethod" classname="SomeClass" time="0.001">
            <failure message="failure message" type="package.Exception">
                <!-- message and type are not required, all text content is added to the created defect -->
                Failure details
            </failure>
        </testcase>
        
        <testcase name="otherMethod" classname="OtherClass" time="0.002">
            <error message="error message" type="package.OtherException">
                Error Details
            </error>
        </testcase>
    </testsuite>
</testsuites>
            

Figure 2.19. xUnitImport Result

xUnitImport Result

Result of importing the above xUnit XML file into Polarion

Test Records Query Extender

Polarion has an embedded SQL database layer that can facilitate Work Item queries in traceability reports and other complex querying needs. One common need is some query function that allows you to search for Work Items (typically test cases, or defects) according to test records... finding "failed test cases executed by me in last week", or "find Defects triggered by Test Cases",for example.

The TEST_RECORDS keyword, with parameters, can be used in the Work Items table for such kinds of queries. It bypasses the usual Lucene query engine and queries the index of the SQL database layer.

SYNTAX: TEST_RECORDS:(testrunId, result, executedBy, executed)

PARAMETERS:

  • testrunId - the ID of a Test Run, qualified with the ID of a project. For example: PRJ1/TR-200

  • result - an option from the Result enumeration (configured in Administration > Testing). Accepts wildcard '*' (meaning 'any').

  • executedBy - the ID of a user who executed the Test Run. Accepts wildcard '*' (meaning 'any').

  • executed - time frame of Test Run execution. Can be a single date or date range. Accepts wildcard '*' (meaning 'any').

Examples:

TEST_RECORDS:(proj1/testRun1)

TEST_RECORDS:(proj1/testRun1, "good result, but not enough")

TEST_RECORDS:(proj1/testRun1, failed,*, 20120130)

TEST_RECORDS:("proj1/testRun1", failed,johndoe, 20120101 TO 20120201)
                    
                

Additional Examples

The following table shows several examples for the TEST_RECORDS element.

Table 2.21. TEST_RECORDS Examples

ExampleNotes

TEST_RECORDS:("myProject/Smoke21","failed")

Finds all failed test cases in Test Run Smoke21

TEST_RECORDS:("myProject/Smoke22","failed",*,*,true)

Finds all Defects linked from failed test records for Test Run Smoke22

TEST_RECORDS:("myProject/Smoke23",*,"johnd","20120702 TO 20120703")

Finds all test cases of Test Run Smoke23 that were executed by user johnd on 2nd or 3rd July, 2012

Note that the TEST_RECORDS element may be used as a sub-query inside another query. For example:

TEST_RECORDS:("myProject/FullTest15","failed") AND severity:smoke AND (assignee.id:johnd OR assignee.id:janeb)