9. Configuring Reports

Working with Report Configurations

There are two basic types of reports: Source code reports which appear in the Reports topic of the portal, and Audits/Metrics reports which appear in various dashboards. All reports are customizable in the Administration interface by means of XML configuration files. This section explains the basics of how to configure the different types of reports.

Source Code Reports

You can configure source code reports for each project. If you are not familiar with the basics of the different scopes, you may want to review Administration Basics: The Administration Interface.

You can configure which reports appear in the Reports topic, specifying the calculation for each report. You can also customize the calculation of each report. The default reports are:

  • Maven Site

  • Unit Test Coverage

  • Unit Tests

  • Javadoc

  • XRef

Two additional reports are available, but are commented out in the default reports configuration file:

  • PMD

  • Checkstyle

There are 2 files involved in this configuration:

  • reports-config.xml : defines the list of source code reports shown in the Reports topic of the portal.

  • descriptors.xml : configures the calculations used by the reports defined in reports-config.xml.

Accessing the Configuration files

To access the configuration files for source code reports:

  1. Log in with administrative rights for the project you want to configure.

  2. Enter the Administration interface.

  3. Navigate to and select the project in the Open Project or Project Group dialog (see User Guide: Accessing Projects).

  4. In the Navigation pane, expand the Reports topic.

  5. To access the reports-config.xml file, click on the Source Code Reports topic.

    To access the descriptors.xml file, click on the Report Descriptors topic.

  6. Use the appropriate link in the Configuration section of the respective configuration page to download the configuration file to your local system, or edit online with the text editor provided.

  7. If you edit the file offline, use the controls in the Upload New Project Configuration section to upload the modified file back to the Polarion repository.

You will find comments in the configuration file that explain the various elements and attributes of the configuration.

Note that source code reports can be computed only if the build artifact (containing Java sources) is a maven2 build artifact, and the build process is running without errors.

Work Item Metrics

Work Item metrics are Work Item-related values (facts) that are calculated by Polarion on projects and project groups. There are two related calculations: trackeranalysis, which calculates metrics on projects, and trackeranalysis-projectgroup, which calculates metrics on project groups.

Value of any Work Item metric is one of the following types:

  • integer (int): non-negative (0, 1, 2, ...)

  • distribution: i.e., a set of integer sub-values that is rendered as a bar with colored stripes, where width of each stripe is proportional to the integer sub-value. Each sub-value has an ID (called subkey) so that it can be referenced e.g. on a dashboard, and a label that is displayed as tool-tip when hovering above the respective stripe in the bar.

  • percentage: e.g 70%, rendered as a colored bar in a way similar to distribution.

  • items-traceability: defines a notion of traceability of Work Items; the value is a distribution of traceable and not-traceable items that satisfy 'itemsQuery', where traceable items are those, from which one can follow the specified links ('link' elements) in specified directions and find at least one item that satisfies the 'coverageQuery'; items that satisfy both 'itemsQuery' and 'coverageQuery' are automatically considered traceable without checking links.

  • commits-traceability: defines a notion of traceability of repository commits; the value is a distribution of traceable and not-traceable commits, where traceable commits are those, that are linked to at least one item from which one can follow the specified links ('link' elements) in specified directions and find at least one item that satisfies the 'coverageQuery'

  • field-sum and field-average: calculates the sum or average of the values stored in a custom field.

You can configure Work Item metrics for projects. If you are not familiar with the basics of the different scopes, you may want to review Administration Basics: The Administration Interface.

This configuration contains elements that define values that may be shown on a dashboard, or used in the definition of a trend chart.

Accessing the Configuration files

Work Item metrics are configured in the workitem-metrics.xml file. To access this file:

  1. Log in with administrative rights for the project you want to configure.

  2. Enter the Administration interface.

  3. Open the project in the Open Project or Project Group dialog (see User Guide: Accessing Projects).

  4. In the Navigation pane, expand the Reports topic.

  5. Click on the Work Item Metrics topic.

  6. Use the appropriate link in the Configuration section to download the configuration file to your local system, or edit online with the text editor provided.

  7. If you edit the file offline, use the controls in the Upload New Project Configuration section to upload the modified file back to the Polarion repository.

The configuration file contains comments that explain the elements, their use, and options for customization.

Work Item Metrics Configuration Syntax

The workitem-metrics.xml defines which Work Item metrics will be calculated on projects during the trackeranalysis calculation. The basic XML syntax and elements are shown in the following code example:

<workitem-metrics>
    <metrics>
        
       <entry>
          <key>KEY</key>
          <label>LABEL</label>
          <description>DESCRIPTION</description>
          <value type="TYPE" ...>
             ...
          </value>
       </entry>
       
       ...
       
    </metrics>
</workitem-metrics>
                        
                    

Where...

  • KEY is a unique identifier for the metric, to be used when referencing the value (e.g. in a dashboard). Required.

  • LABEL is a text label for the metric. Optional (only for in-place defined metrics).

  • DESCRIPTION is a longer text description for the metric. Optional (only for in-place defined metrics).

  • TYPE is the type of the metric (integer, distribution, percentage). Required (only for in place defined metrics).

Internally Defined Work Item Metrics

By default the configuration contains a number of internally defined metrics. For example:

<entry>
    <key>UNRESOLVED-BY-TIMEPOINT</key>
</entry>
<entry>
    <key>MAI</key>
</entry>
                    

You can optionally remove any of these elements. If removed, the internally defined metric will not be calculated during the trackeranalysis calculation. For the complete list of the internally defined Work Item metrics, see Administration Reference: Internally Defined Work Item Metrics.

In-place Defined Metrics

The way the value is calculated is described by the value subelement. There are four types of in place defined metrics:

  • int

  • distribution

  • items-traceability

  • commits-traceability

int

Integer value defined as the size of the search result of a Work Item query. Example:

<entry>
    <key>OPEN</key>
    <label>Open</label>
    <value type="int">
        <query>resolution:###### NULL</query>
    </value>
</entry>
                        

In this example the value of the metric OPEN will be the number of Work Items with no resolution value set.

distribution

There are 2 ways a distribution value can be defined: using an enumeration, and using queries.

Definition using an enumeration

Syntax:

<value type="distribution" enumeration="..." query="..." reverseOrder="..."/>
                        

Where...

  • enumeration is the specification of an enumeration (required), which can be in one of the following formats:

    • enum:ENUM_ID - all options from an enumeration with the specified ID, including type-specific ones.

    • WI_TYPE:enum:ENUM_ID - those options from an enumeration with the specified ID, that are valid for the specified Work Item type.

    • FIELD: (deprecated) - ID of a single-valued Work Item field of an enumeration type. Can be custom, but must not be Work Item type-specific.

  • query is a Work Item query in which substrings "$option$" will be replaced by the ID of an option from the enumeration.

  • reverseOrder - if set to true, the values in the distribution will be sorted in reverse order with respect to the sort order of the enumeration options. (Optional)

  • unresolvedOnly (deprecated) - if set to true, only unresolved items will be counted. (Optional)

  • resolvedOnly (deprecated) - if set to true, only resolved items will be counted. (Optional)

This method uses options from an enumeration type, severity, priority, resolution, or any custom enumeration). Each integer value that is part of the distribution corresponds to one option from the corresponding enumeration, and is equal to the number of Work Items matching a query created using the option ID in a way that depends on attributes of the value element: if there is enumeration id specified, query from the query attribute will be used with $option$ replaced by the option ID. If field ID is specified, a query matching items having the option set in that field will be used (the query will possibly be extended based on the remaining attributes).

The order of values in the distribution is defined by the sort order of the options in the enumeration. ID and label of sub-values is ID and label of the corresponding enumeration option.

Example:

<entry>
    <key>DEFECTS-BY-RISK</key>
    <label>Defects by Risk</label>
    <value type="distribution" enumeration="defect:enum:risk" query="type:defect AND risk:$option$" reverseOrder="true"/>
</entry>
<entry>
    <key>REQ-BY-RELEASE</key>
    <label>Requirements By Release</label>
    <value type="distribution" enumeration="enum:release" query="type:requirement AND targetRelease:$option$"/>
</entry>
                        

Definition using queries

Syntax:

<value type="distribution">
    <query id="..." label="...">...</query>
    ...
</value>
                        

Where...

  • query - the content of the element is a Work Item query. (Required)

  • id is the ID of the sub-value of the distribution. (Required)

  • label is the label of the sub-value of the distribution. (Required)

This method uses sub-elements query of the value element. Each such sub-element corresponds to one value in the distribution, and the value is defined as size of its search result.

Example:

<entry>
    <key>RESOLVED-RATIO</key> 
    <label>Resolved vs. Unresolved</label> 
    <description>Rate of Work Items that have resolution set.</description>
    <value type="distribution">
        <query id="resolved" label="Resolved">NOT resolution:###### NULL</query>
        <query id="not-resolved" label="Unresolved">resolution:###### NULL</query>
    </value>
</entry>
                        

items-traceability

Syntax:

<value type="items-traceability">
    <itemsQuery>...</itemsQuery>
    <links>
        <link id="..." backLink="..."/>
        ...
    </links> 
    <coverageQuery>...</coverageQuery>
</value>
                        

Where...

  • itemsQuery is a query that defines the set of Work Items, starting from the path of which, and consisting of Work Item links, will be searched for.

  • links defines which links can be part of the path.

  • link/id is the ID of the link role (or a regular expression matching link role IDs) that can be part of the path.

  • link/backLink if set to true, only incoming link of the matching ID can be in the path (otherwise only outgoing).

  • coverageQuery is a query that defines a set of items whose path is searched for. (You can use the special value has-linked-revisions that matches items having at least one linked revision.)

Each metric of this type defines a notion of traceability of Work Items. The value is a distribution with two integer sub-values: count of traceable and not-traceable items among those that satisfy itemsQuery. Traceable items are those, from which one can find a path consisting of links matching some conditions (defined by link elements) that leads to at least one item that satisfies the coverageQuery. Items that satisfy both itemsQuery and coverageQuery are automatically considered traceable without checking links. (Empty path is always acceptable).

Example:

<entry>
    <key>TRACEABILITY-REQUIREMENTS-COMMITS</key>
    <label>Traceability Work Items to Commits</label>
    <description>
        Rate of Work Items that are linked to commits.
    </description>  
    <value type="items-traceability">
        <itemsQuery>HAS_VALUE:resolution</itemsQuery>
        <links>
            <link id=".*" backLink="true"/>
            <link id=".*" backLink="false"/>
        </links> 
        <coverageQuery>has-linked-revisions</coverageQuery>
    </value>
</entry>
                        

commits-traceability

This is very similar to items-traceability. The only difference is that there is no element itemsQuery; - the path is searched from all Work Items that are linked to at least one revision (commit) in the project. The sub-values in the distribution correspond to the count of project revisions that are linked to traceable Work Items, and to the count of project revisions that are not linked to a traceable Work Item.

Example:

<entry>
    <key>TRACEABILITY-COMMITS-REQUIREMENTS</key>
    <label>Traceability commits to Work Items</label>
    <description>
        Rate of Commits that are linked to Work Items. 
    </description>
    <value type="commits-traceability">
        <links>
            <link id=".*" backLink="false"/>
            <link id=".*" backLink="true"/>
        </links> 
        <coverageQuery></coverageQuery>
    </value>
</entry>
                        

field-sum and field-average

Calculates the sum or average of values stored in a custom field. (Configured as the 'field' attribute of Work Items returned by a query that is specified as the 'query' attribute.) Calculations are only possible for number (integer, float, currency) type fields.

Example:

<value type="field-average" query="type:testcase" field="passRatio"/>

Work Item Quality Report

You can configure the Work Item Quality report for each project. If you are not familiar with the basics of the different scopes, you may want to review Administration Basics: The Administration Interface.

This configuration enables you to define which of the available metrics should appear in the dashboard.

Accessing the Configuration files

Work Item metrics are configured in the workitem-quality-report.xml file. To access this file:

  1. Log in with administrative rights for the project you want to configure.

  2. Enter the Administration interface.

  3. Navigate to and select the project in the Open Project or Project Group dialog (see User Guide: Accessing Projects).

  4. In the Navigation pane, expand the Reports topic.

  5. Click on the Work Item Quality Report topic.

  6. Use the appropriate link in the Configuration section to download the configuration file to your local system, or edit online with the text editor provided.

  7. If you edit the file offline, use the controls in the Upload New Project Configuration section to upload the modified file back to the Polarion repository.

Process Quality Report

You can configure the Process Quality report for each project. If you are not familiar with the basics of the different scopes, you may want to review Administration Basics: The Administration Interface.

Accessing the Configuration files

Process audit rules for the process quality report are configured in the rule-settings.xml file. The file is commented to help you understand the elements and options. To access this file:

  1. Log in with administrative rights for the project you want to configure.

  2. Enter the Administration interface.

  3. Navigate to and select the project in the Open Project or Project Group dialog (see User Guide: Accessing Projects).

  4. In the Navigation pane, expand the Reports topic.

  5. Click on the Process Quality Report topic.

  6. Use the appropriate link in the Configuration section to download the configuration file to your local system, or edit online with the text editor provided.

  7. If you edit the file offline, use the controls in the Upload New Project Configuration section to upload the modified file back to the Polarion repository.