Test Case Report / Manual Test Case Execution

»Menu: Report / TestCase Report

BasicMBT outputs test case generates into an interactive test case report for manual testing as well as for documentation.

Test Case Report consists of Summary, Requirements, Execution, Progress Chart, Raised Defects, Preamble, Postamble and Test Cases pages. Each page provides features to allow you to interact with and view the execution status of the test cases generated.

With ALM integration (requires ALM plugin), you can also raise the defects to your ALM system directly.

Note what's described in this document are only available when SeqOut plugin is used. For models using other plugins like Selenium plugin, you can still output test cases into text file or html file in your own format when you turn on Output Comment in Model Property dialog to have all MScript comment tags to be written out to the test case report file.

Access Test Case Report

Test Case Report can be accessed from TestOptimal IDE through menu Report / TestCase Report after you have executed the model to generate the test sequence.

Test Case Report UI

When Test Case Report is first open, it displays the Summary page. To access other pages of Test Case Report, hover your mouse over Menu at the upper-left corner of the screen to bring up the navigation pane.

Click on the tabs for different section of the test case report.

KPI field shows the Key Performance Indiciators of the model executions in the current TestOptimal installation. Click on KPI link on the top to open Dashboard.

The icon buttons on the right is the customizable toolbars. Click on the far right icon button to open toolbar customization window.

Tab Summary

Summary tab (shown above) provides a summary of the model execution that produced the test cases.

The model properties, execution parameters and the test coverage graph are displayed.

Click on Open Graph in New Window link to open the test coverage of the model execution. The covered transitions are colored in green while un-covered transitions are colored in red and partially-covered transitions are colored in yellow.

Tab Requirements

Requirements tab displays the requirement coverage and traceability matrix that trace each requirement to all test cases that cover it.

The requirements are listed in columns and test cases are listed in rows.

By default requirements are identified by a sequential number but you can show the requirement name/code by clicking on show tag name link in the table header.

The cells shaded (light blue colored) indicates that particular requirement is covered by the test case displayed in the row. So at a glance you can clearly see if any requirements have not been covered. Similarly you can also see what requirements are covered by each test case.

As the test cases are executed and status recorded in Execution page, the test case status (pass, fail, blocked) are marked with a vertical color bar next to the test case.

Clicking on the test case number will take you to Test Case page for that test case.

The column Weight describes the sum of all weights you have assigned to the transitions involved in the test case. This may be used to identify high priority test cases if Weight assigned to the transitions represent some sort of priority/importance.

The column Sz shows the number of transitions /steps in the test case or length of the test case.

Tab Preamble

Displays the test setup instructions the model generated.

Tab Test Cases

Test Case tab contains a series of individual test case pages each shows the details of a test case.

Click on the individual Test Case on the left to display the test case details on the right.

Each test case when executed or partially executed will show a vertical color bar to the right to indicates the status of the test case.

A test case is passed if all of its steps are passed. If some of the steps are passed and the rest of them are not executed yet, the test case will have partial status.

A test case is failed if any of the steps are failed.

A test case is blocked if at least one step is marked blocked and the rest of the steps are either passed or not executed yet.

The link at the top center of the page takes you to the Test Case Graph representing the test case steps in a graph or Message Sequence Chart (MSC) (depending on how you configure your model).

A test case consists of a series of steps and each step may contain many verification items.

Test steps are listed in the table with instruction displayed in the left column and the verification instruction on the right.

You can mark verification item within each step by clicking on the status buttons (passed, failed, and blocked). Optionally you may enter an explanation in the text area below it.

The test step status and test case status are derived from the status of all verification items and is updated automatically in realtime as you update the status of verification items.

Tab Postamble

Displays the test teardown instruction the model generated.

Tab Exec Status

Exec Status tab summarizes the test case execution status as test cases are executed and status recorded in Test Case pages.

Test Case Matrix section contains the test case execution progress while Requirement Matrix section contains the requirement coverage status from the test case execution.

Clicking on the test case number takes you to the specific Test Case page.

Progress Progress

Progress tab contains various charts to describe the test cases and their execution status.

The charts are automatically updates as test cases are executed and status are recorded with Test Case pages.

Use the control buttons above each chart to filter the chart by test case status or to change the chart type.

Tab Defects

If ALM is enabled, you will see a list of known defects raised so far by the current model from previous test execution.