Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
statstab [2017/12/22 02:25]
admin [Execution Summary] 		— (current)
Line 1: Line 1:
-=====Stats Tab===== 
-Stats Tab shows MBT execution stats collected by //TestOptimal//.  Both current MBT execution and historical executions can be accessed from this tab. The stats are divided into sections: 
-  * Execution Summary 
-  * Overall Coverage 
-  * Test Coverage 
-  * Requirements Coverage 
- 
-Each stats sections can be collapsed or expanded by clicking on the {{http://testoptimal.com/img/minus.png}} and {{http://testoptimal.com/img/plus.png}} sign. 
- 
-{{http://TestOptimal.com/img/IDE_StatsTab.png?500x800}} 
- 
- 
----- 
-====Execution Summary==== 
-//Execution Summary// shows basic settings and progress status for the current MBT execution.  These information includes: 
-  * Browser - name of the browser you selected in [[mbt node|MBT Property]]. 
-  * Plugin(s) - name of the plugin(s) you selected in [[mbt node|MBT Property]]. 
-  * Start Time - date and time when MBT execution was started. 
-  * Elapse Time - how long MBT execution has been running. 
-  * End Time - date and time when MBT execution completed. 
-  * MBT Mode - MBT sequencer selected in [[mbt node|MBT Property]]. 
-  * Virtual Users - number of virtual users specified in [[mbt node|MBT Property]]. 
- 
----- 
-====Overall Coverage==== 
-//Model Coverage// shows the coverage of the model by the MBT execution in real-time (refresh to get the latest stats from the execution): 
-  * Traversals - number of states or transitions traversed / covered. An up arrow or down arrow indicates this number is greater or less than the average number of state or transition traversals from the historical MBT executions. Hover mouse over the number to show to show the average of states or transitions traversals from the historical MBT executions. 
-  * Covered - number of states or transitions covered. Note repeated traversal on the state or transition is only counted once. An up arrow or down arrow to the right of the number indicates the number is greater or less than the average number from the historical MBT executions. Hover mouse over the number to get the average number of states or transitions from the historical MBT executions. 
-  * Un-Covered - number of states or transitions that have not been covered. 
-  * Coverage - coverage of states and transitions in percentage. 
-  * Coverage graph {{http://testoptimal.com/img/covGraph.png}} - model graph with states/transitions colored based on their coverage: green sufficiently covered, yellow insufficient coverage, red no coverage at all. (Rel 3.0.33) 
-  * MScript Coverage - shows the mscript execution coverage during the model execution. (Rel. 4.2) 
- 
----- 
-====Test Coverage==== 
-//Test Coverage// shows the detailed coverage of the model in state and transition traversals. 
- 
-{{http://TestOptimal.com/img/IDE_StatsTab_Coverage.png?600x900}} 
- 
- 
-  * Exceptions - number of MBTExceptions have occurred. An up arrow or down arrow indicates the number of exceptions is greater or less than the average number of exceptions from the historical MBT executions. 
-  * State: Transition - the dropdown list allows you to filter the list to only state or transition. The list in this section shows the state and transition stats. State is listed with state id and transitions is listed with state id: transition event id. 
-  * Trav - number of script method calls. 
-  * L1 to L5 - number of MBT exceptions that have occured. Click on the number to show the details of these exceptions. The details of the exception includes the tag name and line # the exception was raised in mScript. 
-  * Response Time - shows the transition response time in milliseconds. The blue line in the orange bar shows the response time of this particular state/transition in this execution. Hover mouse over the orange bar to show the 3-sigma range for the response time on this state/transition from the historical MBT executions. 
- 
-Clicking on the blue under columns L1 - L5 will bring up the error window showing the error message as well as the link to the screenshot if a screenshot was taken at the time when the error was detected. 
- 
-{{http://TestOptimal.com/img/IDE_StatsTab_Error.png?400x200}} 
- 
----- 
-==== Requirements Coverage==== 
-//Requirements Coverage// shows the coverage on requirements associated with the states, transitions as well as dynamically registered in script:  
-See [[Tags Statistics|Tags Details]] for more details. 
- 
- 
-If you want to save this stat, click on the button "Save". If will add this stat to the historical executions. 
- 
-When running Concurrent models to orchestrate model executions, you must save the stats first and re-open the stats by selecting it in "Runs" tab. Then you will see a dropdown list displayed listing all models involved in the execution batch. The execution batch is also available from [[Server Manager Edition|SvrMgr]] when model is run against [[Server Manager Edition|SvrMgr]]. 
- 
----- 
-====Current Execution ===== 
-By default, the current MBT execution is displayed. //Stats tab// does not automatically refresh.  You may refresh the stats by clicking on {{http://TestOptimal.com/img/refresh.gif}}. If you are viewing past execution and want to go back to current MBT execution, just select the //Current Execution// from the //Stats Desc// drop-down list. 
- 
-=== Save Execution=== 
-Execution results can be saved for future review or comparison.  To save the current MBT execution, hover mouse over {{http://testoptimal.com/img/more.png}} on the far right of the header and select //Save// menu item. You will be prompted to enter a brief description for the execution. The stat description you've enter is also used as an identifier later to access this execution. We suggest to keep the description short to make it easier to identify and access. 
- 
-By default, the maximum number of executions can be saved is set to 5, but you can adjust this setting to fit your needs with [[modelnode|Model Property]]. When the number of executions exceeds this limit, the oldest execution will be removed automatically to make room for the new executions. 
- 
- 
----- 
-====Past Executions==== 
-Past executions saved are listed in //Stats Desc// drop-down list.  You can access any of the past execution by selecting from the drop-down list.  
- 
-Past executions are used to determine the statistical range (3-sigma range) for each state and transition traversal.  This statistical range gives you 95% confidence level that any traversal of that state or transition must fall within.  
- 
-===Delete Execution=== 
-Past executions can be deleted by selecting //Delete// from the hover menu on the far right on the header. 
-