Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision 		Previous revision
sequencers [2020/07/16 16:11]
admin created 		sequencers [2021/02/17 18:16] (current)
admin [Sequencer - MCase]
Line 1: Line 1:
 ====== Model Sequencers ====== ====== Model Sequencers ======
  
-//Model Sequencer// is algorithm-based the test sequence (test cases) generation from the model. +//Model Sequencer// is algorithm-based test case (test sequence) generation from the model. 
  
-The test sequence (test case) generation is referred to as //Generate// action.+The test case generation is referred to as //Generate// action.
  
 The generated test cases can be automated to drive Application Under Test (AUT) through //Groovy-based Scripting// and [[Plugins]].  This step is referred to as //Execute// model action. The generated test cases can be automated to drive Application Under Test (AUT) through //Groovy-based Scripting// and [[Plugins]].  This step is referred to as //Execute// model action.
Line 18: Line 18:
  
 For //Combinatorial Data// models, //Model Sequencer// generates the permutations for the variable value (domain) sets to achieve the specific combination coverage. For //Combinatorial Data// models, //Model Sequencer// generates the permutations for the variable value (domain) sets to achieve the specific combination coverage.
 +
 +Constraints (rules) can be added to //Combinatorial Data// models to remove unwanted combinations.
  
  
Line 26: Line 28:
 The weight of the outgoing transitions determines the probability a transition will be traversed from its source state.  The higher the weight, the more often the transition will be traversed, with a caveat: non-traversed transitions are preferred over already-traversed transitions. The weight of the outgoing transitions determines the probability a transition will be traversed from its source state.  The higher the weight, the more often the transition will be traversed, with a caveat: non-traversed transitions are preferred over already-traversed transitions.
  
-//Random// sequencer is useful for exploratory type of testing by allowing the model to run for an extended period of time to let the algorithm traverse "all possible" paths.+//Random// sequencer is useful for exploratory type of testing by allowing the model to run for an extended period of time to let the algorithm explore "all possible" paths.
  
-//Random// sequencer is also often used for load and stress testing by running model with may //virtual users (VU)// to simulate realistic load of a production systems.+//Random// sequencer is also often used for load and stress testing by running model with many //virtual users (VUs)// to simulate realistic load of a production systems.
  
-Another good use of //Random// sequencer is to find memory leak, locking contention, etc. that are caused by running //AUT// over a long period of time. You can just leave the model running with the elapse time stop condition set to desired duration of time.+Another good use of //Random// sequencer is to find memory leak, locking contention, etc. that are caused by running //AUT// over a long period of time or under heavy load. You can just leave the model running with the elapse time stop condition set to desired duration of time.
  
 The disadvantage of //Random// sequencer is that the model coverage is typically time-dependent - it may take longer time for it to achieve the desired model coverage. The disadvantage of //Random// sequencer is that the model coverage is typically time-dependent - it may take longer time for it to achieve the desired model coverage.
Line 38: Line 40:
  
 =====Sequencer - Optimal===== =====Sequencer - Optimal=====
-//Optimal// sequencer uses uses algorithms for [[https://en.wikipedia.org/wiki/Route_inspection_problem | Route inspection problem (aka Chinese Postman Problem)]] to generate the test cases (paths) that covers all transitions (and thus all states) in the //State-based// models.+//Optimal// sequencer uses the algorithm for [[https://en.wikipedia.org/wiki/Route_inspection_problem | Route inspection problem (aka Chinese Postman Problem)]] to generate the test cases (paths) that covers all transitions (and thus all states) in the //State-based// models.
  
-The obvious advantage of //Optimal// sequencer is that it generates the least number of steps to cover every transitions in the model.  As the result, this sequencer would work better for larger models or if you just want to run all test cases in the least amount of time.+The obvious advantage of //Optimal// sequencer is that it generates the least number of steps to cover all transitions in the model.  As the result, this sequencer would work better for larger models or if you just want to run all test cases in the least amount of time.
  
 The side effect is that the test cases generated by //Optimal// sequencer tends to be longer and fewer test cases with the minimal total number of test steps.  This can be viewed as either an advantage or disadvantage depending on your specific use case. The side effect is that the test cases generated by //Optimal// sequencer tends to be longer and fewer test cases with the minimal total number of test steps.  This can be viewed as either an advantage or disadvantage depending on your specific use case.
Line 49: Line 51:
 //Priority// sequencer prioritizes transitions based on their assigned weight and tries to traversed transitions with higher weight first using graph algorithm to achieve desired model coverage. //Priority// sequencer prioritizes transitions based on their assigned weight and tries to traversed transitions with higher weight first using graph algorithm to achieve desired model coverage.
  
-The test cases generated by //Priority// sequencer usually are shorter than //Optimal// sequencer due to the nature of the algorithm, but the number of test steps to achieve the similar model coverage would be larger.+The test cases generated by //Priority// sequencer usually are shorter than //Optimal// sequencer due to the nature of the algorithm, but the number of test steps to achieve the similar model coverage would be larger compared to //Optimal// sequencer.
  
 //Priority// sequencer might be a good choice if you have certain transitions in the model that you wish to cover earlier and/or prefers shorter test cases. //Priority// sequencer might be a good choice if you have certain transitions in the model that you wish to cover earlier and/or prefers shorter test cases.
 +
 +This sequencer supports following parameters to adjust the sensitivity on choosing higher priority transitions and over traversed transitions:
 +  * TraversedTransCost: integer, default 500
 +  * scale: float >= 0.1, default 1.0 = neutral, < 1.0 for less sensitive (less penalty on choosing lower weight transitions), > 1.0 for more sensitive
 +
 +You may also set above parameters in config.properties for all models:
 +    Sequencer.TraversedTransCost=integer
 +    Sequencer.scale=float
 +
  
  
Line 61: Line 72:
 Often times you may want to only test certain part of the model.  //Mark Optimal// sequencer (and [[#Sequencer - Mark Serial| Mark Serial]]) generates test cases to cover the marked transitions. Often times you may want to only test certain part of the model.  //Mark Optimal// sequencer (and [[#Sequencer - Mark Serial| Mark Serial]]) generates test cases to cover the marked transitions.
  
-//Mark Optimal// sequencer generates the test cases using the same [https://en.wikipedia.org/wiki/Route_inspection_problem | Route inspection problem (aka Chinese Postman Problem)]] as [[#Sequencer - Optimal | Optimal]] sequencer except that only the marked transitions are required to be covered.+//Mark Optimal// sequencer generates the test cases using the same //Postman Problem Algorithm// used by [[#Sequencer - Optimal | Optimal]] sequencer except that only the marked transitions are required to be covered.
  
 //Mark Optimal// sequencer (and [[#Sequencer - Mark Serial| Mark Serial]]) can be used for [[https://en.wikipedia.org/wiki/Exploratory_testing | Exploratory Testing]], e.g. testing a new user stories just added to the model. //Mark Optimal// sequencer (and [[#Sequencer - Mark Serial| Mark Serial]]) can be used for [[https://en.wikipedia.org/wiki/Exploratory_testing | Exploratory Testing]], e.g. testing a new user stories just added to the model.
Line 79: Line 90:
  
 =====Sequencer - MCase===== =====Sequencer - MCase=====
-//MCase// is a collection of transitions that represents a custom test case.  //MCase// sequencer generates the actual test sequence (test steps) to navigate through the model to cover the transitions included in the //MCase//. +//MCase// is a collection of transitions that represents a custom test case.  //MCase// sequencer generates the actual test sequence (test steps) to navigate through the model to cover the transitions included in the //MCase//.  This works very similarly as //Marked Serial// sequencer except that the transition collections are defined in scripts and //MCases// can be executed in IDE as well as remotely.
  
 The main use of //MCase// sequencer is to generate custom test cases, which can be used to test a bug fix, regression testing of important workflows and smoke testing. The main use of //MCase// sequencer is to generate custom test cases, which can be used to test a bug fix, regression testing of important workflows and smoke testing.