Differences

This shows you the differences between two versions of the page.

--- tutorial:sequencers [2020/07/15 03:55]
admin [Sequencer - Mark Serial]
+++ — (current)
@@ Line 1: / Line 1: @@
-====== Tutorial: Model Sequencers - pros and cons ======
-Learning Objectives:
-  * [[#What's Model Sequencer?]]
-  * [[#Types of Sequencers]]
-  * [[#Sequencer - Random]]
-  * [[#Sequencer - Optimal]]
-  * [[#Sequencer - Priority Path]]
-  * [[#Sequencer - Pairwise]]
-  * [[#Sequencer - Mark Optimal]]
-  * [[#Sequencer - Mark Serial]]
-  * [[#Sequencer - MCase]]
-  * [[#Dynamic MCase]]
-  * [[#Sequencer - Concurrent]]
-  * [[#Combinatorial Algorithms]]
-===== What's Model Sequencer? =====
-//Model Sequencer// is algorithm-based the test sequence (test cases) generation from the model.
-The test sequence (test case) generation is referred to as //Generate// action.
-The generated test cases can be automated to drive Application Under Test (AUT) through //Groovy-based Scripting// and [[Plugins]].  This step is referred to as //Execute// model action.
-=====Types of Sequencers=====
-The algorithms used by the sequencers differ depend on the type of models: //State-based// and //Combinatorial Data//.
-For //State-based// models, //Model Sequencer// generates the test sequence by traversing the state diagram using graph-based algorithms.  A graph path from model's initial state to any of the final state is considered a test case.
-Different algorithms achieve different coverage on the model. In combination with //stop conditions//, you can stop the test generation when the desired coverage is achieved.
-For //Combinatorial Data// models, //Model Sequencer// generates the permutations for the variable value (domain) sets to achieve the specific combination coverage.
-=====Sequencer - Random=====
-//Random// sequencer traverses //State-based// models using the variation from the [[https://en.wikipedia.org/wiki/Markov_chain| markov-chain process]] to construct test paths from //iniital// state to //final// states.
-The weight of the outgoing transitions determines the probability a transition will be traversed from its source state.  The higher the weight, the more often the transition will be traversed, with a caveat: non-traversed transitions are preferred over already-traversed transitions.
-//Random// sequencer is useful for exploratory type of testing by allowing the model to run for an extended period of time to let the algorithm traverse "all possible" paths.
-//Random// sequencer is also often used for load and stress testing by running model with may //virtual users (VU)// to simulate realistic load of a production systems.
-Another good use of //Random// sequencer is to find memory leak, locking contention, etc. that are caused by running //AUT// over a long period of time. You can just leave the model running with the elapse time stop condition set to desired duration of time.
-The disadvantage of //Random// sequencer is that the model coverage is typically time-dependent - it may take longer time for it to achieve the desired model coverage.
-=====Sequencer - Optimal=====
-//Optimal// sequencer uses uses algorithms for [[https://en.wikipedia.org/wiki/Route_inspection_problem | Route inspection problem (aka Chinese Postman Problem)]] to generate the test cases (paths) that covers all transitions (and thus all states) in the //State-based// models.
-The obvious advantage of //Optimal// sequencer is that it generates the least number of steps to cover every transitions in the model.  As the result, this sequencer would work better for larger models or if you just want to run all test cases in the least amount of time.
-The side effect is that the test cases generated by //Optimal// sequencer tends to be longer and fewer test cases with the minimal total number of test steps.  This can be viewed as either an advantage or disadvantage depending on your specific use case.
-=====Sequencer - Priority Path=====
-//Priority// sequencer prioritizes transitions based on their assigned weight and tries to traversed transitions with higher weight first using graph algorithm to achieve desired model coverage.
-The test cases generated by //Priority// sequencer usually are shorter than //Optimal// sequencer due to the nature of the algorithm, but the number of test steps to achieve the similar model coverage would be larger.
-//Priority// sequencer might be a good choice if you have certain transitions in the model that you wish to cover earlier and/or prefers shorter test cases.
-=====Sequencer - Pairwise=====
-//Pairwise// sequencer explores combinations of incoming and outgoing transitions of each state in the model and generate test paths that cover all incoming - outgoing transition pairs.
-//Pairwise// sequencer offers more extensive model coverage than most of the sequencers ([[#Sequencer - Optimal | Optimal]] and [[#Sequencer - Priority | Priority]], and others) with a deterministic test sequence.
-=====Sequencer - Mark Optimal=====
-Often times you may want to only test certain part of the model.  //Mark Optimal// sequencer (and [[#Sequencer - Mark Serial| Mark Serial]]) generates test cases to cover the marked transitions.
-//Mark Optimal// sequencer generates the test cases using the same [https://en.wikipedia.org/wiki/Route_inspection_problem | Route inspection problem (aka Chinese Postman Problem)]] as [[#Sequencer - Optimal | Optimal]] sequencer except that only the marked transitions are required to be covered.
-//Mark Optimal// sequencer (and [[#Sequencer - Mark Serial| Mark Serial]]) can be used for [[https://en.wikipedia.org/wiki/Exploratory_testing | Exploratory Testing]], e.g. testing a new user stories just added to the model.
-//Mark Optimal// sequencer is only available when running model in IDE.
-=====Sequencer - Mark Serial=====
-//Mark Serial// sequencer is similar to [[#Sequencer - Mark Optimal]], it generates test cases to cover the marked transitions in the model.
-What distinguishes //Mark Serial// sequencer from //Mark Optimal// sequencer is that the test cases generates follows the order the transitions are marked.  As the result, the test sequence generated by //Mark Serial// sequencer is usually longer.
-//Mark Serlia// sequencer (and [[#Sequencer - Mark Optimal| Mark Optimal]]) can be used for [[https://en.wikipedia.org/wiki/Exploratory_testing | Exploratory Testing]], e.g. testing a new user stories just added to the model.
-//Mark Serial// sequencer allows you to test scenarios that require traversing transitions in a specific order.
-=====Sequencer - MCase=====
-=====Dynamic MCase=====
-=====Sequencer - Concurrent=====