Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
data_design_ide [2021/02/09 19:49]
admin [DEFINE] 		data_design_ide [2021/02/14 02:43] (current)
admin [SCRIPT]
Line 1: Line 1:
-===== Data Design IDE =====+====== Data Design IDE ======
 //Data Design IDE// is the development and execution (automation) of combinatorial test design models. This is typically used to generate a set of minimal permutations of variables to achieve the desired coverage using [[Combinatorial Algorithms| Combinatorial Testing]].  //Data Design IDE// is the development and execution (automation) of combinatorial test design models. This is typically used to generate a set of minimal permutations of variables to achieve the desired coverage using [[Combinatorial Algorithms| Combinatorial Testing]]. 
  
Line 15: Line 15:
  
  
-==== DEFINE ====+===== DEFINE =====
  
 {{wiki:idescreen:datadesign_main.png?800}} {{wiki:idescreen:datadesign_main.png?800}}
  
-=== Variables ===+==== Variables ====
 In //DEFINE//, you define the testing variables as well as the expected result variables. In //DEFINE//, you define the testing variables as well as the expected result variables.
   * //Variable Name// - variables must have unique names and preferably legal java variable names   * //Variable Name// - variables must have unique names and preferably legal java variable names
   * //Data Type// - appropriate data type must be selected especially if the variables are referenced by //Derived// variables   * //Data Type// - appropriate data type must be selected especially if the variables are referenced by //Derived// variables
-  * //Domain Values// - a list of discrete values to be used to generate (populate) [[#DataTable]] or groovy/java expression if //Derived// is checked +  * //Domain Values// - a list of discrete values to be used to generate (populate) [[#GENERATE]] or groovy/java expression if //Derived// is checked 
-  * //Derived// - check this box to exclude the variable from participating in the data generation.  For example the variables you will use to validate result or variables that can be derived from other variables.  If you +  * //Derived// - check this box if the value of the variable depends on values of other variables.  Derived variable usually have static value or expression that evaluates to a value specified in //Domain// attribute.
   * //Coupling// - used to couple / tie this variable to another variable. For example CreditCardType and CreditCardNumber should be coupled together as CreditCardNumber for VISA must match CreditCardType of VISA. Variables coupled together must have the same number of //Domain// values.   * //Coupling// - used to couple / tie this variable to another variable. For example CreditCardType and CreditCardNumber should be coupled together as CreditCardNumber for VISA must match CreditCardType of VISA. Variables coupled together must have the same number of //Domain// values.
  
-=== Constraints / Rules === +=== Domain Expression === 
-When designing test data using combinatorial testing techniques, we often need to remove or avoid certain combinations as they are either unfeasible or unnecessary for whatever reason.  You can certainly remove them manually after the test combinations have been generated, but that's tedious and prone to errors. Depending on number of test combinations generated manually removing unwanted combinations may even be impossible due to large volume.+Domain values for non-//Derived// variables must be static values. 
 + 
 +For //Derived// variables, //Domain// must have exactly one value, which can be static value or //groovy// expression. Below is a simple example to calculate the insurance premium credit of $100 given to good student age < 20: 
 + 
 +   (Age < 20 && GoodStudentStatus)?100:0 
 +    
 +You may use plugin reference in //Domain// value/expression which will be evaluated during  [[#EXECUTE ]]. For example: 
 + 
 +   $VAR.adminUsername              $VAR.adminPassword 
 +   $VAR.normalUsername             $VAR.normalPassword 
 +   $VAR.supervisorUsername       $VAR.supervisorPassword 
 +    
 +For more complex scenario: 
 +   $UTIL.toJSON($VAR.order1Obj) 
 + 
 +where order1Obj is a groovy/java object that is defined/created in //DATA// [[#SCRIPT]]. 
 + 
 +Use [blank] to encode blank string and [null] for null string/object. 
 + 
 +==== Constraints / Rules ==== 
 +When designing test data using combinatorial testing techniques, we often need to remove or avoid certain combinations as they are either unfeasible or unnecessary for whatever reason.  You can remove these unwanted combinations manually after the test combinations have been generated, but that's tedious and prone to errors. Depending on number of test combinations generated, manually removing unwanted combinations may even be impossible due to large volume.
  
 The solution is to create a set of constraints or rules to filter out certain permutations.   The solution is to create a set of constraints or rules to filter out certain permutations.  
Line 38: Line 58:
  
        
-The //condition// and //assertion// are simple boolean expressions:+The //condition// and //assertion// are simple boolean expressions (NOTE: this is not //groovy expression//):
    [variable name] [operator] [value or [variable name] [&& or ||] ...     [variable name] [operator] [value or [variable name] [&& or ||] ... 
  
Line 70: Line 90:
    IF (WINTER=true && TEMPERATURE > 30) || (SUMMER=true && TEMPERATURE < 90) THEN GOOD_WEATHER = true    IF (WINTER=true && TEMPERATURE > 30) || (SUMMER=true && TEMPERATURE < 90) THEN GOOD_WEATHER = true
    
 +Static values specified in comparison must be a valid value for the variable or it will result in errors during generation.
  
 ---- ----
-==== GENERATE ====+===== GENERATE =====
 {{wiki:idescreen:datadesign_generate.png?800}} {{wiki:idescreen:datadesign_generate.png?800}}
  
Line 90: Line 111:
 Be aware that any changes to //DataTable// will affect the coverage, and the changes made will be lost next time //DataTable// is re-generated. In most cases, the customization can be accomplished with //Constraints// and //Derived// as described in [[#DEFINE]] above. Be aware that any changes to //DataTable// will affect the coverage, and the changes made will be lost next time //DataTable// is re-generated. In most cases, the customization can be accomplished with //Constraints// and //Derived// as described in [[#DEFINE]] above.
  
 +==== Mixed Strengths ====
 By default, //Overall Strength// applies to all variables excluding //Derived// variables. In cases you wish to apply more rigorous testing than other variables.  You can do so by selecting //Mixed// in //Overall Strength// and and apply specific level of strength for each subset of the variables.   By default, //Overall Strength// applies to all variables excluding //Derived// variables. In cases you wish to apply more rigorous testing than other variables.  You can do so by selecting //Mixed// in //Overall Strength// and and apply specific level of strength for each subset of the variables.  
  
 {{wiki:idescreen:datadesign_generate_mixedstrength.png?200}} {{wiki:idescreen:datadesign_generate_mixedstrength.png?200}}
 +
 +After selecting //Mixed// strength, section //Variable Interaction (Mixed Overall Strength only)// appears.  Click on "+" to add a  custom interaction/strength set, select the desired strength/algorithm and select the variables respectively.
 +
  
 ---- ----
-==== SCRIPT ====+===== SCRIPT =====
  
 {{wiki:idescreen:datadesign_script.png?800}} {{wiki:idescreen:datadesign_script.png?800}}
Line 104: Line 129:
  
 Refer to [[ide_script | Script Editor ]] for more information about this script editor. Refer to [[ide_script | Script Editor ]] for more information about this script editor.
 +
 +There are two types of scripts: //MAIN// and //DATA//.
 +
 +//MAIN// script is the main program which executes the test cases.
 +
 +//DATA// script is used to instantiate and create objects (groovy/java) to be used in test case execution. 
 +For example, you may create a purchase order object to be posted to REST api:
 +   $VAR.order1 = [product: 'cookie', price: 2.00, quantity: 20]
 +
 +The order object "order1" is assigned to $VAR.  It can then be used in //Domain// expression as:
 +   $UTIL.toJSON($VAR.order1)
 +
 +//INIT// script is for initialization of variables, e.g. environment, user name, password, etc., the variables that may need to be changed from execution to execution.
  
 ---- ----
-==== EXECUTE ====+ 
 +===== EXECUTE =====
  
 {{wiki:idescreen:datadesign_execute.png?800}} {{wiki:idescreen:datadesign_execute.png?800}}
Line 115: Line 154:
  
 The executions currently are not counted in Dashboard stats. The executions currently are not counted in Dashboard stats.
 +
 +Executions can be archived/saved for future access.  
 +
 +Historical executions can be accessed by clicking on //Past// link.