Differences

This shows you the differences between two versions of the page.

--- data_design_ide [2021/02/10 02:10] – [GENERATE] admin
+++ data_design_ide [2024/01/02 22:56] (current) – removed admin
@@ Line 1: / Line 1: @@
-===== Data Design IDE =====
-//Data Design IDE// is the development and execution (automation) of combinatorial test design models. This is typically used to generate a set of minimal permutations of variables to achieve the desired coverage using [[Combinatorial Algorithms| Combinatorial Testing]].
-Most of the software fault are caused by the interactions of 2-3 variables ([[https://csrc.nist.gov/projects/automated-combinatorial-testing-for-software | NIST]]).
-By using the appropriate combinatorial algorithm, you could cut down the number of test cases by 90% and still achieve sufficient test coverage.
-There are 4 tabs in this IDE that matches the development steps:
-  * [[#DEFINE ]]
-  * [[#GENERATE ]]
-  * [[#SCRIPT ]]
-  * [[#EXECUTE ]]
-----
-==== DEFINE ====
-{{wiki:idescreen:datadesign_main.png?800}}
-=== Variables ===
-In //DEFINE//, you define the testing variables as well as the expected result variables.
-  * //Variable Name// - variables must have unique names and preferably legal java variable names
-  * //Data Type// - appropriate data type must be selected especially if the variables are referenced by //Derived// variables
-  * //Domain Values// - a list of discrete values to be used to generate (populate) [[#GENERATE]] or groovy/java expression if //Derived// is checked
-  * //Derived// - check this box if the value of the variable depends on values of other variables.  Derived variable usually have static value or expression that evaluates to a value specified in //Domain// attribute.
-  * //Coupling// - used to couple / tie this variable to another variable. For example CreditCardType and CreditCardNumber should be coupled together as CreditCardNumber for VISA must match CreditCardType of VISA. Variables coupled together must have the same number of //Domain// values.
-== Domain Expression ==
-Domain values for non-//Derived// variables must be static values.
-For //Derived// variables, //Domain// must have exactly one value, which can be static value or //groovy// expression. Below is a simple example to calculate the insurance premium credit of $100 given to good student age < 20:
-   (Age < 20 && GoodStudentStatus = true)?100:0
-You may use plugin reference in //Domain// value/expression which will be evaluated during  [[#EXECUTE ]]. For example:
-   $VAR.adminUsername              $VAR.adminPassword
-   $VAR.normalUsername             $VAR.normalPassword
-   $VAR.supervisorUsername       $VAR.supervisorPassword
-For more complex scenario:
-   $UTIL.toJSON($VAR.order1Obj)
-where order1Obj is a groovy/java object that is defined/created in //DATA// [[#SCRIPT]].
-Use [blank] to encode blank string and [null] for null string/object.
-=== Constraints / Rules ===
-When designing test data using combinatorial testing techniques, we often need to remove or avoid certain combinations as they are either unfeasible or unnecessary for whatever reason.  You can remove these unwanted combinations manually after the test combinations have been generated, but that's tedious and prone to errors. Depending on number of test combinations generated, manually removing unwanted combinations may even be impossible due to large volume.
-The solution is to create a set of constraints or rules to filter out certain permutations.
-Constraints are expressed as a simple expression:
-   IF condition THEN assertion
-{{wiki:idescreen:datadesign_define_constraints.png}}
-The //condition// and //assertion// are simple boolean expressions:
-   [variable name] [operator] [value or [variable name] [&& or ||] ...
-For example:
-   IF (OS = "Windows") THEN (Browser = "IE" || Browser = "FireFox" || Browser = "Netscape")
-The //operator// can be:
-  * **>** : greater than, numeric fields only
-  * **<** : less than, numeric fields only
-  * **=** : equals
-  * **!=** : not equals
-  * **>=** : greater than or equal, numeric fields only
-  * **<****=** : less than or equal, numeric fields only
-Arithmetic operations are also supported:
-   IF DEPOSIT_AMOUNT - WITHDRAW_AMOUNT <= 0 THEN REJECT_TRANSATION = true
-Only numeric variables can participate in the arithmetic operations. Following arithmetic operations are supported:
-  * **+** : addition
-  * **-** : substration
-  * ***** : multiplication
-  * **/** : division
-  * **%** : mod operation
-When comparing with text, the text must be enclosed with double quotes (").
-Numeric constants may be used in the comparison, for example:
-   IF AGE >= 19 THEN ADULT = true
-Parenthesis can be used to help organize complex expression, for example:
-   IF (WINTER=true && TEMPERATURE > 30) || (SUMMER=true && TEMPERATURE < 90) THEN GOOD_WEATHER = true
-Static values specified in comparison must be a valid value for the variable or it will result in errors during generation.
-----
-==== GENERATE ====
-{{wiki:idescreen:datadesign_generate.png?800}}
-To generate //DataTable//, select an algorithm from //Overall Strength// drop-down list and click start button next to it.
-    * pairwise
-    * 3-wise
-    * 4-wise
-    * 5-wise
-    * 6-wise
-    * Mixed
-You may customize /edit //DataTable// under exceptional cases.
-  * remove rows - invalid permutations (rows)
-  * add rows - additional required permutations
-  * edit values - change value, usually done for derived variables
-Be aware that any changes to //DataTable// will affect the coverage, and the changes made will be lost next time //DataTable// is re-generated. In most cases, the customization can be accomplished with //Constraints// and //Derived// as described in [[#DEFINE]] above.
-=== Mixed Strengths ===
-By default, //Overall Strength// applies to all variables excluding //Derived// variables. In cases you wish to apply more rigorous testing than other variables.  You can do so by selecting //Mixed// in //Overall Strength// and and apply specific level of strength for each subset of the variables.
-{{wiki:idescreen:datadesign_generate_mixedstrength.png?200}}
-After selecting //Mixed// strength, section //Variable Interaction (Mixed Overall Strength only)// appears.  Click on "+" to add a  custom interaction/strength set, select the desired strength/algorithm and select the variables respectively.
-----
-==== SCRIPT ====
-{{wiki:idescreen:datadesign_script.png?800}}
-Script to be executed to process rows in //DataTable//.  As shown above, the script just loops through all rows in //$DATASET// which represents the //DataTable// and perform REST calls to test REST API on every row in //DataTable// and logs the results.
-//$DATASET.notifyClient(row)// sends the result back to IDE. If the execution was started with REST api, the result will be written to the output file specified in the REST request.
-Refer to [[ide_script | Script Editor ]] for more information about this script editor.
-There are two types of scripts: //MAIN// and //DATA//.
-//MAIN// script is the main program which executes the test cases.
-//DATA// script is used to instantiate and create objects (groovy/java) to be used in test case execution.
-For example, you may create a purchase order object to be posted to REST api:
-   $VAR.order1 = [product: 'cookie', price: 2.00, quantity: 20]
-The order object "order1" is assigned to $VAR.  It can then be used in //Domain// expression as:
-   $UTIL.toJSON($VAR.order1)
-----
-==== EXECUTE ====
-{{wiki:idescreen:datadesign_execute.png?800}}
-As script executes, IDE receives and displays the execution result one row at a time in the tabular format.
-The header shows the execution progress.  The execution results can also be exported to tab delimited file - click on the export button on the header.
-The executions currently are not counted in Dashboard stats.
-Executions can be archived/saved for future access.
-Historical executions can be accessed by clicking on //Past// link.