Apr 24, 2015 - The choice of a model for MBT with right properties depends on the application domain. ⢠We focus on the domain of Case Management.
A Test Generator for ModelBased Testing Ella Roubtsova, OU Nederland Serguei Roubtsov. TU Eindhoven
A Test Generator for Model-Based Testing 4/24/2015
BMSD 2014, Luxembourg
1
What is a software test? It is a procedure or a sequence of actions that allows one to prove that the software meets the requirements
4/24/2015
BMSD 2014, Luxembourg
2
Model based testing MBT
• A model for MBT should be different from the implementation model (the UML test profile, 2005) • The model should provides obvious reduction of testing effort. 4/24/2015
BMSD 2014, Luxembourg
3
Application Domain Case Management solution • The choice of a model for MBT with right properties depends on the application domain • We focus on the domain of Case Management Solutions (CMS) that covers the systems that “coordinate a service request in finance, health, legal, citizen or human resources-related matters, on behalf of a subject such as a customer, a citizen, or an employee” Davenport and Nohria 4/24/2015
BMSD 2014, Luxembourg
4
Example of a process supported with a Case Management solution
4/24/2015
BMSD 2014, Luxembourg
5
Case Management
http://www.pega.com/products/case-management 4/24/2015
BMSD 2014, Luxembourg
6
Requirements for a model to generation tests for CMSs • be deterministic, • present only visible system states, • enable expressions with relevant data and data transformation, • separate the decision modules and cluster the data, • facilitate independent modelling of a change and its composition with the old model can reduce modelling and testing effort. 4/24/2015
BMSD 2014, Luxembourg
7
Inside UML
I1 : User
: Graph Maker
a1
b1
a2
b2
O1
: Graph Drawer
I2 a1
1: IGetGraph
a2
2: IDataSeries
I1
a3
3: IDataSeries:structure
I2
a4
4: IDraw
a5
5: IDraw:structure
a1
a2
IO1i
IO2i
i1!
IO1
i2!
IO2
O2
o1?
o2?
IO1o
IO2o
b1
b2
O1 O2
o2!
a6
i1?
6: IGetGraph:true
F1
F0
BMSD 2014, Luxembourg
o1!
F3 i2?
4/24/2015
F4
i2?
F2
i1? o1!
o2!
F5
8
Outside UML Petri Nets
instance check2
check1
client
checked1
look
choice
choose
chosen
checked2
more
deny2
ready
pay
deny1
CSP parallel Composition
Protocol Models
E
All machines that recognize E are in state, where they are able to accept E
LTSs by Kervinen et al.
Save Number
Singleton
Local Storage: Name PhoneNumber
Initialise none
Update, FindCall
some
Delete
Insert
creates
submitted
Calling Machine Modify= {Insert, Update, Delete}
Off Call End
On Initialise Initialised
Call Enabled
Off
Calling1
EndCallSaving
End Calling2 FindCall
4/24/2015
Local Storage: Phone Identifier Number Called
BMSD 2014, Luxembourg
Off
9
Results of the Survey Existing behaviour models are mostly for requirements and system understanding and not directly applicable for MBT
4/24/2015
BMSD 2014, Luxembourg
10
Properties of a Protocol Model
Is deterministic, presents only visible system states, enables expressions with relevant data and data transformation, facilitates independent modelling of a change and its composition with the old model, separates the decision modules and cluster the data However, protocol models can hide generation of some elements of states in callbacks
4/24/2015
BMSD 2014, Luxembourg
11
Protocol model CSP parallel composition
4/24/2015
BMSD 2014, Luxembourg
12
CSP parallel composition and its use for MBT
4/24/2015
BMSD 2014, Luxembourg
13
Behaviours, Call Backs, Local Storage, Derived State
4/24/2015
BMSD 2014, Luxembourg
14
MBT preparation: Open hidden semantics
4/24/2015
BMSD 2014, Luxembourg
15
MBT Preparation: Apply CSP parallel composition rules Accept
Accept
Protocol Machine (1)
Refuse
Refuse Ignore Accept
Ignore
Refuse
Protocol Machine (2)
Ignore
Ignore
and
Ignore
=
Ignore
Refuse
or
Refuse
=
Refuse
Anything else 4/24/2015
= AOM Luxembourg 2009 BMSD 2014,
Accept 16
Protocol Model –Based Testing Methodology
4/24/2015
BMSD 2014, Luxembourg
17
Algorithm for the CSP parallel composition of two protocol machines 𝑒 ∈ 𝐸𝐴 ∩ 𝐸𝐵 A.x*e → A.y B.2*e → B.3 ----------------A.x;B.2*e→A.y;B.3
4/24/2015
BMSD 2014, Luxembourg
18
Iterative generation of the System Protocol Machine
4/24/2015
BMSD 2014, Luxembourg
19
Generation of traces • Each unique trace has to start from the transition in which the initial state of each object is @new. This way a test case can cover the whole life cycle of the objects involved in a trace. • Each possible transition in a particular trace has to be repeated only once. This way we decrease the number of test steps while still providing complete transition coverage. • Traversing Transition tree
4/24/2015
BMSD 2014, Luxembourg
20
Protocol model CSP parallel composition
4/24/2015
BMSD 2014, Luxembourg
21
Traces and Observational Consistency
4/24/2015
BMSD 2014, Luxembourg
22
Advantages for Regression Testing • The goal of regression testing is to ensure that changes such as those mentioned above have not introduced new faults. • Common methods of regression testing include rerunning previously completed tests and checking whether program behavior has changed and whether previously fixed faults have re-emerged. • Tests generated from protocol machines contain previously fulfilled tests and do not demand the rerunning previously completed tests. 4/24/2015
BMSD 2014, Luxembourg
23
Meet the requirements for MBT • We found a modelling approach that meets the requirements for MBT of highly changeable Case Management Solutions. • We developed and prototyped a method for MBT using the Protocol Models for test generation. • We have shown with experiments that the method decreases the modelling and regression testing efforts of CMSs.
4/24/2015
BMSD 2014, Luxembourg
24
Protocol ModelBased Testing Methodology
4/24/2015
BMSD 2014, Luxembourg
25
Future Work • Automation of the model preparation • Partitioning a protocol model into the groups of independent protocol machines in order to reduce testing effort
4/24/2015
BMSD 2014, Luxembourg
26
