Model-Driven Software Product Line Testing: An Integrated Approach
Andy Schürr (1) , Sebastian Oster (1), Florian Markert (2)
Andy Schürr
[email protected] Tel.+49 6151 16 6940
(1) Real-Time Systems Lab (2) Computer Systems Lab Dept. of Electrical Engineering and Information Technology TUD – Technische Universität Darmstadt Merckstr. 25, D-64283 Darmstadt, Germany www.etit.tu-darmstadt.de 26.01.2010
© author(s) of these slides 2009 including research results of the research network ES and TU Darmstadt otherwise as specified at the respective slide
Model-Driven Software Product Line Testing: An Integrated Approach
Andy Schürr (1) , Sebastian Oster (1), Florian Markert (2)
Feature Models Andy Schürr
[email protected] Tel.+49 6151 16 6940
(1) Real-Time Systems Lab (2) Computer Systems Lab Dept. of Electrical Engineering and Information Technology TUD – Technische Universität Darmstadt Merckstr. 25, D-64283 Darmstadt, Germany www.etit.tu-darmstadt.de 26.01.2010
© author(s) of these slides 2009 including research results of the research network ES and TU Darmstadt otherwise as specified at the respective slide
Model-Driven Software Product Line Testing: An Integrated Approach
Andy Schürr (1) , Sebastian Oster (1), Florian Markert (2)
Andy Schürr
[email protected] Tel.+49 6151 16 6940
Classification Trees
(1) Real-Time Systems Lab (2) Computer Systems Lab Dept. of Electrical Engineering and Information Technology TUD – Technische Universität Darmstadt Merckstr. 25, D-64283 Darmstadt, Germany www.etit.tu-darmstadt.de 26.01.2010
© author(s) of these slides 2009 including research results of the research network ES and TU Darmstadt otherwise as specified at the respective slide
Model-Driven Software Product Line Testing: An Integrated Approach
Andy Schürr (1) , Sebastian Oster (1), Florian Markert (2)
Andy Schürr
[email protected] Tel.+49 6151 16 6940
(1) Real-Time Systems Lab (2) Computer Systems Lab Dept. of Electrical Engineering and Information Technology TUD – Technische Universität Darmstadt Merckstr. 25, D-64283 Darmstadt, Germany www.etit.tu-darmstadt.de 26.01.2010
© author(s) of these slides 2009 including research results of the research network ES and TU Darmstadt otherwise as specified at the respective slide
Car Configurator I
SOFSEM 2010
ES – Real-Time Systems Lab
Car Configurator II
SOFSEM 2010
ES – Real-Time Systems Lab
Automotive Software Configurations • • • •
70 – 100 Electronic Control Units Up to 10 billion lines of code Motor ECU: ~ 35.000 possible configurations Billions of indivual software configurations
= Software
SOFSEM 2010
ES – Real-Time Systems Lab
Software Feature Interaction?!
Smart parking break
SOFSEM 2010
+
Climatronic
=
ES – Real-Time Systems Lab
Outlook State-of-the-Art Software Product Line Engineering Black Box (Embedded) SW Testing Naïve Software Product Line Testing
„Our“ Approach Integrated Software Product Line Testing Meta-Modeling and Model Transformations Summary and Future Work
SOFSEM 2010
ES – Real-Time Systems Lab
Software Product Lines Software Product Line (SPL) Engineering: a process to develop different products which share a common set of features …
Systematic reuse of Software artifacts: requirements, features, components, etc. Benefits compared to individual system development • saving of costs • shorter time to market • increasing quality SOFSEM 2010
ES – Real-Time Systems Lab
Software Product Line Engineering
Development for reuse
Development with reuse
K. Pohl, G. Böckle, and F. van der Linden. Software Product Line Engineering: Foundations, Principles and Techniques. Springer-Verlag New York, Inc., 2005. SOFSEM 2010
ES – Real-Time Systems Lab
State-of-the-Art: Feature Modeling
• Describe the variabilities and commonalities of an SPL • Many similar approaches with different notations
„Our“ feature model editor (generated with metacase tool MOFLON) SOFSEM 2010
ES – Real-Time Systems Lab
Running Example – Car Seat
SOFSEM 2010
ES – Real-Time Systems Lab
Outlook State-of-the-Art Software Product Line Engineering Black Box (Embedded) SW Testing Naïve Software Product Line Testing
„Our“ Approach Integrated Software Product Line Testing Meta-Modeling and Model Transformations Summary and Future Work
SOFSEM 2010
ES – Real-Time Systems Lab
State-of-the-Art: Black Box Testing + … The Classification Tree Method (developed by Daimler) • Acyclic graph decomposing system function into input parameters • Equivalence classes of input parameter values (legal values, illegal values, stress test values, … ) • Heuristics for input parameter value selection
SOFSEM 2010
ES – Real-Time Systems Lab
Classification Tree of Running Example
CTE/ES © (RAZORCAT)
• The car is not moving and its door is opened • The seat moves back due to the automatic entry function • The seat has moved back and the driver enters the car • The driver has entered the car and closes the door • The seat moves back to its initial position and stops SOFSEM 2010
ES – Real-Time Systems Lab
State-of-the-Art: Black Box Testing + … The Classification Tree Method (developed by Daimler) • Acyclic graph decomposing system function into input parameters • Equivalence classes of input parameter values (legal values, illegal values, stress test values, … ) • Heuristics for input parameter value selection
Model-Based Testing Techniques • Executable Simulink / UML / Ptolemy / … model • Generation of tests based on model coverage criteria • Usage of model checking techniques •
SOFSEM 2010
…
ES – Real-Time Systems Lab
Ptolemy II © Simulation Environment © developed at Berkely
Sensors Motors
Seat Position
Enactable Models SOFSEM 2010
ES – Real-Time Systems Lab
Ptolemy II Simulation Environment
Seat automatic Seat Driver Door moves Idle opened entered forward finished SOFSEM 2010
ES – Real-Time Systems Lab
Outlook State-of-the-Art Software Product Line Engineering Black Box (Embedded) SW Testing Naïve Software Product Line Testing
„Our“ Approach Integrated Software Product Line Testing Meta-Modeling and Model Transformations Summary and Future Work
SOFSEM 2010
ES – Real-Time Systems Lab
Software Product Line Testing
State-of-the-Art: • Product by product • Regression tests • …
Feature Models & Classification Trees • • • •
SOFSEM 2010
Select representative subset of all feature combinations Use heuristics for selection of critical feature combinations Define „classification tree“-based black box tests Combine with model-based software development
ES – Real-Time Systems Lab
„Naïve“ SPL Black Box Testing Approach
1. Construct software product line feature model 2. Normalize feature model & select product instance 3. Derive classification tree for selected product 4. Define suite of test cases for classification tree
1. SOFSEM 2010
2.
3.
4. ES – Real-Time Systems Lab
1. Construct Feature Model (FM)
A a
b c
A
C
B
f
e
a
b
B
A
C
B
C
f d
d
A
C
[...] [...] [...]
[...] [...]
[...] [...]
[...] [...] [...]
[...] [...]
[...] [...]
require exclude
SOFSEM 2010
[...] [...]
[...] [...] [...]
[...] [...]
[...] [...] [...]
ES – Real-Time Systems Lab
2. Select Product Instance (PI)
SOFSEM 2010
ES – Real-Time Systems Lab
3. Derive Classification Tree (CT)
A a
b c
A
C
B
f
e
a
b
B
A
C
B
C
f d
d
A
C
[...] [...] [...]
[...] [...]
[...] [...]
[...] [...] [...]
[...] [...]
[...] [...]
require exclude
SOFSEM 2010
[...] [...]
[...] [...] [...]
[...] [...]
[...] [...] [...]
ES – Real-Time Systems Lab
4. Define Test Cases (TC)
A a
b c
A
C
B
f
e
a
b
B
A
C
B
C
f d
d
A
C
[...] [...] [...]
[...] [...]
[...] [...]
[...] [...] [...]
[...] [...]
[...] [...]
require exclude
SOFSEM 2010
[...] [...]
[...] [...] [...]
[...] [...]
[...] [...] [...]
ES – Real-Time Systems Lab
„Naïve“ SPL Black Box Testing: Summary
SOFSEM 2010
ES – Real-Time Systems Lab
Outlook State-of-the-Art Software Product Line Engineering Black Box (Embedded) SW Testing Naïve Software Product Line Testing
„Our“ Approach Integrated Software Product Line Testing Meta-Modeling and Model Transformations Summary and Future Work
SOFSEM 2010
ES – Real-Time Systems Lab
FMT – Integrated FM & CT Testing Approach Feature models and classification trees are popular Both languages share a substantial subset of constructs Both languages may profit from each other Combination supports integrated feature & test case definition
+ Feature Models + Classification Trees SOFSEM 2010
= =
FMT ES – Real-Time Systems Lab
FMT Language Constructs
SOFSEM 2010
ES – Real-Time Systems Lab
Advantages of FMT Approach “Naïve” Approach
FMT-based Approach SOFSEM 2010
ES – Real-Time Systems Lab
FMT of the Running Example : selection at construction time
SPL
: selection at runtime
Features
… Functions, Sensors, … Param Values
SOFSEM 2010
ES – Real-Time Systems Lab
Model-Based Software Product Line Testing
SOFSEM 2010
ES – Real-Time Systems Lab
Outlook State-of-the-Art Software Product Line Engineering Black Box (Embedded) SW Testing Naïve Software Product Line Testing
„Our“ Approach Integrated Software Product Line Testing Meta-Modeling and Model Transformations Summary and Future Work
SOFSEM 2010
ES – Real-Time Systems Lab
Implementation with MetaCase Tool MOFLON
Version 1.4
MOF MOF2.x 2.x Metamodel Metamodel Def. Def.Language Language TU Darmstadt
SOFSEM 2010
ES – Real-Time Systems Lab
FMT Metamodel (MOF 2.0)
SOFSEM 2010
ES – Real-Time Systems Lab
Implementation with MetaCase Tool MOFLON
Version 1.4
MOF MOF2.x 2.x Metamodel Metamodel Def. Def.Language Language TU Dresden
TU Darmstadt
OCL OCL2.x 2.x Constraint Constraint Def. Def.Language Language
SDM SDM Fujaba FujabaGraph Graph Transformations Transformations University of Kassel University of Paderborn
SOFSEM 2010
ES – Real-Time Systems Lab
Feature Model Transformation
requires
root ∧ ( A → root ) ∧ (a → A) ∧ ( A → a ) = root ∧ (¬A ∨ root ) ∧ (¬a ∨ A) ∧ (¬A ∨ a ) = root ∧ [(¬a ∧ ¬A) ∨ ( A ∧ a )]
=
root ∧ ( A → root) ∧ (a → root) ∧ (a → A) ∧ ( A → a) =root ∧ (¬A ∨ root) ∧ (¬a ∨ root) ∧ (¬a ∨ A) ∧ (¬A ∨ a) =root ∧ [( ¬a ∧ ¬A) ∨ ( A ∧ a)]
Verification of model transformation SOFSEM 2010
ES – Real-Time Systems Lab
Feature Model Transformation II
SOFSEM 2010
ES – Real-Time Systems Lab
SDM Implementation of FMT Transformation - 1
SOFSEM 2010
ES – Real-Time Systems Lab
SDM Implementation of FMT Transformation - 2
SOFSEM 2010
ES – Real-Time Systems Lab
SDM Implementation of FMT Transformation - 3
SOFSEM 2010
ES – Real-Time Systems Lab
Summary and Challenges Summary Integrated feature and test case modeling approach Heuristics (pairwise combination) for test candidate selection Heuristics (boundary tests, … ) for test case selection Prototype development with metacase tool MOFLON Future Work Characterization of independent/interacting features (data flow analysis of related code fragments) Evaluation using real-world examples Integration with RE, design, … activities … SOFSEM 2010
ES – Real-Time Systems Lab
Future Work – FMT & RE Integration (Un)Parser Definition
MOF Meta Model
CodeAnalysis Fragment Results
Model Analysis & Refactoring FMT Modell File
(Un)Parser API Adapter
OCL Constraints
FMT Model
Model Synchronizer
SDM Graph Transformation generates
CodeRE Fragment File
SOFSEM 2010
(Un)Parser API Adapter
RE Model
(Un)Parser Definition
MOF Meta Model
TGG (QVT-like) Translations
ES – Real-Time Systems Lab
Implementation with MetaCase Tool MOFLON Monticore Monticore Parser, Parser,… … RWTH Aachen
Version 1.4
MOF MOF2.x 2.x Metamodel Metamodel Def. Def.Language Language TU Dresden
TU Darmstadt
OCL OCL2.x 2.x Constraint Constraint Def. Def.Language Language
TU Darmstadt SOFSEM 2010
SDM SDM Fujaba FujabaGraph Graph Transformations Transformations TGG TGG(QVT-like) (QVT-like) M2M M2MTranslation Translation Def. Def.Language Language
University of Kassel University of Paderborn
ES – Real-Time Systems Lab
Thank you for your attention !!!
MOFLON 1.4
SOFSEM Participant
See also: http://www.moflon.org/ SOFSEM 2010
ES – Real-Time Systems Lab
Model-Driven Software Development Research 6. 6.Model-Based Model-Based&& Product Product Line LineTesting Testing
Sebastian Oster [TUD,BMBF] Harald Cichos [TUD]
[CASED, TUD]
5. 5. Model-Driven Model-Driven Security Security Engineering Engineering
4. 4. Software SoftwareModernization Modernization
Application Application Areas: Areas: Sebastian Rose [EAG,SAP] Felix Klar [TUD,Daimler]
Automotive Automotive SW SW
Lars Patzina [LOEWE] Sven Patzina [TUD] Gergely Varro [AvH]
Automation Automation SW SW … …
2. 2. Modeling ModelingLanguage Language&& Tool ToolIntegration Integration&&P2P P2P
Patrick Mukherjee [DFG] NN [DFG]
SOFSEM 2010
3. 3.Domain-Specific Domain-SpecificEng. Eng. Languages Languages&&Methods Methods
1. 1. Meta-Models Meta-Models && Model Model Transformations (OMG) Transformations (OMG)
Elodie Legros [BMBF] Martin Wieber [BMBF] Antony Anjorin [DFG,Siemens] Marius Lauder [DFG,Siemens] ES – Real-Time Systems Lab