Model-Driven Software Product Line Testing

Model-Driven Software Product Line Testing: An Integrated Approach

Andy Schürr (1) , Sebastian Oster (1), Florian Markert (2)

Andy Schürr [email protected] Tel.+49 6151 16 6940

(1) Real-Time Systems Lab (2) Computer Systems Lab Dept. of Electrical Engineering and Information Technology TUD – Technische Universität Darmstadt Merckstr. 25, D-64283 Darmstadt, Germany www.etit.tu-darmstadt.de 26.01.2010

© author(s) of these slides 2009 including research results of the research network ES and TU Darmstadt otherwise as specified at the respective slide



Feature Models Andy Schürr [email protected] Tel.+49 6151 16 6940






Classification Trees








Car Configurator I

SOFSEM 2010

ES – Real-Time Systems Lab

Car Configurator II

SOFSEM 2010


Automotive Software Configurations • • • •

70 – 100 Electronic Control Units Up to 10 billion lines of code Motor ECU: ~ 35.000 possible configurations Billions of indivual software configurations

= Software

SOFSEM 2010


Software Feature Interaction?!

Smart parking break

SOFSEM 2010

+

Climatronic

=


Outlook State-of-the-Art Software Product Line Engineering Black Box (Embedded) SW Testing Naïve Software Product Line Testing

„Our“ Approach Integrated Software Product Line Testing Meta-Modeling and Model Transformations Summary and Future Work

SOFSEM 2010


Software Product Lines Software Product Line (SPL) Engineering: a process to develop different products which share a common set of features …

Systematic reuse of Software artifacts: requirements, features, components, etc. Benefits compared to individual system development • saving of costs • shorter time to market • increasing quality SOFSEM 2010


Software Product Line Engineering

Development for reuse

Development with reuse

K. Pohl, G. Böckle, and F. van der Linden. Software Product Line Engineering: Foundations, Principles and Techniques. Springer-Verlag New York, Inc., 2005. SOFSEM 2010


State-of-the-Art: Feature Modeling

• Describe the variabilities and commonalities of an SPL • Many similar approaches with different notations

„Our“ feature model editor (generated with metacase tool MOFLON) SOFSEM 2010


Running Example – Car Seat

SOFSEM 2010




SOFSEM 2010


State-of-the-Art: Black Box Testing + … The Classification Tree Method (developed by Daimler) • Acyclic graph decomposing system function into input parameters • Equivalence classes of input parameter values (legal values, illegal values, stress test values, … ) • Heuristics for input parameter value selection

SOFSEM 2010


Classification Tree of Running Example

CTE/ES © (RAZORCAT)

• The car is not moving and its door is opened • The seat moves back due to the automatic entry function • The seat has moved back and the driver enters the car • The driver has entered the car and closes the door • The seat moves back to its initial position and stops SOFSEM 2010


State-of-the-Art: Black Box Testing + … The Classification Tree Method (developed by Daimler) • Acyclic graph decomposing system function into input parameters • Equivalence classes of input parameter values (legal values, illegal values, stress test values, … ) • Heuristics for input parameter value selection

Model-Based Testing Techniques • Executable Simulink / UML / Ptolemy / … model • Generation of tests based on model coverage criteria • Usage of model checking techniques •

SOFSEM 2010

…


Ptolemy II © Simulation Environment © developed at Berkely

Sensors Motors

Seat Position

Enactable Models SOFSEM 2010


Ptolemy II Simulation Environment

Seat automatic Seat Driver Door moves Idle opened entered forward finished SOFSEM 2010




SOFSEM 2010


Software Product Line Testing

State-of-the-Art: • Product by product • Regression tests • …

Feature Models & Classification Trees • • • •

SOFSEM 2010

Select representative subset of all feature combinations Use heuristics for selection of critical feature combinations Define „classification tree“-based black box tests Combine with model-based software development


„Naïve“ SPL Black Box Testing Approach

1. Construct software product line feature model 2. Normalize feature model & select product instance 3. Derive classification tree for selected product 4. Define suite of test cases for classification tree

1. SOFSEM 2010

2.

3.

4. ES – Real-Time Systems Lab

1. Construct Feature Model (FM)

A a

b c

A

C

B

f

e

a

b

B

A

C

B

C

f d

d

A

C

[...] [...] [...]

[...] [...]

[...] [...]

[...] [...] [...]

[...] [...]

[...] [...]

require exclude

SOFSEM 2010

[...] [...]

[...] [...] [...]

[...] [...]

[...] [...] [...]


2. Select Product Instance (PI)

SOFSEM 2010


3. Derive Classification Tree (CT)

A a

b c

A

C

B

f

e

a

b

B

A

C

B

C

f d

d

A

C

[...] [...] [...]

[...] [...]

[...] [...]

[...] [...] [...]

[...] [...]

[...] [...]

require exclude

SOFSEM 2010

[...] [...]

[...] [...] [...]

[...] [...]

[...] [...] [...]


4. Define Test Cases (TC)

A a

b c

A

C

B

f

e

a

b

B

A

C

B

C

f d

d

A

C

[...] [...] [...]

[...] [...]

[...] [...]

[...] [...] [...]

[...] [...]

[...] [...]

require exclude

SOFSEM 2010

[...] [...]

[...] [...] [...]

[...] [...]

[...] [...] [...]


„Naïve“ SPL Black Box Testing: Summary

SOFSEM 2010




SOFSEM 2010


FMT – Integrated FM & CT Testing Approach Feature models and classification trees are popular Both languages share a substantial subset of constructs Both languages may profit from each other Combination supports integrated feature & test case definition

+ Feature Models + Classification Trees SOFSEM 2010

= =

FMT ES – Real-Time Systems Lab

FMT Language Constructs

SOFSEM 2010


Advantages of FMT Approach “Naïve” Approach

FMT-based Approach SOFSEM 2010


FMT of the Running Example : selection at construction time

SPL

: selection at runtime

Features

… Functions, Sensors, … Param Values

SOFSEM 2010


Model-Based Software Product Line Testing

SOFSEM 2010




SOFSEM 2010


Implementation with MetaCase Tool MOFLON

Version 1.4

MOF MOF2.x 2.x Metamodel Metamodel Def. Def.Language Language TU Darmstadt

SOFSEM 2010


FMT Metamodel (MOF 2.0)

SOFSEM 2010


Implementation with MetaCase Tool MOFLON

Version 1.4

MOF MOF2.x 2.x Metamodel Metamodel Def. Def.Language Language TU Dresden

TU Darmstadt

OCL OCL2.x 2.x Constraint Constraint Def. Def.Language Language

SDM SDM Fujaba FujabaGraph Graph Transformations Transformations University of Kassel University of Paderborn

SOFSEM 2010


Feature Model Transformation

requires

root ∧ ( A → root ) ∧ (a → A) ∧ ( A → a ) = root ∧ (¬A ∨ root ) ∧ (¬a ∨ A) ∧ (¬A ∨ a ) = root ∧ [(¬a ∧ ¬A) ∨ ( A ∧ a )]

=

root ∧ ( A → root) ∧ (a → root) ∧ (a → A) ∧ ( A → a) =root ∧ (¬A ∨ root) ∧ (¬a ∨ root) ∧ (¬a ∨ A) ∧ (¬A ∨ a) =root ∧ [( ¬a ∧ ¬A) ∨ ( A ∧ a)]

Verification of model transformation SOFSEM 2010


Feature Model Transformation II

SOFSEM 2010


SDM Implementation of FMT Transformation - 1

SOFSEM 2010



SOFSEM 2010



SOFSEM 2010


Summary and Challenges Summary Integrated feature and test case modeling approach Heuristics (pairwise combination) for test candidate selection Heuristics (boundary tests, … ) for test case selection Prototype development with metacase tool MOFLON Future Work Characterization of independent/interacting features (data flow analysis of related code fragments) Evaluation using real-world examples Integration with RE, design, … activities … SOFSEM 2010


Future Work – FMT & RE Integration (Un)Parser Definition

MOF Meta Model

CodeAnalysis Fragment Results

Model Analysis & Refactoring FMT Modell File

(Un)Parser API Adapter

OCL Constraints

FMT Model

Model Synchronizer

SDM Graph Transformation generates

CodeRE Fragment File

SOFSEM 2010

(Un)Parser API Adapter

RE Model

(Un)Parser Definition

MOF Meta Model

TGG (QVT-like) Translations


Implementation with MetaCase Tool MOFLON Monticore Monticore Parser, Parser,… … RWTH Aachen

Version 1.4

MOF MOF2.x 2.x Metamodel Metamodel Def. Def.Language Language TU Dresden

TU Darmstadt

OCL OCL2.x 2.x Constraint Constraint Def. Def.Language Language

TU Darmstadt SOFSEM 2010

SDM SDM Fujaba FujabaGraph Graph Transformations Transformations TGG TGG(QVT-like) (QVT-like) M2M M2MTranslation Translation Def. Def.Language Language

University of Kassel University of Paderborn


Thank you for your attention !!!

MOFLON 1.4

SOFSEM Participant

See also: http://www.moflon.org/ SOFSEM 2010


Model-Driven Software Development Research 6. 6.Model-Based Model-Based&& Product Product Line LineTesting Testing

Sebastian Oster [TUD,BMBF] Harald Cichos [TUD]

[CASED, TUD]

5. 5. Model-Driven Model-Driven Security Security Engineering Engineering

4. 4. Software SoftwareModernization Modernization

Application Application Areas: Areas: Sebastian Rose [EAG,SAP] Felix Klar [TUD,Daimler]

Automotive Automotive SW SW

Lars Patzina [LOEWE] Sven Patzina [TUD] Gergely Varro [AvH]

Automation Automation SW SW … …

2. 2. Modeling ModelingLanguage Language&& Tool ToolIntegration Integration&&P2P P2P

Patrick Mukherjee [DFG] NN [DFG]

SOFSEM 2010

3. 3.Domain-Specific Domain-SpecificEng. Eng. Languages Languages&&Methods Methods

1. 1. Meta-Models Meta-Models && Model Model Transformations (OMG) Transformations (OMG)

Elodie Legros [BMBF] Martin Wieber [BMBF] Antony Anjorin [DFG,Siemens] Marius Lauder [DFG,Siemens] ES – Real-Time Systems Lab