Model-Driven Software Product Line Testing

5 downloads 557 Views 1MB Size Report
Jan 26, 2010 - Tel.+49 6151 16 6940. Model-Driven Software Product Line Testing: .... (legal values, illegal values, stress test values, … ) • Heuristics for input ...
Model-Driven Software Product Line Testing: An Integrated Approach

Andy Schürr (1) , Sebastian Oster (1), Florian Markert (2)

Andy Schürr [email protected] Tel.+49 6151 16 6940

(1) Real-Time Systems Lab (2) Computer Systems Lab Dept. of Electrical Engineering and Information Technology TUD – Technische Universität Darmstadt Merckstr. 25, D-64283 Darmstadt, Germany www.etit.tu-darmstadt.de 26.01.2010

© author(s) of these slides 2009 including research results of the research network ES and TU Darmstadt otherwise as specified at the respective slide

Model-Driven Software Product Line Testing: An Integrated Approach

Andy Schürr (1) , Sebastian Oster (1), Florian Markert (2)

Feature Models Andy Schürr [email protected] Tel.+49 6151 16 6940

(1) Real-Time Systems Lab (2) Computer Systems Lab Dept. of Electrical Engineering and Information Technology TUD – Technische Universität Darmstadt Merckstr. 25, D-64283 Darmstadt, Germany www.etit.tu-darmstadt.de 26.01.2010

© author(s) of these slides 2009 including research results of the research network ES and TU Darmstadt otherwise as specified at the respective slide

Model-Driven Software Product Line Testing: An Integrated Approach

Andy Schürr (1) , Sebastian Oster (1), Florian Markert (2)

Andy Schürr [email protected] Tel.+49 6151 16 6940

Classification Trees

(1) Real-Time Systems Lab (2) Computer Systems Lab Dept. of Electrical Engineering and Information Technology TUD – Technische Universität Darmstadt Merckstr. 25, D-64283 Darmstadt, Germany www.etit.tu-darmstadt.de 26.01.2010

© author(s) of these slides 2009 including research results of the research network ES and TU Darmstadt otherwise as specified at the respective slide

Model-Driven Software Product Line Testing: An Integrated Approach

Andy Schürr (1) , Sebastian Oster (1), Florian Markert (2)

Andy Schürr [email protected] Tel.+49 6151 16 6940

(1) Real-Time Systems Lab (2) Computer Systems Lab Dept. of Electrical Engineering and Information Technology TUD – Technische Universität Darmstadt Merckstr. 25, D-64283 Darmstadt, Germany www.etit.tu-darmstadt.de 26.01.2010

© author(s) of these slides 2009 including research results of the research network ES and TU Darmstadt otherwise as specified at the respective slide

Car Configurator I

SOFSEM 2010

ES – Real-Time Systems Lab

Car Configurator II

SOFSEM 2010

ES – Real-Time Systems Lab

Automotive Software Configurations • • • •

70 – 100 Electronic Control Units Up to 10 billion lines of code Motor ECU: ~ 35.000 possible configurations Billions of indivual software configurations

= Software

SOFSEM 2010

ES – Real-Time Systems Lab

Software Feature Interaction?!

Smart parking break

SOFSEM 2010

+

Climatronic

=

ES – Real-Time Systems Lab

Outlook ‰ State-of-the-Art ƒ Software Product Line Engineering ƒ Black Box (Embedded) SW Testing ƒ Naïve Software Product Line Testing ‰

„Our“ Approach ƒ Integrated Software Product Line Testing ƒ Meta-Modeling and Model Transformations ƒ Summary and Future Work

SOFSEM 2010

ES – Real-Time Systems Lab

Software Product Lines ‰ Software Product Line (SPL) Engineering: a process to develop different products which share a common set of features …

‰ Systematic reuse of Software artifacts: requirements, features, components, etc. ‰ Benefits compared to individual system development • saving of costs • shorter time to market • increasing quality SOFSEM 2010

ES – Real-Time Systems Lab

Software Product Line Engineering

Development for reuse

Development with reuse

K. Pohl, G. Böckle, and F. van der Linden. Software Product Line Engineering: Foundations, Principles and Techniques. Springer-Verlag New York, Inc., 2005. SOFSEM 2010

ES – Real-Time Systems Lab

State-of-the-Art: Feature Modeling

• Describe the variabilities and commonalities of an SPL • Many similar approaches with different notations

„Our“ feature model editor (generated with metacase tool MOFLON) SOFSEM 2010

ES – Real-Time Systems Lab

Running Example – Car Seat

SOFSEM 2010

ES – Real-Time Systems Lab

Outlook ‰ State-of-the-Art ƒ Software Product Line Engineering ƒ Black Box (Embedded) SW Testing ƒ Naïve Software Product Line Testing ‰

„Our“ Approach ƒ Integrated Software Product Line Testing ƒ Meta-Modeling and Model Transformations ƒ Summary and Future Work

SOFSEM 2010

ES – Real-Time Systems Lab

State-of-the-Art: Black Box Testing + … ‰ The Classification Tree Method (developed by Daimler) • Acyclic graph decomposing system function into input parameters • Equivalence classes of input parameter values (legal values, illegal values, stress test values, … ) • Heuristics for input parameter value selection

SOFSEM 2010

ES – Real-Time Systems Lab

Classification Tree of Running Example

CTE/ES © (RAZORCAT)

• The car is not moving and its door is opened • The seat moves back due to the automatic entry function • The seat has moved back and the driver enters the car • The driver has entered the car and closes the door • The seat moves back to its initial position and stops SOFSEM 2010

ES – Real-Time Systems Lab

State-of-the-Art: Black Box Testing + … ‰ The Classification Tree Method (developed by Daimler) • Acyclic graph decomposing system function into input parameters • Equivalence classes of input parameter values (legal values, illegal values, stress test values, … ) • Heuristics for input parameter value selection

‰ Model-Based Testing Techniques • Executable Simulink / UML / Ptolemy / … model • Generation of tests based on model coverage criteria • Usage of model checking techniques •

SOFSEM 2010



ES – Real-Time Systems Lab

Ptolemy II © Simulation Environment © developed at Berkely

Sensors Motors

Seat Position

Enactable Models SOFSEM 2010

ES – Real-Time Systems Lab

Ptolemy II Simulation Environment

Seat automatic Seat Driver Door moves Idle opened entered forward finished SOFSEM 2010

ES – Real-Time Systems Lab

Outlook ‰ State-of-the-Art ƒ Software Product Line Engineering ƒ Black Box (Embedded) SW Testing ƒ Naïve Software Product Line Testing ‰

„Our“ Approach ƒ Integrated Software Product Line Testing ƒ Meta-Modeling and Model Transformations ƒ Summary and Future Work

SOFSEM 2010

ES – Real-Time Systems Lab

Software Product Line Testing

‰ State-of-the-Art: • Product by product • Regression tests • …

‰ Feature Models & Classification Trees • • • •

SOFSEM 2010

Select representative subset of all feature combinations Use heuristics for selection of critical feature combinations Define „classification tree“-based black box tests Combine with model-based software development

ES – Real-Time Systems Lab

„Naïve“ SPL Black Box Testing Approach

1. Construct software product line feature model 2. Normalize feature model & select product instance 3. Derive classification tree for selected product 4. Define suite of test cases for classification tree

1. SOFSEM 2010

2.

3.

4. ES – Real-Time Systems Lab

1. Construct Feature Model (FM)

A a

b c

A

C

B

f

e

a

b

B

A

C

B

C

f d

d

A

C

[...] [...] [...]

[...] [...]

[...] [...]

[...] [...] [...]

[...] [...]

[...] [...]

require exclude

SOFSEM 2010

[...] [...]

[...] [...] [...]

[...] [...]

[...] [...] [...]

ES – Real-Time Systems Lab

2. Select Product Instance (PI)

SOFSEM 2010

ES – Real-Time Systems Lab

3. Derive Classification Tree (CT)

A a

b c

A

C

B

f

e

a

b

B

A

C

B

C

f d

d

A

C

[...] [...] [...]

[...] [...]

[...] [...]

[...] [...] [...]

[...] [...]

[...] [...]

require exclude

SOFSEM 2010

[...] [...]

[...] [...] [...]

[...] [...]

[...] [...] [...]

ES – Real-Time Systems Lab

4. Define Test Cases (TC)

A a

b c

A

C

B

f

e

a

b

B

A

C

B

C

f d

d

A

C

[...] [...] [...]

[...] [...]

[...] [...]

[...] [...] [...]

[...] [...]

[...] [...]

require exclude

SOFSEM 2010

[...] [...]

[...] [...] [...]

[...] [...]

[...] [...] [...]

ES – Real-Time Systems Lab

„Naïve“ SPL Black Box Testing: Summary

SOFSEM 2010

ES – Real-Time Systems Lab

Outlook ‰ State-of-the-Art ƒ Software Product Line Engineering ƒ Black Box (Embedded) SW Testing ƒ Naïve Software Product Line Testing ‰

„Our“ Approach ƒ Integrated Software Product Line Testing ƒ Meta-Modeling and Model Transformations ƒ Summary and Future Work

SOFSEM 2010

ES – Real-Time Systems Lab

FMT – Integrated FM & CT Testing Approach ‰ Feature models and classification trees are popular ‰ Both languages share a substantial subset of constructs ‰ Both languages may profit from each other ‰ Combination supports integrated feature & test case definition

+ Feature Models + Classification Trees SOFSEM 2010

= =

FMT ES – Real-Time Systems Lab

FMT Language Constructs

SOFSEM 2010

ES – Real-Time Systems Lab

Advantages of FMT Approach “Naïve” Approach

FMT-based Approach SOFSEM 2010

ES – Real-Time Systems Lab

FMT of the Running Example : selection at construction time

SPL

: selection at runtime

Features

… Functions, Sensors, … Param Values

SOFSEM 2010

ES – Real-Time Systems Lab

Model-Based Software Product Line Testing

SOFSEM 2010

ES – Real-Time Systems Lab

Outlook ‰ State-of-the-Art ƒ Software Product Line Engineering ƒ Black Box (Embedded) SW Testing ƒ Naïve Software Product Line Testing ‰

„Our“ Approach ƒ Integrated Software Product Line Testing ƒ Meta-Modeling and Model Transformations ƒ Summary and Future Work

SOFSEM 2010

ES – Real-Time Systems Lab

Implementation with MetaCase Tool MOFLON

Version 1.4

MOF MOF2.x 2.x Metamodel Metamodel Def. Def.Language Language TU Darmstadt

SOFSEM 2010

ES – Real-Time Systems Lab

FMT Metamodel (MOF 2.0)

SOFSEM 2010

ES – Real-Time Systems Lab

Implementation with MetaCase Tool MOFLON

Version 1.4

MOF MOF2.x 2.x Metamodel Metamodel Def. Def.Language Language TU Dresden

TU Darmstadt

OCL OCL2.x 2.x Constraint Constraint Def. Def.Language Language

SDM SDM Fujaba FujabaGraph Graph Transformations Transformations University of Kassel University of Paderborn

SOFSEM 2010

ES – Real-Time Systems Lab

Feature Model Transformation

requires

root ∧ ( A → root ) ∧ (a → A) ∧ ( A → a ) = root ∧ (¬A ∨ root ) ∧ (¬a ∨ A) ∧ (¬A ∨ a ) = root ∧ [(¬a ∧ ¬A) ∨ ( A ∧ a )]

=

root ∧ ( A → root) ∧ (a → root) ∧ (a → A) ∧ ( A → a) =root ∧ (¬A ∨ root) ∧ (¬a ∨ root) ∧ (¬a ∨ A) ∧ (¬A ∨ a) =root ∧ [( ¬a ∧ ¬A) ∨ ( A ∧ a)]

Verification of model transformation SOFSEM 2010

ES – Real-Time Systems Lab

Feature Model Transformation II

SOFSEM 2010

ES – Real-Time Systems Lab

SDM Implementation of FMT Transformation - 1

SOFSEM 2010

ES – Real-Time Systems Lab

SDM Implementation of FMT Transformation - 2

SOFSEM 2010

ES – Real-Time Systems Lab

SDM Implementation of FMT Transformation - 3

SOFSEM 2010

ES – Real-Time Systems Lab

Summary and Challenges ‰ Summary ƒ Integrated feature and test case modeling approach ƒ Heuristics (pairwise combination) for test candidate selection ƒ Heuristics (boundary tests, … ) for test case selection ƒ Prototype development with metacase tool MOFLON ‰ Future Work ƒ Characterization of independent/interacting features (data flow analysis of related code fragments) ƒ Evaluation using real-world examples ƒ Integration with RE, design, … activities ƒ … SOFSEM 2010

ES – Real-Time Systems Lab

Future Work – FMT & RE Integration (Un)Parser Definition

MOF Meta Model

CodeAnalysis Fragment Results

Model Analysis & Refactoring FMT Modell File

(Un)Parser API Adapter

OCL Constraints

FMT Model

Model Synchronizer

SDM Graph Transformation generates

CodeRE Fragment File

SOFSEM 2010

(Un)Parser API Adapter

RE Model

(Un)Parser Definition

MOF Meta Model

TGG (QVT-like) Translations

ES – Real-Time Systems Lab

Implementation with MetaCase Tool MOFLON Monticore Monticore Parser, Parser,… … RWTH Aachen

Version 1.4

MOF MOF2.x 2.x Metamodel Metamodel Def. Def.Language Language TU Dresden

TU Darmstadt

OCL OCL2.x 2.x Constraint Constraint Def. Def.Language Language

TU Darmstadt SOFSEM 2010

SDM SDM Fujaba FujabaGraph Graph Transformations Transformations TGG TGG(QVT-like) (QVT-like) M2M M2MTranslation Translation Def. Def.Language Language

University of Kassel University of Paderborn

ES – Real-Time Systems Lab

Thank you for your attention !!!

MOFLON 1.4

SOFSEM Participant

See also: http://www.moflon.org/ SOFSEM 2010

ES – Real-Time Systems Lab

Model-Driven Software Development Research 6. 6.Model-Based Model-Based&& Product Product Line LineTesting Testing

Sebastian Oster [TUD,BMBF] Harald Cichos [TUD]

[CASED, TUD]

5. 5. Model-Driven Model-Driven Security Security Engineering Engineering

4. 4. Software SoftwareModernization Modernization

Application Application Areas: Areas: Sebastian Rose [EAG,SAP] Felix Klar [TUD,Daimler]

Automotive Automotive SW SW

Lars Patzina [LOEWE] Sven Patzina [TUD] Gergely Varro [AvH]

Automation Automation SW SW … …

2. 2. Modeling ModelingLanguage Language&& Tool ToolIntegration Integration&&P2P P2P

Patrick Mukherjee [DFG] NN [DFG]

SOFSEM 2010

3. 3.Domain-Specific Domain-SpecificEng. Eng. Languages Languages&&Methods Methods

1. 1. Meta-Models Meta-Models && Model Model Transformations (OMG) Transformations (OMG)

Elodie Legros [BMBF] Martin Wieber [BMBF] Antony Anjorin [DFG,Siemens] Marius Lauder [DFG,Siemens] ES – Real-Time Systems Lab