DLR.de • Chart 1
>Model-Based Testing Approach for MATLAB/Simulink using System Entity Structure and Experimental Frames> Dr.Umut Durak, > 14 April 2015
Model-Based Testing Approach for MATLAB/Simulink using System Entity Structure and Experimental Frames Artur Schmidt University of Wismar Research Group CEA
Umut Durak German Aerospace Center (DLR) Institute of Flight Systems
Christoph Rasch University of Wismar Research Group CEA
Thorsten Pawletta University of Wismar Research Group CEA
[email protected]
[email protected]
[email protected]
[email protected]
DLR.de • Chart 2
>Model-Based Testing Approach for MATLAB/Simulink using System Entity Structure and Experimental Frames> Dr.Umut Durak, > 14 April 2015
Agenda
1. Motivation 2. Model Based Testing in MATLAB/Simulink
3. Proposed Approach 4. Case Study
5. Conclusion
DLR.de • Chart 3
>Model-Based Testing Approach for MATLAB/Simulink using System Entity Structure and Experimental Frames> Dr.Umut Durak, > 14 April 2015
1. Motivation
Simulation V&V and Testing • The advancements in modeling tools and infrastructures have enabled us to construct and execute more complex models. • Verification and Validation (V&V) of these models still remains an important problem
• Simulation Testing: simulating a model under various conditions and comparing its behavior with the system it represents • Consider the infinite number of scenarios that need to be simulated to test a model against the system it represents
DLR.de • Chart 4
>Model-Based Testing Approach for MATLAB/Simulink using System Entity Structure and Experimental Frames> Dr.Umut Durak, > 14 April 2015
1. Motivation
Requirements of Simulation Testing • Adaptability, flexibility and automation • to minimize the huge amount of effort required for testing
DLR.de • Chart 5
>Model-Based Testing Approach for MATLAB/Simulink using System Entity Structure and Experimental Frames> Dr.Umut Durak, > 14 April 2015
2. MBT in MATLAB/Simulink
Model Based Testing • Model Based Testing (MBT) is defined as automating test case generation from a test model • Widely used in software testing community • Limited application in modeling and simulation
DLR.de • Chart 6
>Model-Based Testing Approach for MATLAB/Simulink using System Entity Structure and Experimental Frames> Dr.Umut Durak, > 14 April 2015
2. MBT in MATLAB/Simulink
MATLAB/Simulink Popular Model Based Design and simulation environment for cyberphysical systems from MathWorks, Inc.
DLR.de • Chart 7
>Model-Based Testing Approach for MATLAB/Simulink using System Entity Structure and Experimental Frames> Dr.Umut Durak, > 14 April 2015
2. MBT in MATLAB/Simulink
Previous Efforts • Model-in-the-Loop for Embedded System Test – Test Harness (MiLEST) (Zander) • Provides well-structured libraries for test data generation, test control and test validation functions. • The SUT and the test specification were both designed in MATLAB/Simulink. • Predefined test patterns introduced a systematic test design as a MATLAB/Simulink model on the same layer of the SUT.
• Simulink Verification and Validation • Realization of MBT in MATLAB/Simulink. • Provides library blocks that target test functions.
DLR.de • Chart 8
>Model-Based Testing Approach for MATLAB/Simulink using System Entity Structure and Experimental Frames> Dr.Umut Durak, > 14 April 2015
3. Proposed Approach
Model Based Simulation Testing • A model based testing approach that is based upon fundamental theories of modeling and simulation
DLR.de • Chart 9
>Model-Based Testing Approach for MATLAB/Simulink using System Entity Structure and Experimental Frames> Dr.Umut Durak, > 14 April 2015
3. Proposed Approach
Experimental Frame • Experimental Frame specifies a limited set of circumstances under which a model has to be observed EF = < T, I, O, C, Ωi, Ωc, SU > where: • T is the time base, • I is the set of input variables, • O is the set of output variables, • C is the set of control variables, • Ωi is the set of admissible input segments, • Ωc is the set of admissible control segments and • SU is a set of summary mappings.
DLR.de • Chart 10
>Model-Based Testing Approach for MATLAB/Simulink using System Entity Structure and Experimental Frames> Dr.Umut Durak, > 14 April 2015
3. Proposed Approach
Simulation Testing and Experimental Frames • MUT (Model Under Test) is the SUT • Test inputs are produced by the Generator. • The Acceptor and transducer form a test oracle. • The Transducer calculates outcome measures in the form of performance indices, comparative values, statistics etc. • The Acceptor corresponds to a decision unit that decides if an experiment is valid or not.
EF = < T, I, O, C, Ωi, Ωc, SU >
DLR.de • Chart 11
>Model-Based Testing Approach for MATLAB/Simulink using System Entity Structure and Experimental Frames> Dr.Umut Durak, > 14 April 2015
3. Proposed Approach
Modeling in Model Based Methodologies • Model is a physical, mathematical, or otherwise logical representation of a system, entity, phenomenon or process. • Descriptive • Describing the reality of a system or a context • Simulation Modeling Perspective
• Prescriptive • Determine the scope or details at which to study a problem • Defining how a system shall be implemented • Engineering Perspective
DLR.de • Chart 12
>Model Based Simulation Engineering> Dr.Umut Durak, • Presentation > 14 April 2015
3. Proposed Approach
The Equation of Model Based Methodologies
Models + Transformations = Software
Brambilla, Marco, Jordi Cabot, and Manuel Wimmer. "Model-driven software engineering in practice." Synthesis Lectures on Software Engineering 1.1 (2012): 1-182.
DLR.de • Chart 13
>Model-Based Testing Approach for MATLAB/Simulink using System Entity Structure and Experimental Frames> Dr.Umut Durak, > 14 April 2015
3. Proposed Approach
System - Model - Metamodel • Model is a simplification of a system built with an intended goal in mind • Metamodel is the explicit specification of an abstraction (a simplification)
Bézivin, Jean, and Olivier Gerbé. "Towards a precise definition of the OMG/MDA framework." Automated Software Engineering, 2001.(ASE 2001). Proceedings. 16th Annual International Conference on. IEEE, 2001. Bézivin, Jean. "On the unification power of models." Software & Systems Modeling 4.2 (2005): 171-188.
DLR.de • Chart 14
>Model-Based Testing Approach for MATLAB/Simulink using System Entity Structure and Experimental Frames> Dr.Umut Durak, > 14 April 2015
3. Proposed Approach
Metamodeling and Transformations • Source: Experimental Frame Ontology • Target: MATLAB/Simulink, the executable test model • Methodology: System Entity Structure and Model Base Framework
DLR.de • Chart 15
>Model-Based Testing Approach for MATLAB/Simulink using System Entity Structure and Experimental Frames> Dr.Umut Durak, > 14 April 2015
3. Proposed Approach
System Entity Structure and Model Base Framework • The SES is a high-level ontology for the specification of a set of system structures and parameter settings for modeling and simulation. • The SES is represented by a directed and labeled tree with links to Basic Models (BMs) in the Model Base (MB).
• Pruning is the operation in which a distinct system structure is derived from an SES. • The result is called Pruned Entity Structure (PES). • Translation is used to generate an executable simulation model (EM) based on the information of the PES and BMs from the MB.
DLR.de • Chart 16
>Model-Based Testing Approach for MATLAB/Simulink using System Entity Structure and Experimental Frames> Dr.Umut Durak, > 14 April 2015
3. Proposed Approach
Using SES/MB and EF
DLR.de • Chart 17
>Model-Based Testing Approach for MATLAB/Simulink using System Entity Structure and Experimental Frames> Dr.Umut Durak, > 14 April 2015
3. Proposed Approach
Pruning SES_Vars = {selG=step, selA=ltt, selT=mc, userTh=20}
SES
PES
DLR.de • Chart 18
>Model-Based Testing Approach for MATLAB/Simulink using System Entity Structure and Experimental Frames> Dr.Umut Durak, > 14 April 2015
3. Proposed Approach
Translation PES
EM
DLR.de • Chart 19
>Model-Based Testing Approach for MATLAB/Simulink using System Entity Structure and Experimental Frames> Dr.Umut Durak, > 14 April 2015
3. Proposed Approach
SES/MB in MATLAB/Simulink
SES
MB
DLR.de • Chart 20
>Model-Based Testing Approach for MATLAB/Simulink using System Entity Structure and Experimental Frames> Dr.Umut Durak, > 14 April 2015
3. Proposed Approach
Pruning in MATLAB/Simulink
SES
PES
DLR.de • Chart 21
>Model-Based Testing Approach for MATLAB/Simulink using System Entity Structure and Experimental Frames> Dr.Umut Durak, > 14 April 2015
3. Proposed Approach
Translation in MATLAB/Simulink MB
EM
PES
Infrastructure - Overview
3. Proposed Approach
DLR.de • Chart 22
>Model-Based Testing Approach for MATLAB/Simulink using System Entity Structure and Experimental Frames> Dr.Umut Durak, > 14 April 2015
DLR.de • Chart 23
>Model-Based Testing Approach for MATLAB/Simulink using System Entity Structure and Experimental Frames> Dr.Umut Durak, > 14 April 2015
4. Case Study
Application in Robotics • Robotic Control & Visualization Toolbox for MATLAB/Simulink • for controlling of simulated and real robot arms • for simulation of robot arm movements, including handled parts in a virtual environment
• The validation of the simulation for robot arm movements
expected results are measured on a real robot using the same input segments.
DLR.de • Chart 24
>Model-Based Testing Approach for MATLAB/Simulink using System Entity Structure and Experimental Frames> Dr.Umut Durak, > 14 April 2015
4. Case Study
Test Scenario
DLR.de • Chart 25
>Model-Based Testing Approach for MATLAB/Simulink using System Entity Structure and Experimental Frames> Dr.Umut Durak, > 14 April 2015
4. Conclusion
Wrap-up • Model-Based Testing is adopted for effective and efficient testing of simulation models in MATLAB/Simulink • Test models are • specified using System Entity Structure • transformed into executable tests employing components from a Model Base that consists of basic blocks for Experimental Frames. • A case study from robotics domain • testing a robot arm simulation against real robot trajectory
DLR.de • Chart 26
>Model-Based Testing Approach for MATLAB/Simulink using System Entity Structure and Experimental Frames> Dr.Umut Durak, > 14 April 2015
4. Conclusion
Achievements
• A Model Base that encompasses Experimental Frame components constitutes a reusable asset library for model testing. • Adaptability • Flexibility.
• Transformation tool automatically generates executable test cases from a test specification model based on SES • Efficiency • Effectiveness
DLR.de • Chart 27
>Model-Based Testing Approach for MATLAB/Simulink using System Entity Structure and Experimental Frames> Dr.Umut Durak, > 14 April 2015
4. Conclusion
Future Work • Intended to be employed on a larger scale to validate flight models in the Air Vehicle Simulator (AVES) research facility of the German Aerospace Center (DLR) Institute of Flight Systems.
DLR.de • Chart 28
>Model-Based Testing Approach for MATLAB/Simulink using System Entity Structure and Experimental Frames> Dr.Umut Durak, > 14 April 2015
Model-Based Testing Approach for MATLAB/Simulink using System Entity Structure and Experimental Frames Questions and Answers Artur Schmidt University of Wismar Research Group CEA
Umut Durak German Aerospace Center (DLR) Institute of Flight Systems
Christoph Rasch University of Wismar Research Group CEA
Thorsten Pawletta University of Wismar Research Group CEA
[email protected]
[email protected]
[email protected]
[email protected]
