Sustainable Architectures Sustainable Architectures ...

Andreas Ulrich | SAGRA Workshop 2015

Sustainable Architectures – A Tester’s Perspective Unrestricted © Siemens AG 2015. All rights reserved

About myself

I’m Dr.-Ing. Andreas Ulrich • PhD in i Comp. C Science S i on specification ifi ti and d testing t ti off concurrentt systems t from f Magdeburg University, Germany in 1998 • Studying also at HKUST, Hong Kong • Joint Siemens Corporate Technology, Munich, Germany as a researcher • Cooperation with CRIM research institute, Montreal, Canada • Testing, monitoring, trace analysis, test generation and MBT • Involved in standardization of the test language TTCN-3 at ETSI • Initiator and main contributor for the design of the ETSI Test Description Language (TDL) for domain-specific test modeling • Experiences • Model-based testing and test automation • Domain-specific test modeling • System analysis, formal verification

Page 2

September 2015 Corporate Technology

Unrestricted © Siemens AG 2015. All rights reserved

Handzettel 1

Siemens milestones Across 170 years of history 1959 Simatic (electronic automation)

1879 Electric train 1847 Werner von Siemens founds the company

1939 Electron microscope

1881 Telephone switchboard

1850

1860

1870

1880

1890

1900

1910

1920

1930

2000 Wind turbine rotor blades in one cast

1962 Thyristors for energy transmission 1974 Computer tomograph

1953 High-purity silicon

1840

2000 Syngo user interface

1940

1950

1960

1970

1980

1990

2009 Most efficient gas turbine, 370 MW 2000

2010

1958 Heart pacemaker 1847 Telegraph

1866 Dynamo

Page 3

2013 PLM Software 1988 Megabit chip

1924 Traffic light 1881 Electric streetcar

1965 Integrated circuit

1935 Coaxial cable


2020

2010 Biograph mMR

1985 ICE – top speed 300 km/h

2015 Somatom Force


Siemens Corporate Technology’s global presence Partner to customers all over the world

St. Petersburg g Moscow

Princeton

Beijing

Berkeley

Shanghai

Moscow

Tokyo

Gurgaon Pune Bangalore Erlangen Munich

Prague Nuremberg g Bratislava Vienna Budapest Cluj

Brasov

Istanbul Country with CT facility

Page 4

> 500 employees


100 ‒ 500 employees

Other selected facilities Status May 2015 Unrestricted © Siemens AG 2015. All rights reserved

Handzettel 2

Outline

• Approaching the topic: Software architecture • Make architecture testable: Design for Testability (DFT) • Example: Validating Control of Process Automation Systems • Test architectures • Example: Application of TTCN-3 testing at Siemens Healthcare • Future trends • Conclusions

Page 6



Approaching the topic: Software architecture How a tester sees software architecture • SW architecture is a thought abstraction over code to structure it • Interfaces • Components • Distribution • Lowest level of abstraction (built into the programming language) • Function • Class • Method • Higher levels supported by a design process only • Get blurred over time • Semantics of architectural elements and their properties need to be captured

Page 7



Handzettel 3

Spot the architecture! Typical examples of dependency graphs for badly evolved software products

Software architecture recovery and reverse-engineering become a necessity! Page 8



Software architecture is important

• Represents design decisions and constraints on • Functional/non-functional requirements • System decomposition, interface definitions • Provides system context, scope and partitioning of the development work • Foundation for most other development artifacts • Serves as a vehicle for communication among all stakeholders • Developers, testers, managers, … • Is key for systematic re-use, change management and maintenance • Software product families and ecosystems • Iterative development and regression testing • Evolution of the software product

We better get SA right from the beginning!

• Hard to get changed later on

 Software architecture quality

Page 9



Handzettel 4

Development and test levels in the Vee diagram

based on

User Requirements

Acceptance Testing

System Requirements

System Testing

Architecture Design

Integration Testing

Detailed Design

Unit Testing Coding

Page 10



Verification & Validation of architecture design

based on

User Requirements

Acceptance Testing

System Requirements Reviews, inspections, TDD/BDD, Rapid prototyping, Simulation, formal methods, Architecture quality models

System Testing

Architecture Design

Integration Testing

Unit Specification

Interface tests, Capability tests, Conformance tests, Interoperability p y tests

Unit Testing Coding

Page 11



Handzettel 5

Make architecture testable: Design for Testability (DFT) DFT – A standard approach to hardware testing • Increasing complexity in chip design requires a standardized and systematic approach to production testing of digital circuitry • Automatic generation of test patterns is much simpler if certain DFT rules are followed  Impacts time to market!

• Main objective of DFT in microelectronics: Ensure that the system is assembled correctly from smaller units!

(BDTi: NVIDIA Tegra 2)

• Assumption: If the functionality of smaller units is correct and their assembly is correct, then the overall functionality is correct

Spot the architecture! • Carry DFT over to software systems

Page 12



Recap: Structure of a test case for software testing

• Test purpose • A description of a single requirement or a set of them that should be tested • Preamble • Drives the SUT from its stable initial state to its test state, where execution of the test body starts from • Test body • Implements the test purpose • Postamble • Drives the SUT from the state reached after execution of the test body (back) to the initial state; resets the SUT • State identification (optional) • Checks that the SUT reached the correct end state after execution of the test body • Test verdict • Assignment of verdicts (pass, fail, inconclusive) to the different possible responses of the SUT during test execution • From intermediates verdicts the final verdict of the test case is computed

Page 13


Initial State

Preamble Postamble

Test State

Test Body End State Test Body

[State identification] End State Identification


Handzettel 6

DFT for software systems Ingredients of software DFT • Component-based system architecture Test point

Test probe

Test probe

Test point

• Stable system interfaces accessible for testing • System interface to environment / user  test points (black-box) • Inter-component communication  test probes (grey-box) • System interfaces need to offer: • Controllability  A test case needs to control the execution of the system under test • Observability  A test case needs to observe (intra) system communication • Further properties of a software architecture • System initializes in a known stable initial state • System reset • Preset to known system states • System partitioning for integration testing

Page 14

End-to-end test in a GERAN mobile network with additional test probes

DFT is a property of the software architecture!



DFT: Observability and controllability

Observability

Controllability

• T Tester t observes b system t output t t to t reason about b t correctness t off an executed test case ( test verdict)

• T Tester t provides id stimuli ti li tto system t under d test t t to t drive d i test t t case execution

• Large systems cannot be validated at system/environment interface alone  additional test probes for intra-system communication

• Combined usually with observation of SUT responses and other output

•  Point of Observation (PO)

•  Point of Control and Observation (PCO)

SUT

Page 15

PO


Tester

SUT

PCO

Tester


Handzettel 7

DFT: Stable state vs. transient state

Stable state

Transient state

• S System t initialization, i iti li ti system t resett and d presett require i a stable state • Only stable end states of a test case can be identified and verified • Enables fault injection

• T Transient i t states t t occur by b chance h during d i execution, ti cannott be b reliably tested  Limits the achievable test coverage and fault detection capability • Issue mainly pertinent in testing concurrent systems

Transient system state

Stable system state S t System input i t #1

Next state #1

Page 16

S t System input i t #2

Next state #2

System input #1

System output #3

Next state #1


Next state #3


Example: Validating Control of Process Automation Systems

Process automation systems characteristics • Long operation time, e.g. >10 years of uninterrupted operation • High availability requested!  redundancy, fail-safe operation • Large scale systems • Up to 100 PLCs and more • Up to physical 100,000 I/O signals • Concurrent engineering; up to 10 engineering stations • Large lifespan of plant, >30 years • Lifespan of automation solution typically 10..15 years only • Step-wise migration during production necessary  sometimes mirrored facilities used for validation • High scalability, from 100 to 100,000 I/Os

Page 17



Handzettel 8

Example: Verification and test of fault-tolerant Simatic S7-400H PLC A distributed scheduler algorithm for redundant PLC operation in process automation systems • First version of S7-400H was released in 1998 • Siemens has captured about 25% of the market of high-performance redundant automation systems • S7-400H offers • Transparent redundancy w.r.t. programming model und configuration • Hot-standby with 50 msec failover time • Guaranteed process data continuity • S7-400H employs a distributed task scheduler as part of the RTOS • Local scheduling of tasks • Duplication of automation system events • Symmetric algorithm to determine next task Scheduling algorithm must guarantee certain properties  Correctness: both PLCs must agree on next task  Fairness: handling of low-priority events must not be starved  Subjected to formal verification! Page 18



Example: Modeling and verification of the distributed task scheduler Model-checker SPIN was the chosen verification tool • Approach • Reverse-engineering of relevant C-code for existing features • Manual translation from C-code to Promela and back • Extension of the model with new features • Issues • Identification of the interfaces: what is part of model, what is environment? • Identification of variables in C-code that contribute to the model state • Terminating task scheduler function enables a reset mechanism

stubs

PLC0 task_scheduler

PLC1 task_scheduler

stubs

• Final model: ~1000 LOC, 54m states

hs_exchange Page 19



Handzettel 9

Example: Validation of the implementation against the verified model Passive testing approach via a PO to observe data exchange between PLCs • C-code implementation needs to conform to the verified model to ensure interoperability between both PLCs

Task scheduler #0

• Issue: Deeply embedded code (part of RTOS)

Verified model

 Runtime verification of execution traces

Trace data

Converter

Intermediate format

SPIN simulator

OK

Monitor PO

 fail Comm. Trace Dump

The implementation of a monitor inside the PLC firmware enabled the validation of the task scheduler  DFT is essential! Page 20


Task scheduler #1


Test architectures

• Testing complex systems result frequently in complex test setups • A tester reflects the inverse behavior of the system under test PCO1

•  Gives rise to test architectures!

PO2

Tester

SUT PCO3

Siemens Ysio Digital x-ray clinical imaging device Page 21



Handzettel 10

Test specification and test execution language TTCN-3 Testing and Test Control Notation, version 3 • A test specification and test execution language • http://www.ttcn-3.org/ • Standardized by ETSI as ES 201 873 series • First version in 2000 • Stable and mature since 2003 • Until today still actively maintained • Contains built-in language features that are essential in testing • Support of alternative and concurrent system behavior • Matching mechanism for received system outputs • Test verdicts etc. • Applicable for all kinds of black-box testing for reactive and distributed / concurrent systems, e.g. • Telecom systems (ISDN, UMTS, LTE, …) • Internet applications (IoT, M2M, …) • Automotive (CAN, MOST, …) • SW platforms in general

TTCN-3 builds on the principles of DFT: PCO/PO, stable state, system reset

Page 22



TTCN-3 based test architecture A sustainable architecture for building test systems TE – TTCN-3 Executable • Compiled TTCN-3 sources

Test System User TM

CD

CH

TCI

TE

TRI

SA

PA SUT

Major standard parts: • ETSI ES 201 873-1: TTCN-3 Core Language • ETSI ES 201 873-5: TTCN-3 Runtime Interface (TRI) • ETSI ES 201 873-6: TTCN-3 Control Interfaces (TCI) Page 23


SA – System Adapter • Interface to the SUT (application dependent) PA – Platform Adapter • Integration with platform resources, e.g. timers CD – Coder/Decoder • Mapping of concrete/abstract data (application dependent) TM – Test Management • Test execution control CH – Component Handling • Manages creation/termination/distribution of tester components SUT – System Under Test


Handzettel 11

Example: Application of TTCN-3 testing at Siemens Healthcare Testing the collimators of the Ysio X-ray devices series  High safety requirements on software, electronics, and mechanics!

Collimator • Controls shape and density of X-rays • Up to 12 motor-driven, independent axles • Communicates over a CAN fieldbus

Page 24



Example: Testing of CAN-Bus Devices with TTCN-3 Testing components for clinical imaging devices TTCN-3 tester teststand

light emitter

SUT image field CCD camera

TTCN-3 enables fast and efficient test automation Page 25



Handzettel 12

Future trend: Domain-specific test modeling ETSI ES 203 119 Test Description Language (TDL) leverages model-based testing in agile projects • TDL enables multi-paradigmatic modeling • Adjustable to application domains and stakeholders • Supports multiple syntaxes Graphical syntax

Tabular syntax

Textual syntax

Test configuration, test description behavior

Test data value specification used in test description parameters

Default syntax; glue code

Page 26



Future trend: The rise of standardized reference architectures for IoT

1. 2. 3. 4. 5. 6. 7.

NIST: Cloud Computing p g Reference Architecture,, Sep. p 2011 ITU-T D2.1: M2M service layer: Requirements and architectural framework, Focus Group Technical Report, V 1.0, Apr. 2014 Open Interconnect Consortium: OIC Core Specification, V 0.9.1, 2015 Industrial Internet Consortium: Industrial Internet Reference Architecture, V 1.6, Jun. 2015 OneM2M: Functional architecture of the oneM2M Services Platform, Jan. 2015 IEEE P2413™/D0.0.2: Standard for an Architectural Framework for the Internet of Things (IoT), Mar. 2015 ...

Increasing demand for interoperable systems Page 27



Handzettel 13

Conclusions

• Sustainable software architectures embrace Design for Testability aspects • Software architects need to know how to verify, validate and test the architecture they design • TTCN-3 is a good example for sustainable architectures in testing • St Standardization d di ti h helps l b building ildi architectures hit t ffor interoperable systems – Get involved!

Page 28



Contact

Dr. Andreas Ulrich Siemens AG Corporate Technology Otto-Hahn-Ring 6 81739 München Germany

E-mail: [email protected]

Internet siemens.com/innovation

Page 29



Handzettel 14