The UML Testing Profile

The UML Testing Profile - Tutorial at the ECOOP 2004 www.fokus.fraunhofer.de/u2tp Ina Schieferdecker, Technical University Berlin/FOKUS Øystein Haugen, University of Oslo June 2004 © U2TP Consortium

Agenda Motivation Basics of Testing Overview of the UML Testing Profile Example Test Specifications The UML Profile and the MOF Model The Mappings Concluding Remarks

ECOOP, Oslo, June 2004

2

© U2TP Consortium

1

Agenda


Motivation Basics of Testing Overview of the UML Testing Profile Example Test Specifications The UML Profile and the MOF Model The Mappings Concluding Remarks

ECOOP, Oslo, June 2004

3

© U2TP Consortium

The Problem Software Increases in complexity, concurrency, and dynamics Quality is key • Functionality • Performance • Scalability • Reliability • Usability • Efficiency • Maintainability • ...


Testing is Means to obtain objective quality metrics about systems in their target environment Central means to relate requirements and specification to the real system

ECOOP, Oslo, June 2004

4

© U2TP Consortium

2

Testing Today Is

  

But often

    

Important Means to obtain approval Time critical



Rarely practiced Unsystematic Performed by hand Error-prone Considered being destructive Uncool „If you are a bad programmer you might be a tester“

Conjecture: There is a lack of appropriate test methods and techniques ECOOP, Oslo, June 2004

5

© U2TP Consortium

Testing is … A technical process Performed by experimenting with a system In a controlled environment following a specified procedure With the intent of observing one or more characteristics of the system By demonstrating the deviation of the system’s actual status from the required status/specification.

ECOOP, Oslo, June 2004

6

© U2TP Consortium

3

Integrated Development and Testing

Developer

Testing throughout the process

Integrator

Heterogeneity increases Systems Integrator ECOOP, Oslo, June 2004

7

© U2TP Consortium

Extreme View

Developer Testing tight to Development e.g. Java Integrator Testing tight to Specification e.g. TTCN-3 Systems Integrator ECOOP, Oslo, June 2004

8

© U2TP Consortium

4

Balanced View

Developer Testing tight to Development e.g. JUnit Integrator Testing tight to Services e.g. TTCN-3

Systems Integrator ECOOP, Oslo, June 2004

9

© U2TP Consortium

An Answer: Model-Based View

use case diagrams

Developer

T e le p h o n e C a t a lo g

C h e c k S tatu s S ales p e r so n P la c e O rd e r C u s to m e r

class diagrams

S h ip p in g C le rk

F ill O r d e r s

E s t a b l is h C r e d it

Testing tight to Development

S u p e r v is o r

c a rt

c lie n t

S h o p p in g C a rt

C u s to m e r

c u s t o m e r O rd e r

c a r tO r d e r

1

1 1 ..*

0 ..1

1

Integrator

A cc ou nt

ac co untO rder

1 ..*

1

o rd e r

e.g. JUnit

{x o r}

O rd e rH e a d e r

0 ..*

1

o rg a n iz a ti o n O rd e r

a cc oun t

O rg a n iz a tio n c lie n t

ite m

* L i n e I te m

interactions

s e rv ic e Ite m

s e rv i c e

S e rv i c e

sd N

1

1 {x o r}

1

state machines p ro d u c tIt e m

produ ct

1

R e a d A m o u n tS M

P ro d u c t

a b o rt

s[u]:B

s[k]:B

s e le c tA m o u n t

Testing tight to Specification

o th e rA m o u n t

m3() m3()

ECOOP, Oslo, June 2004

e.g. TTCN-3

am ount

Systems Integrator

e n te rA m o u n t ok

10

a b o rt a b o rte d

© U2TP Consortium

5

UML and Testing UML-based test generation, e.g. Integration testing, Siemens, 2000 Component test framework (TINA), FOKUS, 2000 Component testing (COTE), INRIA, 2001

UML-based test notation, e.g. Agedis, EC IST project TeLa, COTE project UML Testing Profile, OMG

ECOOP, Oslo, June 2004

11

© U2TP Consortium

UML and Testing Model Driven Architecture as new OMG strategy One objective of UML 2.0 is executable UML meaning • Code generation • Simulation • Validation • Test generation “..., the expanded role of the OMG must be built on rock-solid testing, certification and branding. …“

ECOOP, Oslo, June 2004

12

© U2TP Consortium

6

Goals of the UML Testing Profile Definition of a testing profile to capture all information that would be needed by different test processes To allow black-box testing (i.e. at UML interfaces) of computational models in UML

A testing profile based upon UML 2.0 That enables the test definition and test generation based on structural (static) and behavioral (dynamic) aspects of UML models, and That is capable of inter-operation with existing test technologies for black-box testing

Define Test purposes for computational UML models, which should be related to relevant system interfaces Test components, test configurations and test system interfaces Test cases in an implementation independent manner

ECOOP, Oslo, June 2004

13

© U2TP Consortium

U2TP Partners A consortium of testers, UML vendors and users dedicated to make UML applicable for software testing Submitters

Supporters

Ericsson IBM FOKUS Motorola Rational Softeam Telelogic University of Lübeck

ECOOP, Oslo, June 2004

iLogix ScapaTechnologies IRISA

14

© U2TP Consortium

7

Agenda Motivation


Basics of Testing Overview of the UML Testing Profile Example Test Specifications The UML Profile and the MOF Model The Mappings Concluding Remarks

ECOOP, Oslo, June 2004

15

© U2TP Consortium

Types of Testing

Level system system integration unit functionality functionality load/performance load/performance robustness interoperability interoperability usability …

box white box black box grey box

Accessibility

Aspect

ECOOP, Oslo, June 2004

16

© U2TP Consortium

8

Test Concepts: Black-Box Testing

Test Case

Stimulus

Response • Assignment of a Test Verdict Port

System Under Test (SUT) ECOOP, Oslo, June 2004

17

© U2TP Consortium

Test Concepts: Local behavior Stimuli are given to the system under test (SUT) at well-defined interfaces Observations are taken and compared to the expected ones – the observations are validated and arbitrated The arbitration decides upon the further testing and/or the assignment of test results – the verdicts A detailed test procedure consisting of various stimuli and observations is called test case Several test cases constitute a test suite The relation between test cases and their execution is defined in a test control Typically, a test case describes a tree structure: one stimuli – several possible observations Alternatives and defaults are used for an efficient description of such trees ECOOP, Oslo, June 2004

18

© U2TP Consortium

9

Test Concepts: Distributed Behavior Typically, distributed SUTs require distributed test setups Such test setups are realized via test components which have to be coordinated and synchronized All test components have a local verdict, which contributes to the overall verdict The initial test configuration defines the test components used initially for a test case – the test configuration can change dynamically Test components may use utility parts to use/access information outside the test system

ECOOP, Oslo, June 2004

19

© U2TP Consortium

Test Concepts: Data Both, stimuli and observation are described as test data Test data for stimuli and observations can be unspecific in their properties and values – data pools and data partitions are used In addition, test data for observations can be unspecific in certain elements – wildcards are used Coding rules describe the encoding/decoding used on interfaces to the SUT

ECOOP, Oslo, June 2004

20

© U2TP Consortium

10

Agenda Motivation Basics of Testing


Overview of the UML Testing Profile Example Test Specifications The UML Profile and the MOF Model The Mappings Concluding Remarks

ECOOP, Oslo, June 2004

21

© U2TP Consortium

The Testing Profile Roots

• Arbiter • Validation actions • Data pools

UML Testing Profile Graphical Format of TTCN-3

UML 2.0

MSC-2000

ECOOP, Oslo, June 2004

UML 1.x

22

SDL-2000

Software Testing like JUnit, TET, etc.

Protocol Testing like TTCN-3

• Test control • Wildcards • Defaults • Test components

© U2TP Consortium

11

1st Root: TTCN-3 The new standardised test specification and test implementation language Developed from 1999 – 2002 at the European Telecommunications Standards Institute (ETSI)

Developed based on experiences from previous TTCN editions Removal of OSI specific concepts; Improvement of concepts; Introduction of new concepts

Applicable for all kinds of black-box testing for reactive and distributed systems, e.g., Telecom systems (ISDN, ATM, GSM, UMTS); Internet (IP, IP based protocols and applications); Software systems (Java, XML); Middleware platforms and component-based systems (CORBA, .Net, EJB) ECOOP, Oslo, June 2004

23

© U2TP Consortium

TTCN-3

msc mi_synch1_conc1

ASN.1 Types & Values

mtc

TTCN-3 Core Notation

ISAP1

MSAP2

Tabular Format

Other IDL Types & Values 2 : testcase myTestcase () runs on MTCType system TSIType { mydefault := activate (OtherwiseFail); verdict.set(pass); XML : connect(PTC_ISAP1:CP_ISAP1,mtc:CP_ISAP1); : map(PTC_ISAP1:ISAP1, system:TSI_ISAP1); : Other Types PTC_ISAP1.start(func_PTC_ISAP1()); C, C++, & Values n PTC_MSAP2.start(func_PTC_MSAP2()); JAVA Synchronization(); all component.done; log(”Correct Termination”);

Graphical Format

Presentation UML Format n

Testing Profile

}

: ECOOP, Oslo, June 2004

24

© U2TP Consortium

12

2nd Root: UML 2.0 Developed by OMG (Object Management Group) 1999-2004, adoption June 2003, available 2004 UML 2.0 Infrastructure RFP • metamodel restructuring in order for Core to be reusable by other OMG languages UML 2.0 Superstructure RFP • new and improvement/extension of UML concepts UML 2.0 OCL RFP • defining an OCL metamodel UML 2.0 Diagram Interchange RFP • ensuring diagram interchange between different tools

ECOOP, Oslo, June 2004

25

© U2TP Consortium

UML 2.0 Improvements More unified conceptual base Parts in Internal structure, Collaborations, Use cases and indirectly in Interactions

More unified semantics Higher precision

Improved expressiveness Structured Classes, Sequence Diagrams and Statemachines Activities merged with actions Collaborations aligned with structured classes Patterns (templates) and frameworks support


More powerful and expressive than UML 1.4
Tighter and more consistent than UML 1.4
Executable UML becomes possible ECOOP, Oslo, June 2004

26

© U2TP Consortium

13

UML 2.0 Profiles Use of UML in

      

UML has many “semantic-free

Analysis Design/implementation Directly executable notation (eg xUML) Architecture description Process engineering, workflow Website structures Data Modeling

zones”, so called “semantic variation points”



E.g. detailed semantics of state machines, ...


Profiles



Specializations of UML by stereotypes, providing special semantics

with obviously different (and inconsistent) semantics

ATM

ECOOP, Oslo, June 2004

27

© U2TP Consortium

UML 2.0 Profile Walkthrough (1) Define profile(s) based on reference metamodel may use other packages for its definition

Extension

Metamodel

«profile»

U2TP

UML «reference» «profile»

U2TP

ECOOP, Oslo, June 2004

JavaTypes «reference»

«metaclass» StructuredClassifier

«stereotype» TestComponent zone: TimeZone[0..1]

«import»

«profile»

Java

28

© U2TP Consortium

14

UML 2.0 Profile Walkthrough (2) Specify model based on UML metamodel

A class definition BankTest

IHardware

HWEmulator -pinOk : Boolean -enteredPIN : String -message : String

hwCom

ECOOP, Oslo, June 2004

IATM

29

© U2TP Consortium

UML 2.0 Profile Walkthrough (3) Apply profile(s) to model
make it possible to apply stereotypes of the profile to the model elements «profile»

U2TP

«apply» BankTest

IHardware

HWEmulator -pinOk : Boolean -enteredPIN : String -message : String

ECOOP, Oslo, June 2004

hwCom IATM

30

© U2TP Consortium

15

UML 2.0 Profile Walkthrough (4)

Apply stereotypes to model elements as desired «profile»

U2TP

«apply» BankTest

testComponent»

«

IHardware

HWEmulator

-pinOk : Boolean -enteredPIN : String -message : String -

hwCom IATM

t1 : Timer

ECOOP, Oslo, June 2004

zone = Stuttgart

31

© U2TP Consortium

Concepts of the Testing Profile Test architecture Test structure, test components and test configuration

Test data Data and templates used in test procedures

Test behavior Dynamic aspects of test procedures

Test time Time quantified definition of test procedures

ECOOP, Oslo, June 2004

32

© U2TP Consortium

16

Test Architecture Realization System Under Test (SUT) Test components Test context with test configuration and test cases Test verdict arbitration with arbiter Test coordination with scheduler

Test Data Realization Individual coding rule definition Wildcards * and ? Concrete test data with data pool, data partition and data selector

ECOOP, Oslo, June 2004

33

© U2TP Consortium

Test Behavior Realization Test objectives Test cases Test verdicts: pass, fail, inconclusive Defaults behaviors on different levels Utility part

Test Time Realization Clock Timezone definition for synchronizing test components Timer operations

ECOOP, Oslo, June 2004

34

© U2TP Consortium

17

Concepts beyond TTCN-3 Unification of test cases: Test case as a composition of test cases Test behavior defines the execution of a test case

Separation of test behavior and verdict handling Arbiter is a special component to evaluate the verdict Validation actions are used to set the verdict

Abstract test cases that work on data partitions rather than individual data Data partitions to describe value ranges for observations and stimuli

Test architecture with test deployment support Part of the test specification is the definition of deployment requirements for a test case

ECOOP, Oslo, June 2004

35

© U2TP Consortium

Concepts beyond UML Defaults within test behavior Concentration on main flow of test behavior Default hierarchy to handle different concerns

Wildcards within test data Flexible definition of value sets

Timers and time constraints Time controlled test behavior

Arbitration and verdicts Assessment of test behavior

Coding attributes Encoding/decoding for data exchange with the SUT

ECOOP, Oslo, June 2004

36

© U2TP Consortium

18

Agenda Motivation Basics of Testing Overview of the UML Testing Profile


Example Test Specifications The UML Profile and the MOF Model The Mappings Concluding Remarks

ECOOP, Oslo, June 2004

37

© U2TP Consortium

The Example OTC Market Makers Clearing Company SWIFTBureau

SWIFTNet SWIFTBureau

US Bank Network EU Bank Network

US Bank SSSB Client

ECOOP, Oslo, June 2004

EU Bank SSSB Client

38

© U2TP Consortium

19

UML Model of the System Level Archicture

ATM

«import»

HWControl

«import»

«import»

Bank

Money

System Integration Test System Test Unit Test

«import» «import»

SWIFTNetwork

ECOOP, Oslo, June 2004

39

© U2TP Consortium

Unit-Level Test

ATM

«import»

HWControl

«import»

Bank

«import»

Money

«import» «import»

SWIFTNetwork

ECOOP, Oslo, June 2004

40

© U2TP Consortium

20

Unit-Level Test Money

IMoney + fAmount : Integer + fCurrency : String + equals(m:Object): Boolean

Money

MoneyBag

+ fAmount : Integer + fCurrency : String

fAmount : Integer fCurrency : String

+ Money(a: Amount, c:String) + add(m: Money): IMoney

+ contains(m: IMoney): Boolean + add(m: IMoney): IMoney

Classes to be tested ECOOP, Oslo, June 2004

41

© U2TP Consortium

A Unit-Level Test Suite „Test“ package Unit-level test suite MoneyUnitTest

«TestContext» « » MoneyTest



Money Bank addSameMoney() addSameMoney():Verdict addDifferentMoney() addDifferentMoney():Verdict

Unit-level test cases

ECOOP, Oslo, June 2004

42

© U2TP Consortium

21

A Test Case

Test suite object performing the test case

sd addSameMoney():Verdict

Class instances of the SUT

self

Money(20, ”USD”)

«sut» money1:Money

«sut» money2:Money

Money(50, ”USD”)

add(money2) get: money2

add (-) : new Money (70, ”USD”)

return pass

Return of test verdict

ECOOP, Oslo, June 2004

43

© U2TP Consortium

System-Level Test

ATM

«import»

HWControl

«import»

Bank

«import»

Money

«import» «import»

SWIFTNetwork

ECOOP, Oslo, June 2004

44

© U2TP Consortium

22

System Level Test System-level test suite with public and private test cases

ATMTest

«testContext» ATMSuite

«testComponent» HWEmulator

-verdict : Verdict -amount : IMoney -targetBank : SwiftId -targetAccount : String -sourceAccount : String

-pinOk : Boolean -enteredPIN : hwCom String -message : String -t1 : Timer

Test component

IHardware

IATM «import»

«testCase» +validWiring() : Verdict «testCase» +invalidPIN() : Verdict «testCase» -authorizeCard() : Verdict

ATM

IBank

-accounts

«testComponent» BankEmulator

*

«interface» IAccount

ECOOP, Oslo, June 2004

Miscellaneous classes 45

© U2TP Consortium

Test Configuration Coding rule part

SUT part «testContext» class ATMSuite

coding ”Encrypted”

«sut» atm : BankATM

be : BankEmulator bankCom

atmPort

hwe : HWEmulator

Test component parts ECOOP, Oslo, June 2004

current : CardData

Utility part 46

© U2TP Consortium

23

Test Control (execution of test suite) Referring test case behaviors

sd ATMSuite

ref

verdict = invalidPIN

[verdict == pass] ref

ECOOP, Oslo, June 2004

[verdict == fail]

verdict = validWiring

47

© U2TP Consortium

Data partition

A Test Case sd invalidPIN

{readOnly} Integer invalidPIN; { current.isPinCorrect(invalidPIN) == false } «sut» atm

hwe

Setting a timer

current

Stimulus storeCardData(current)

Stopping a timer

t1(2.0) display(”Enter PIN”)

Observation

t1 isPinCorrect(invalidPIN)

Duration constraint

{0 .. 3}

isPinCorrect(invalidPIN) isPinCorrect : false

isPinCorrect : false display(”Invalid PIN”)

Setting arbitrated verdict ECOOP, Oslo, June 2004

display(”Enter PIN again”)

Function return value

«validationAction» pass

48

© U2TP Consortium

24

A Test Case with Default (extracts) sd validWiring «sut» atm

hwe atmPort

Default application

bankCom

ref amount = acceptMoney wireMoney(

default DisplayDefault

display(”Transaction Accepted”) selectOperation : true

«validationAction» pass

ECOOP, Oslo, June 2004

49

© U2TP Consortium

Defaults Defining an eventspecific default

Applying a componentspecific default «testComponent » HWEmulator

sd DisplayDefault default HWEmulator::hweDefault

self

alt

-pinOk : Boolean -enteredPIN : String hwCom -message : String -t1 : Timer

IHardware

IATM

display(*)

hweDefault «validationAction» inconc

Wildcards ?

«validationAction» fail

ECOOP, Oslo, June 2004

ejectCard / setverdict(fail)

t1 / setverdict(fail);

*

Defining a componentspecific default 50

display(msg) / if (msg == ”Connection lost”) then setverdict(inconc); else setverdict(fail);

© U2TP Consortium

25

System-Integration-Level Test

ATM

«import»

HWControl

«import»

«import»

Bank

Money

«import» «import»

SWIFTNetwork

ECOOP, Oslo, June 2004

51

© U2TP Consortium

System-Integration-Level Tests Load tests

SWIFTTest «testContext» SWIFTSuite -numUsers:Integer = 0 -pc:Float

«testCase» -runUSTrxn(p:USTrxnData):Verdict «testCase» -runEUTrxn(p:EUTrxnData):Verdict «testCase» -Wiring():Verdict «testCase» +loadTest (maxUsers:Integer,p:Float): Verdict

«interface» Arbiter

«import»

«testComponent» TransactionController -initUSbal: IMoney -initEUbal: IMoney

SwiftNetwork

IHardware

IATM IAccount «import»

Bank default TransactionController::tcDefault «import»

ATM

LoadArbiter

«testComponent» loadManager

-numPass:Integer -numOther:Integer

«import»

TestData

completed()

ECOOP, Oslo, June 2004

User-defined arbiter 52

© U2TP Consortium

26

Test Configuration

SUT parts «testContext» « »

SWIFTSuite swiftPort

«sut» euBank : Bank bankPort

accountPort

swiftPort

«sut» network : SWIFTNetwork us

eu

«sut» usBank : Bank bankPort

bankPortc

accountPort

bankPort bankCom

bankCom

«sut» usATM: ATM

«sut» euATM: ATM

atmPort

atmPort

«testComponent» tc: transactionController

Test component parts

Utility part «testComponent» «testComponent » lm: loadManager

dp: dataPool

ECOOP, Oslo, June 2004

la: LoadArbiter

53

© U2TP Consortium

Test Data TestData :EUTrxnData[1]

TrxnData *

account = ”Fred Bloggs” balance = 10,000 amount = 3500 cardData = Card1

account : String balance: Integer amount: IMoney cardData: CardData

:EUTrxnData[2]