Incremental Development of Model Transformation ... - Semantic Scholar

IBM Research

Incremental Development of Model Transformation Chains Using Automated Testing

Jochen Küster, Thomas Gschwind, Olaf Zimmermann IBM Research - Zurich

ACM/IEEE 12th International Conference on Model Driven Engineering Languages and Systems (MODELS), Denver, October 2009 © 2009 IBM Corporation

IBM Research

Outline Motivation and Case Study Overview Requirements for Development of Transformation Chains Techniques and Approach – Test Design Techniques – Test Architecture – Development Process

Results Conclusion & Outlook

2

© 2009 IBM Corporation

IBM Research

Motivation and Context Process Models are key artifact in modern SOAoriented software development

BPMN

Business-Driven Development comprises – modeling of processes at different levels of abstraction – in different modeling languages (BPMN,BPEL) – in a multi-user environment by different people Change Management Solution for Process Models – see “Language-Independent Change Management of Process Models” by C. Gerth et al – framework for change management of process models

3


IBM Research

Overview of Change Management Solution BPMN to Workflow Graph Transformation

PST Computation

Correspondence Computation/ Process Matching

Difference Computation

Dependency Computation

Filtering of Non-BPMN Differences

BPMN Workflow Graph 1

BPMN

Workflow Graph 2

Process Structure Tree 1


Difference Model Joint Process Structure Tree

Difference Model

BPMN - InsertFragment(V1,”Record Claim”, Difference Difference Decision, fA) Model Model With - InsertActivity(V1,”Retrieve add. Data”, With Dependencies Merge, Decision) Dependencies -…

Transformation for constructing Difference Model – obtained by instantiating the framework for change management

Transformation chain involving 6 individual transformations Each transformation has to be implemented in Java (tool integration issue)

How to ensure quality of the transformation chain? 4


IBM Research

Challenges in Transformation Chain Design and Implementation BPMN to Workflow Graph Transformation

PST Computation






BPMN

Workflow Graph 2



Joint Process Structure Tree

Difference Model

BPMN Difference Model With Dependencies

Difference Model With Dependencies

Dependencies between transformations – changes in correspondence computation affect Difference Model – changes in WFG and PST computation affect Difference Model – …

Dependencies between different input model pairs – functionality written for one input model pair may affect other input model pairs – infinitely many input model pairs make this difficult

Diversity of input models – transformation chain has to produce a Difference Model for all possible combinations

5


IBM Research

Outline Motivation and Case Study Overview Requirements for Development of Transformation Chains Techniques and Approach – Test Design Techniques – Architecture of Test Framework – Development Process


6


IBM Research

Requirements for Development Approach (1) How to ensure quality of a transformation chain? Development process must be iterative and incremental – development will take place in several iterations – transformation chain can be developed in teams – How to organize the development process for a transformation chain?

Testing of transformation chain is required – important for quality assurance – How to design test cases for a transformation chain?

A fully automated test environment is needed – manual testing on large number of input models is impossible in practice – How to establish such an automated test environment?

7


IBM Research

Requirements for Development Approach (2) How to ensure quality of a transformation chain? Specific transformation chain requirements – software engineer must be able to change/add functionality of a transformation without breaking transformation chain – fixing defects requires a mechanism to ensure that changes of a transformation are side-effect free – refactoring for ensuring code quality of the transformations is required, typical refactorings (e.g. [1]) need to be applied – How to fulfill these requirements in the development environment and in designing test cases?

[1] M. Fowler: Refactoring. Improving the Design of Existing Code. Addison Wesley,1999 8


IBM Research



9


IBM Research

Test Design Techniques for Model Transformation Chains Test design techniques for deriving test cases for a transformation chain – test cases for transformation chain and each individual transformation Three different test design techniques – Inspected reference outcomes – Invariant validation – Deviation test data

10


IBM Research

Test Design Techniques for Automated Testing (1) Inspected reference outcomes

textually specified result of a transformation

creation: manually inspect the outcome of the transformation once and save it BPMN Difference Model - InsertFragment(V1,”Record Claim”, Decision, fA) BPMN

Inspected Reference Outcomes

- InsertActivity(V1,”Retrieve add. Data”, Merge, Decision) -…

usage: automatic testing verifies outcome when applying a transformation

advantage: – very valuable during defect removal and refactoring – detection of unwanted side effects when changing transformations

drawback: only limited numbers of test cases possible (manual overhead)

11


IBM Research

Test Design Techniques for Automated Testing (2) Invariant validation:

invariants about the outcome of a transformation

creation: encode invariants in Java or OCL BPMN to Workflow Graph Transformation


PST Computation



Difference Model


Invariants: WFG and PST properties

Invariants:

• every node is reachable

•no cyclic dependencies

• StructuredNode has exactly one entry/exit edge

12

usage: check invariants automatically when applying transformation

advantage: large number of test cases possible, no manual overhead © 2009 IBM Corporation

IBM Research

Test Design Techniques for Automated Testing (3) Deviation test data

test data about created model structures

creation: determine test data to be computed, compute it automatically for each test case Difference Model - InsertFragment(V1,”Record Claim”, Decision, fA) - InsertActivity(V1,”Retrieve add. Data”, Merge, Decision)

#InsertFragment #InsertActivity #...

Repository for Deviation Test Data

-…

usage: in subsequent test runs, compare data automatically and report deviations

advantage: – valuable while refactoring and defect removal for detecting side effects – identification of input models where deviation occurs

13


IBM Research

Tests and Transformation Overview BPMN to Workflow Graph Transformation

PST Computation






BPMN

Workflow Graph 2



Inspected Reference Outcomes Invariant test

Deviation Test

14

Joint Process Structure Tree

Difference Model


BPMN Difference Model With Dependencies

Inspected Reference Outcomes Invariant test Of Models 1

Invariant test Of Models 2

Deviation Test 1

Invariant test Of Models 3

Deviation Test 2


IBM Research

Architecture of Test Framework Test case input models are created manually Analysis Component is added for evaluating test results – based on test design techniques presented – produces detailed test result report

Analysis Component Model Transformation Execution Component

Test Case Input Models

15

Model Transformation Chain

Models

Reference Outcome Analysis/ Invariant Validation/ Deviation Analysis/



Test Result Reports


IBM Research

Architecture of Test Framework

Manual Inspection Component simplifies creation of test case reference outcomes

Execution component reuses JUnit framework for test case execution

Test Case Execution Component (JUnit)

Manual Inspection Component

Analysis Component Model Transformation Execution Component

Test Case Input Models

16

Model Transformation Chain

Models

Reference Outcome Analysis/ Invariant Validation/ Deviation Analysis/



Test Result Reports


IBM Research

Iterative and Incremental Development Process Implement model transformation

Create first set of test cases

Run test cases

Defects? Yes No

Improve model transformation implementation by removing defects

Increase set of test cases

17

No focus on requirements analysis Needs the automated testing environment Inspired by agile development

Implement functionality

New test cases required? Yes No New functionality? Yes

No Refactoring?

Refactor Yes

No


IBM Research



18


IBM Research

Defects versus Test Case Input Models ~300 test case input model pairs with inspected reference outcomes fully automatic test environment based on the test framework architecture (completion in ~ 20 sec) refactoring needs automated testing 140

120

100

80 Test Case Input Models 60

Defects

40

20

0 12:00 10:53 11:23 12:46 16:23 10:17 10:22 11:24 12:19 10:25 11:59 13:47 16:44 15:49 11:10 12:00 17:00 15:09 13:33 11:45 14:35 13:15 13:58 16:29 13:36 13:53 15:19 15:23 10:52 10.10. 28.10. 28.10. 28.10. 28.10. 29.10. 29.10. 29.10. 29.10. 30.10. 30.10. 10.11. 11.11. 13.11. 25.11. 26.11. 26.11. 01.12. 02.12. 04.12. 04.12. 05.12. 05.12. 05.12. 23.12. 23.12. 23.12. 23.12. 08.01.

19

Date and Time


IBM Research

Lessons Learnt Incremental development for model transformation chains is required in practice Iterative improvement of the transformations is the reality Systematic quality improvement is not possible without automated testing Code quality is an important issue when implementing model transformations in Java One has to choose the encoding of the inspected reference outcomes carefully

20


IBM Research

Conclusion & Outlook Incremental and iterative development process for developing high-quality transformation chains Test design techniques and test framework for automated testing of transformation chains Further research challenges include: – Model coverage of test case input models – Automated generation of test case input models

21
