Incremental Development of Model Transformation Chains Using ...

3 downloads 1497 Views 102KB Size Report
Requirements for Development of Transformation Chains ... oriented software development ... Development process must be iterative and incremental.
IBM Research

Incremental Development of Model Transformation Chains Using Automated Testing

Jochen Küster, Thomas Gschwind, Olaf Zimmermann IBM Research - Zurich

ACM/IEEE 12th International Conference on Model Driven Engineering Languages and Systems (MODELS), Denver, October 2009 © 2009 IBM Corporation

IBM Research

Outline  Motivation and Case Study Overview  Requirements for Development of Transformation Chains  Techniques and Approach – Test Design Techniques – Test Architecture – Development Process

 Results  Conclusion & Outlook

2

© 2009 IBM Corporation

IBM Research

Motivation and Context  Process Models are key artifact in modern SOAoriented software development

BPMN

 Business-Driven Development comprises – modeling of processes at different levels of abstraction – in different modeling languages (BPMN,BPEL) – in a multi-user environment by different people  Change Management Solution for Process Models – see “Language-Independent Change Management of Process Models” by C. Gerth et al – framework for change management of process models

3

© 2009 IBM Corporation

IBM Research

Overview of Change Management Solution BPMN to Workflow Graph Transformation

PST Computation

Correspondence Computation/ Process Matching

Difference Computation

Dependency Computation

Filtering of Non-BPMN Differences

BPMN Workflow Graph 1

BPMN

Workflow Graph 2

Process Structure Tree 1

Process Structure Tree 2

Difference Model Joint Process Structure Tree

Difference Model

BPMN - InsertFragment(V1,”Record Claim”, Difference Difference Decision, fA) Model Model With - InsertActivity(V1,”Retrieve add. Data”, With Dependencies Merge, Decision) Dependencies -…

 Transformation for constructing Difference Model – obtained by instantiating the framework for change management

 Transformation chain involving 6 individual transformations  Each transformation has to be implemented in Java (tool integration issue)

How to ensure quality of the transformation chain? 4

© 2009 IBM Corporation

IBM Research

Challenges in Transformation Chain Design and Implementation BPMN to Workflow Graph Transformation

PST Computation

Correspondence Computation/ Process Matching

Difference Computation

Dependency Computation

Filtering of Non-BPMN Differences

BPMN Workflow Graph 1

BPMN

Workflow Graph 2

Process Structure Tree 1

Process Structure Tree 2

Joint Process Structure Tree

Difference Model

BPMN Difference Model With Dependencies

Difference Model With Dependencies

 Dependencies between transformations – changes in correspondence computation affect Difference Model – changes in WFG and PST computation affect Difference Model – …

 Dependencies between different input model pairs – functionality written for one input model pair may affect other input model pairs – infinitely many input model pairs make this difficult

 Diversity of input models – transformation chain has to produce a Difference Model for all possible combinations

5

© 2009 IBM Corporation

IBM Research

Outline  Motivation and Case Study Overview  Requirements for Development of Transformation Chains  Techniques and Approach – Test Design Techniques – Architecture of Test Framework – Development Process

 Results  Conclusion & Outlook

6

© 2009 IBM Corporation

IBM Research

Requirements for Development Approach (1) How to ensure quality of a transformation chain?  Development process must be iterative and incremental – development will take place in several iterations – transformation chain can be developed in teams – How to organize the development process for a transformation chain?

 Testing of transformation chain is required – important for quality assurance – How to design test cases for a transformation chain?

 A fully automated test environment is needed – manual testing on large number of input models is impossible in practice – How to establish such an automated test environment?

7

© 2009 IBM Corporation

IBM Research

Requirements for Development Approach (2) How to ensure quality of a transformation chain?  Specific transformation chain requirements – software engineer must be able to change/add functionality of a transformation without breaking transformation chain – fixing defects requires a mechanism to ensure that changes of a transformation are side-effect free – refactoring for ensuring code quality of the transformations is required, typical refactorings (e.g. [1]) need to be applied – How to fulfill these requirements in the development environment and in designing test cases?

[1] M. Fowler: Refactoring. Improving the Design of Existing Code. Addison Wesley,1999 8

© 2009 IBM Corporation

IBM Research

Outline  Motivation and Case Study Overview  Requirements for Development of Transformation Chains  Techniques and Approach – Test Design Techniques – Architecture of Test Framework – Development Process

 Results  Conclusion & Outlook

9

© 2009 IBM Corporation

IBM Research

Test Design Techniques for Model Transformation Chains  Test design techniques for deriving test cases for a transformation chain – test cases for transformation chain and each individual transformation  Three different test design techniques – Inspected reference outcomes – Invariant validation – Deviation test data

10

© 2009 IBM Corporation

IBM Research

Test Design Techniques for Automated Testing (1) Inspected reference outcomes 

textually specified result of a transformation



creation: manually inspect the outcome of the transformation once and save it BPMN Difference Model - InsertFragment(V1,”Record Claim”, Decision, fA) BPMN

Inspected Reference Outcomes

- InsertActivity(V1,”Retrieve add. Data”, Merge, Decision) -…



usage: automatic testing verifies outcome when applying a transformation



advantage: – very valuable during defect removal and refactoring – detection of unwanted side effects when changing transformations



drawback: only limited numbers of test cases possible (manual overhead)

11

© 2009 IBM Corporation

IBM Research

Test Design Techniques for Automated Testing (2) Invariant validation: 

invariants about the outcome of a transformation



creation: encode invariants in Java or OCL BPMN to Workflow Graph Transformation

Dependency Computation

PST Computation

BPMN Workflow Graph 1

Process Structure Tree 1

Difference Model

Difference Model With Dependencies

Invariants: WFG and PST properties

Invariants:

• every node is reachable

•no cyclic dependencies

• StructuredNode has exactly one entry/exit edge

12



usage: check invariants automatically when applying transformation



advantage: large number of test cases possible, no manual overhead © 2009 IBM Corporation

IBM Research

Test Design Techniques for Automated Testing (3) Deviation test data 

test data about created model structures



creation: determine test data to be computed, compute it automatically for each test case Difference Model - InsertFragment(V1,”Record Claim”, Decision, fA) - InsertActivity(V1,”Retrieve add. Data”, Merge, Decision)

#InsertFragment #InsertActivity #...

Repository for Deviation Test Data

-…



usage: in subsequent test runs, compare data automatically and report deviations



advantage: – valuable while refactoring and defect removal for detecting side effects – identification of input models where deviation occurs

13

© 2009 IBM Corporation

IBM Research

Tests and Transformation Overview BPMN to Workflow Graph Transformation

PST Computation

Correspondence Computation/ Process Matching

Difference Computation

Dependency Computation

Filtering of Non-BPMN Differences

BPMN Workflow Graph 1

BPMN

Workflow Graph 2

Process Structure Tree 1

Process Structure Tree 2

Inspected Reference Outcomes Invariant test

Deviation Test

14

Joint Process Structure Tree

Difference Model

Difference Model With Dependencies

BPMN Difference Model With Dependencies

Inspected Reference Outcomes Invariant test Of Models 1

Invariant test Of Models 2

Deviation Test 1

Invariant test Of Models 3

Deviation Test 2

© 2009 IBM Corporation

IBM Research

Architecture of Test Framework  Test case input models are created manually  Analysis Component is added for evaluating test results – based on test design techniques presented – produces detailed test result report

Analysis Component Model Transformation Execution Component

Test Case Input Models

15

Model Transformation Chain

Models

Reference Outcome Analysis/ Invariant Validation/ Deviation Analysis/

Inspected Reference Outcomes

Repository for Deviation Test Data

Test Result Reports

© 2009 IBM Corporation

IBM Research

Architecture of Test Framework 

Manual Inspection Component simplifies creation of test case reference outcomes



Execution component reuses JUnit framework for test case execution

Test Case Execution Component (JUnit)

Manual Inspection Component

Analysis Component Model Transformation Execution Component

Test Case Input Models

16

Model Transformation Chain

Models

Reference Outcome Analysis/ Invariant Validation/ Deviation Analysis/

Inspected Reference Outcomes

Repository for Deviation Test Data

Test Result Reports

© 2009 IBM Corporation

IBM Research

Iterative and Incremental Development Process Implement model transformation

Create first set of test cases

Run test cases

Defects? Yes No

Improve model transformation implementation by removing defects

Increase set of test cases

  

17

No focus on requirements analysis Needs the automated testing environment Inspired by agile development

Implement functionality

New test cases required? Yes No New functionality? Yes

No Refactoring?

Refactor Yes

No

© 2009 IBM Corporation

IBM Research

Outline  Motivation and Case Study Overview  Requirements for Development of Transformation Chains  Techniques and Approach – Test Design Techniques – Architecture of Test Framework – Development Process

 Results  Conclusion & Outlook

18

© 2009 IBM Corporation

IBM Research

Defects versus Test Case Input Models  ~300 test case input model pairs with inspected reference outcomes  fully automatic test environment based on the test framework architecture (completion in ~ 20 sec)  refactoring needs automated testing 140

120

100

80 Test Case Input Models 60

Defects

40

20

0 12:00 10:53 11:23 12:46 16:23 10:17 10:22 11:24 12:19 10:25 11:59 13:47 16:44 15:49 11:10 12:00 17:00 15:09 13:33 11:45 14:35 13:15 13:58 16:29 13:36 13:53 15:19 15:23 10:52 10.10. 28.10. 28.10. 28.10. 28.10. 29.10. 29.10. 29.10. 29.10. 30.10. 30.10. 10.11. 11.11. 13.11. 25.11. 26.11. 26.11. 01.12. 02.12. 04.12. 04.12. 05.12. 05.12. 05.12. 23.12. 23.12. 23.12. 23.12. 08.01.

19

Date and Time

© 2009 IBM Corporation

IBM Research

Lessons Learnt  Incremental development for model transformation chains is required in practice  Iterative improvement of the transformations is the reality  Systematic quality improvement is not possible without automated testing  Code quality is an important issue when implementing model transformations in Java  One has to choose the encoding of the inspected reference outcomes carefully

20

© 2009 IBM Corporation

IBM Research

Conclusion & Outlook  Incremental and iterative development process for developing high-quality transformation chains  Test design techniques and test framework for automated testing of transformation chains  Further research challenges include: – Model coverage of test case input models – Automated generation of test case input models

21

© 2009 IBM Corporation