Migrating a Large Modeling Environment from XML/UML to Xtext/GMF Practitioner Report Moritz Eysholdt
Johannes Rupprecht
itemis AG
[email protected]
VSA Group
[email protected]
Abstract
1.
If you use UML to drive code generation, what do you do when a UML model grows so large that the tool needs 60 minutes to open it? What if it turns out that UML doesn’t provide the expressiveness you need? What if your UML Editor barely provides feedback about the validity of your models and the turnaround time from modeling to code-generation and compilation is somewhere between setting up a new can of coffee and reading a day’s amount of slashdot.org news? Time to look for a different modeling solution... This paper reports how VSA (Verrechnungsstelle der Süddeutschen Apotheken GmbH, one of the most well-known billing centers for pharmacies in Germany) migrated their in-house modeling environment and their existing models from UML to an Eclipse Modeling based solution: Models of all five pre-existing modeling languages are now persisted textually and can be edited using Xtext. Models of two chosen modeling languages can additionally be edited graphically using GMF-based editors which store their models textually using Xtext. The modeling environment provides tight integration with the JDT for referencing and navigating to Java elements. Custom validation rules provide the earliest feedback possible for the modeler. An Eclipse Builder triggers incremental code generation, which keeps the turnaround time to a minimum. This paper describes the old modeling environment, addresses the challenges of the migration and outlines the new modeling environment. Furthermore, it documents how to integrate GMF with Xtext. The project has been realized in cooperation between VSA and itemis AG. VSA had the knowledge about the existing modeling environment, itemis AG, a consulting company with several fulltime Eclipse Modeling Committers, brought in their knowledge about EMF, Xtext, GMF, etc.
The VSA, a company of 1.100 employes, based in Germany, which has specialized in billing and IT in the health care business, uses model driven software development for in-house software development. The tool chain which has been established during the last eight years included modeling in UML, exporting the modes to XML and generating Java code from the XML. However, the models grew so large that the UML tool needed up to an hour to load the largest module. It became obvious that due to misuse of UML, modeling in UML was sometimes more complex than in XML. This paper reports about the project in which new modeling tools have been created to replace the existing ones. Furthermore, existing models were migrated to the new infrastructure. In cooperation with the company itemis, the existing UML- and XML-based languages have completely been replaced with textual languages (external DSLs, implemented with Xtext). For selected languages, graphical editors (implemented with the Graphical Modeling Framework, GMF) have been developed. The graphical and textual editors operate on the same files. Therefore, the user can freely choose his/her preferred representation and there is neither round-trip engineering nor other synchronization needed to keep textual and graphical representation synchronized. Furthermore, mechanisms for a fully-automated migration of the existing models and diagrams have been developed.
2.
Introduction
About VSA Group
The VSA (Verrechnungsstelle Süddeutscher Apotheken; The name translates to Clearing House of the South German Pharmacies) was founded in 1983 and is owned by 7000 German pharmacists. The main business units are: • VSA: Clearing for German Pharmacies • azh: Clearing and IT for German health care professionals
Keywords Modeling, Migration, MDSD, DSL, Xtext, Eclipse, EMF, GMF
• awinta: Pharmacy administration systems
150 of the employees are tasked with software development, which is distributed over several projects and several locations. The developed systems include internal systems (e.g. the two clearing systems), online systems for the clearing customers and pharmacy administration systems. Development is mostly done in Java, but also C#, C++, C and Objective-C are used. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. SPLASH 2010 Oct 2010, Reno, NV, USA c 2010 ACM and itemis AG. . . $10.00 Copyright
3.
About itemis AG
Four companies applied for implementing this project, three of them proposed a solution based on Xtext and GMF and itemis was chosen due to their competence with Xtext.
Employing 140 people, itemis has its headquarter in Lünen (close to Dortmund), Germany and branches in Canada, Switzerland and France. As a Strategic Member of the Eclipse Foundation itemis employs several Eclipse committers of which four are working full time on the Xtext project. This put itemis in the unique position to extend Xtext with features (such as integration with the Java language and support for incremental building) that were needed for this project. Model Driven Software Development and Domain-Specific Languages are the main focus of itemis. This knowledge is mainly applied in the fields of enterprise applications, embedded systems and mobile devices (e.g. based on the iPhone and Android platform).
4.
• Assistance with entering arithmetic/logical expressions, e.g. the
conditions used for the find expressions. Modeling in UML led to additional and similar problems: • Opening the largest UML module with MagicDraw took up to
one hour. • Opening the largest UML diagram took up to ten minutes. • Turnaround times were too slow. The process was as follows:
Change an UML diagram, export XML files, generate Java files, compile them via ANT. A developer had to go through this process each time he/she wanted to ensure his/her model is correct. • Modeling was not intuitive since UML was misused to express
The Modeling Environment
Around 2000 to 2002, the VSA re-implemented their existing systems in Java. This created the need for supporting frameworks and led to the development of custom frameworks for object relational mapping and user interface workflows, since back in these days no viable other frameworks were available. The frameworks grew as well as the projects using them. Multiple database implementations and different UI technologies had to be supported. XML files were introduced from which Java code was generated. This allowed for some degree of platform/technology independence and it allowed to work at a higher level of abstraction. Later, UML was introduced to add another layer of abstraction on top of the existing XML models. The tool of choice was MagicDraw (by No Magic) and custom plug-ins were developed to allow the XML export. The name given to this modeling environment is marvin. The following concepts could be described in XML/UML:
things UML is not suited for. • Partly, the UML model was more complex than the correspond-
ing XML file. • Modeling was occasionally cumbersome, because there was
no assistance in editing the models. For example, when other model elements were referenced by their name or when entering arithmetic/logical expressions in a custom, ANTLR-based language. • Workflow diagrams became too big and thereby hard to handle.
The largest workflow had more than 700 nodes. • Not all functionality of the XML files could be expressed via
UML. • Roundtrip-engineering between XML and UML was not possi-
ble. I.e. there was no way to re-create an UML diagram from XML files.
• Persistent Objects (entities), including their columns/attributes,
inheritance, master-detail-relationships and find-methods.
• No custom validation was available in the UML tool.
• Persistent Views, including query statements and find methods.
• No content assist was available in the UML tool.
• Workflows, consisting of activities and steps to describe the
• Branching was not possible, therefore modeling was restricted
paths through user interface masks.
to one branch.
• Masks, to describe forms in Java/Swing and HTML • ObjectModel, to describe objects for databinding in forms
The following projects are examples for what has been developed with this tool chain: Project jump is about administration of pharmacies and project nProd is about clearing for pharmacies.
Effort (Man Years) Lines of code Classes Persistent Entities Persistent Views Workflows Masks UML diagrams
5.
jump 200 1.400.000 20.500 916 369 278 1.150 2000
nProd ? 1.000.000 13.300 560 96 124 612 1000
Problems with the Old Modeling Environment
Modeling in XML was inefficient due to the verbose syntax itself and the lack of good tool support. Tools which could check tags/attributes based on the XML Schema were not sufficient, since the more relevant aspects of assistance are: • Instant feedback of custom validation rules. • Instant feedback and content assist for references to other model
elements and files.
• There was no navigation between code and model.
It is out of question that not all of the points listed above can or should be blamed on the UML tool. However, re-factoring the existing models to make optimal use of the UML tooling would have required the same or more effort than the project this paper reports about.
6.
Objectives for the New Modeling Environment
The objectives for the new modeling environment can be summarized as follows: • Textual languages for all existing XML-languages based on
Xtext. • Graphical modeling for selected aspects of selected languages
based on GMF (no UML!). Furthermore, there should be a mechanism to partition workflow diagrams. • Integration of the legacy code generator as an Eclipse Builder • Tooling for a 100% automatic migration of existing projects,
including layout information from MagicDraw. Furthermore, the ability to migration one project at a time is needed. • Support for mixing migrated and not-yet-migrated projects,
since projects can depend on each other. It is not possible to migrate all projects at once since they are involved in different release cycles.
7.
The Value of Eclipse/XMF/Xtext/GMF
To keep the efforts for this project in an affordable scale, it was imperative to build on existing frameworks as much as possible. Furthermore, since the modeling tools are used in Java projects, it is valuable for them to be integrated with a Java development environment. This is available with Eclipse: Eclipse’s JDT for Java development, Eclipse’s extensibility via plug-ins, Eclipse’s rich ecosystem of modeling tools and frameworks. For modeling, the Eclipse Modeling Framework (EMF) provides support for XML/XML Schema and is the foundation for integrating textual languages based on Xtext with graphical languages based on GMF. Furthermore, all these frameworks are available as open source.
8.
been implemented using Xtext’s declarative API for EValidators. Validation is run as the user types, i.e. the user gets instant feedback about potential errors/warnings. This works well for the moment, but in case longer-running validation rules will be introduced, they should only be run on save or when explicitly triggered. • Open Model Elements dialog: This dialog resembles JDT’s
Open Type dialog. By entering a name or a search pattern the user can quickly identify all matched model elements no matter where they are stored in his/her workspace. Furthermore, navigation to these elements is provided. • Incremental build: Xtext integrates with Eclipse’s Builder in-
frastructure: Each time a file is modified on the hard disk, the Builder is triggered to update Xtext’s index with the modified model elements and run validation. Furthermore, the VSA’s existing code generator has been integrated. The user experience is as follows: If he/she modifies and saves a file, the generated Java code is updated in a matter of seconds. Code generation runs only if validation passes without errors.
The Project
To realize this project, two developers from itemis and four developers from VSA have been working in cooperation. Two of the VSA’s developers were working part time. Working in cooperation ensured training on the new technologies for the VSA developers. The project’s extent was defined in a specification document. Development was distributed over four different locations. A shared infrastructure consisting of a subversion repository and a Trac bugtracker (which includes a wiki) supported the teamwork. Weekly telephone conferences were held to check the status and discuss problems.
9.
• Validation: In this project many custom validation rules have
9.1
Domain.xtext and View.xtext
The New Modeling Environment
From the five XML/UML based languages, five textual languages with equivalent semantics have been created. For selected aspects, graphical editors have been created using GMF. Since tooling for all textual languages has been created using Xtext, the editors provide a similar user experience as Eclipse’s JDT. The following features are provided. For the very most cases, Xtext’s default behavior was just fine. For all other cases, it is mentioned what has been customized: • Syntax highlighting • Outline View: Structure, labels and icons have been cus-
tomized. • Scoping has been customized. In Xtext, a scope is the list of
visible names for one cross reference. Linking and content assist delegate to scoping. • Content assist, which provides suggestions for keywords and all
model elements and Java elements that can be referenced at the current cursor’s position in the document.
Figure 1. Textual and graphical editors of the persist-languages
• Validation of cross references: If a model element or Java ele-
ment identified by a name can not be resolved, an error marker is created. • Navigation: If the user holds CTRL and clicks on a cross ref-
erence or presses F3, the editor will navigate to the referenced element. If this element is in a different file or is a Java element, a new editor will be opened. • Find references: When selecting a model element, the Find
References View provides a list of all model elements that hold references to the selected element. This feature is especially valuable to gather knowledge about existing code before refactoring it. • Formatting: Formatting automatically optimizes indentation,
line-breaks and white-spaces. Custom formatting rules have been implemented for a well-readable appearance of the source code.
The language for persistent objects (called domain-language from now on) and the language for persistent views build the first group of languages. Both extend a common language named PersistCommon which provides constructs that are shared by both languages. Example: Syntax of an expression language for find-methods. 9.2
The Graphical Domain Editor
Furthermore, a GMF-based editor for the domain language has been created. Its visual appearance resembles a UML class diagram. Its behavior, however, differs from typical GMF-based editors. Usually one diagram editor operates on one file representing the semantic model. In this case, the diagram is intended to display nodes for all semantic model elements (the ones for which nodes are defined). Deleting a node from the diagram means to delete the element from the semantic model as well. However, the diagram editor developed in this project operates on an arbitrary amount of
semantic models. This became necessary since one domain (think of one entity) is persisted in one file and is expected to be one node in the diagram. Therefore, when a new diagram file is created there is initially no semantic model and the diagram is empty. The user may then add nodes to the diagram by drag’n’dropping domainfiles from Eclipse’s Package Explorer onto the diagram. If nodes are related, edges are automatically added. Furthermore, an action in the context menu can be used to add all related elements of one node. 9.3
Workflow.xtext, ObjectModel.xtext and Mask.xtext
UNIX file system, allow to have a representation of a semantic element at a position in a diagram that does not correspond with the actual position in the semantic model.
10.
From XML to a Textual Language
When an XML Schema and example XML files are available, as in this project, this can be used as a starting point to develop a textual language. The basic idea is to look at an XML file and think of ways how to make the contents more readable. To illustrate this idea, the reader may think of the following steps: 1. Replace all “....” with sometag {...}. If the object has a name by which it can identified, the nameattribute may be removed and the name inserted behind the tag’s name: “sometag elementname {...}”. 2. Remove all remaining “” and “/>”. 3. Remove directives such as “
