Highway: a Domain Specific Language for Enterprise Application Integration Vitomir Kovanovic
Dragan Djuric
School of Interactive Arts and Technology Simon Fraser University
Faculty of Organizational Sciences University of Belgrade
[email protected]
[email protected]
ABSTRACT Highway is a domain-specific language for implementing enterprise application integration solutions in a technology independent and functional manner. As an internal DSL developed on top of Clojure programming language, Highway uses functional programming techniques in order to simplify enterprise application integration development. In this paper we focus on abstractions and language constructs that define Highway’s approach to integration. We also cover implementation of enterprise integration patterns using Highway since they represent various common situations in enterprise application integration development.
General Terms Languages
Keywords Highway, Enterprise Application Integration, Domain Specific Language
1.
INTRODUCTION
One of the central activities in enterprise software development is the continuous process of integration of various technologically heterogeneous parts into an evolving system. Enterprise application integration (EAI) projects have a very high failure rate due not only to technological complexity, but because the systems being integrated are critically tied to the business process itself. The solutions to typical problems in EAI have been continuously rediscovered and improved, and the most typical solutions are encoded as EAI patterns - descriptions of the tried and tested general solutions to typical recurring problems. However, the implementation of these patterns (solutions) is not an easy task itself and introduces its own share of complexity. General-purpose programming languages do not provide internal support for enterprise application integra-
tion. Of course, it is possible to extend the grammar of a specific language to include such support. However, extending grammars of widely used languages may not be easy and the relevant community may not adopt it easily. An alternative is to develop and use special-purpose libraries, but it always adds complexity and keeps the impedance mismatch. Yet another alternative is to use meta-programming [9], a craft and a process of using tools and languages for creating, modifying, adapting, adjusting, and otherwise transforming other programs. Some programming languages support meta-programming on mainstream platforms in such a way that developers can extend the language/platform with the features they need and implement first-class support for enterprise application integration. The result is Highway: an internal domain-specific language for implementing enterprise application integration solutions in a technology independent and functional manner. As an internal DSL developed on top of Clojure programming language Highway uses functional programming techniques in order to simplify enterprise application integration development.
2. THE FOUNDATIONS 2.1 Main Challenges in EAI Even more than typical software development projects, EAI projects have high failure rate - according to the EAI Consortium[2], around 70% of EAI projects have failed to deliver the desired goals, mainly due to the following challenges:
• Enterprise integration requires changes in the company management. It is not only about communication between software systems but also between different departments and business units. • EAI systems are critical for business because of their mediating role in the business process management. Failures in software integration are very expensive and represent a big threat to the stability of the business process. • For the most part, the developers of integration solutions do not have control over existing software systems. Usually they are large legacy systems that were incorporated into the business process over the years, using various strategies that are often incompatible.
basic evaluation of quality and usability of developed DSL. • Support for transformation rules Since integration usually involves communicating with different software systems and defining transformation of messages between them it is crucial to support creation of transformation rules. DSL should support expressing various different message transformations so that the process of developing integration solution becomes more easier.
Figure 1: Content based router pattern
• Language should support functional programming techniques and message transformations in a functional way
Figure 2: Message filter pattern
Enterprise integration is usually implemented in a series of message transformations from one participating application service to another. For example, message coming from some JMS queue can be converted to the e-mail message sent to an address that is being monitored by some internal application that knows how to interpret those messages.
• There are various communication technologies varying from binary socket communication, Remote Procedure Calls(RPC), CORBA, Web services, XML, JSON and others. The integration techniques are still in the development and integrating applications must be able to communicate using all of this technologies.
2.2
• Transparent and unified access to the underlying communication technologies
Enterprise Integration Patterns
Integration logic should be the same weather in involves JMS, XML, RPC or any other technology. Communication technologies are always changing and they should have minimal impact on the stability of integration software system. This would allow easier evolution of information system since individual parts of the system can be changed without significantly affecting integration system.
Enterprise integration patterns (EIP) represent best practices and techniques for enterprise application integration that software engineers have been discovering and using for a long time. EI patterns are described in depth in [6], particularly message-based integration patterns that are the main focus of this paper. Software pattern itself, is a structured way of documenting software engineering techniques, popularized by [5].
• Support for Enterprise Integration Patterns
One of the most popular integration patterns, Content Based Router shown on Figure 1, is presented here as an illustration. It is used when a system needs to dispatch the message to different destinations based on the message content. The solution is to use a message consumer that encapsulate the business logic for dispatching and then sends the message to the appropriate destination depending of the content. Another typical pattern is Message Filter pattern shown on Figure 2, which is used in situations when some of the messages should be filtered and not sent to the destination. There are many different enterprise integration patterns covering various situations such as message filtering, dispatching, re-sequencing, splitting and aggregating, delaying, load balancing, handling failed deliveries and many more.
3.
THE REQUIREMENTS OF EAI DSL
In order to properly define DSL for application integration we must define what are the basic requirements for one such language. • Provide abstractions and concepts specific to the enterprise application integration in order to simplify development of integration solutions Language should define most important concepts so that engineers can develop integration solutions more easily. This is the basic requirement that allows most
Language should allow simple and concise expression of most patterns since they are so common and popular in practice. This criteria is also very important as a way to evaluate expressive power of DSL since integration patterns cover various real problems and most of application integration solutions are just different compositions of those patterns.
4.
HIGHWAY: THE MAIN CONCEPTS
Highway is a domain specific language for enterprise application integration that helps in an easier development of integration software. Because EAI using messaging is easily expressed as a functional transformation of messages between participating applications and their services, Highway has five language constructs to express these transformations:
• Message is the most basic concept in Highway. Everything is based around message consumption, production and transformation. Each message is composed of two parts; It’s body which can be of any type, depending on the purpose of message and map of headers where each header entry is name-value pair where name is simple java.lang.String and value that can be of any type.
["new widget-order" {"amount" 1 "price" 100 "user_id" 1001}]
(route (from "jms:new_orders") (get-header "type" (equals "widget")) (to "jms:widget_orders"))
Figure 3: Simple Highway message Figure 5: Message filter pattern (route (from "jms:new_orders") (router (get-header "type" (equals "widget")) "jms:widget_inventory" (get-header "type" (equals "gadget")) "jms:gadget_inventory")) Figure 4: Content based router pattern • Endpoint represents a participating application and its services involved in the process integration. In Highway, they are defined as strings in the form of protocol://destination name. Such definition of Endpoints is useful because it hides all communication details from the developers, who do not need to write low level communication code. It is very similar to the concept of Wrapper described in [4] but also more abstract since it is on the language runtime to provide support for various communication technologies. • Route is any transformation of messages between two or more Endpoints. As message transformation is at the heart of message-based EAI, Route is the most important concept in Highway and most of the DSL is focused around easier route creation. Typical route will read from one endpoint, transform the data and then based on some business logic transfer that data to the some other endpoint. The notion of route as a basic element of EAI is very similar to the concept of Flow described in [4]. • Predicate is the function that receives message as an argument and returns boolean value. It is very simple, yet very powerful construct used in every functional programming language as an input to the generic functions such as filter function. • Processor is the function that receives message as an input and also returns a message as an output. It is the most general processing component that can be used to implement various different message transformation such as Message Enrichment, Filtering, Normalizing and other transformations described in [6].
Highway is implemented as an internal DSL on top of Clojure programming language. The decision to use Clojure is justified with its meta-programming capabilities and also its minimalist syntax. Highway built-in DSL constructs are based in Apache Camel integration engine which is also used as an underlying integration technology. Additionally, GuaranaDSL’s Wrappers and Processes are very similar to the Highway’s Endpoints and Routes[4]. Figure 4 shows typical example of content based router pattern using Highway DSL. Every route is constructed using route macro which accepts route definition elements as its arguments. In this example router function that is used
RouteBuilder builder = new RouteBuilder() { public void configure() { from("jms:new_orders").choice() .when(header("type").isEqualTo("widget")) .to("jms:widget_inventory") .when(header("type").isEqualTo("gadget")) .to("jms:gadget_inventory"); } }; Figure 6: Apache Camel implementation of content based router pattern
to construct conditional routing needed for message routing. Message is being transferred from JMS queue new_orders to the queues gadget_inventory and widget_inventory based on the content of message header type. This is very simple example but still it is very common in the real software systems. Message Filter pattern (figure 5), is another example of Highway for popular EI pattern. JMS queue new_orders receives both Widget and Gadget orders and this route filters only widget orders and transfers them to the JMS queue widget_orders using the Highway’s built-in filter function.
5. EVALUATION 5.1 Highway vs. Camel In order to evaluate Highway we will compare its main features with Apache Camel since it is one of the most popular EAI technologies and also very mature software product. Apache Camel uses fluent interface[3] API design style and allows creation of EAI systems using Java language. Highway is on the other hand built on top of Clojure and hence supports Clojure’s advanced features such as macros and usage of high order functions. Since Highway internally uses Camel they are mostly of the same capabilities but Highway code is much more shorter and concise due to the functional programming style. This is the biggest overall difference between Camel and Highway. Java as a platform for implementing EAI solutions has severe limitations due to its lack of support for functional programming techniques. In figures 4 and 5 we have shown Highway’s implementation of two very popular patterns. In figures 6 and 7 are shown corresponding implementations using Camel’s fluent Java interface. As we can see Camel’s implementation is much more verbose and do not correspond to the usual way of writing Java programs. Also purpose of various methods is not clear just by looking at their names and documentation. They need to be looked in the context of DSL expression and not traditional Java API browsing style.
RouteBuilder builder = new RouteBuilder() { public void configure() { from("jms:new_orders") .filter(header("type").isEqualTo("widget")) .to("jms:widget_orders"); } };
• Support for Enterprise Integration Patterns Highway support most of the integration patterns explained in the [6]. Many of the patterns such as “Message Channel”, “Message” and “Message Endpoint” are integrated in the language design and therefore there is no special effort required to implement those patterns.
6. Figure 7: Apache Camel implementation of message filter pattern If we compare Highway and Camel according to the requirements mentioned in Section 3 we can see that they both meet all of the requirements except that Camel solution do not allow usage of functional programming techniques for EAI development. Also Highway’s code is a little bit shorter due to the flexible nature of Lisp inspired syntax.
5.2
Does it Meet the Requirements?
In Section 3 we have mentioned requirements that good DSL language for EAI should satisfy. Here we take a closer look how Highway stands against those requirements. • Provide abstractions and concepts specific to the enterprise application integration As we mentioned in Section 4, Highway provides very various concepts specific to the domain of EAI. These concepts define most important elements that compose any enterprise integration system. Some of these concepts are found in other integration solutions such as GuaranaDSL’s Processes, Wrappers and Tasks[4] and some, such as Predicates, are common in the functional programming world. • Support for transformation rules Highway main goal is to support creation of transformation rules. Programs in Highway are mostly consisting of definitions of Routes and each route represents one transformation rule that connects two or more endpoints in a business process. Route abstraction is very similar to the GuaranaDSL’s IntegrationLink[4] abstraction and RouteDefinition concept in Apache Camel[8, 7]. • Support for functional programming Since Highway is internal DSL on top of Clojure, it has all functional capabilities of its host language that enable usage of functional principles in the process of designing integration solutions. This highly affects overall usability of the language since it is possible to create highly abstract transformations as higher order functions that can be easily specialized to meet different specific details. • Transparent and unified access to the underlying communication technologies Highway uses the same syntax as Apache Camel for defining access to the communication technology. It is based on the mature URI[1] standard and it allows unambiguous definition of different resources in the enterprise system.
CONCLUSIONS
DSL proposal presented in this paper contributes to improving development of integration solutions by providing basic concepts and building blocks for application integration development. Biggest change in comparison to the existing EAI tools and languages is that Highway enables functional programming in the context of EAI development. This is very important since functional programming techniques provide abstractions that are very useful in the context of EAI and can help reducing time and costs of integration solution development. Also Highway allows simple and consistent way for accessing various communication technologies. As enterprise system evolves over time, technologies used to communicate between software systems are also changing and we need a systematic approach to embrace those changes. Highway enables expressing most of the enterprise integration patterns that are common in practice. They represent proven designs and also common vocabulary among application integration practitioners and by providing support for integration patterns Highway contributes to minimizing software engineering effort to produce integration solutions and also standardizes integration development process.
7.
REFERENCES
[1] T. Berners-Lee, R. Fielding, and L. Masinter. Uniform resource identifiers (uri): generic syntax, 1998. [2] EAI Consortium. Avoiding eai disasters, 2004. Retrieved March 15, 2010, from Information Management www.information-management.com/news/8086-1.html. [3] M. Fowler and R. Parsons. Domain-specific languages. Addison-Wesley Professional, 2010. [4] R.Z. Frantz, A.M.R. Quintero, and R. Corchuelo. A domain-specific language to design enterprise application integration solutions. International Journal of Cooperative Information Systems (IJCIS), 20(2):143–176, 2011. [5] E. Gamma, R. Helm, R. Johnson, and J. Vlissides. Design patterns: elements of reusable object-oriented software, volume 206. Addison-wesley Reading, MA, 1995. [6] G. Hohpe and B. Woolf. Enterprise integration patterns: Designing, building, and deploying messaging solutions. Addison-Wesley Professional, 2004. [7] C. Ibsen and J. Anstey. Camel in Action. Manning Publications Co., 2010. [8] P. Kolb. Realization of eai patterns with apache camel. Student Research Project, Universit¨ at Stuttgart, Germany, 2008. [9] T. Sheard. Accomplishments and research challenges in meta programming. Lecture Notes in Computer Science, 2196:2–44, 2001.
