Coping with the Complexity of SOA Systems with Message Forensics Hideyuki Fukuhara and Tetsu Saburi Masayuki Hisada Takafumi Hayashi, Atushi Kara, Toshiaki Miyazaki, and Jiro Iwase NetOne Systems Inc. Dept. of Research and Development, School of Computer Sci. and Eng. Tokyo, Japan NST, Inc. The University of Aizu Email: {h–fukuhara,t–saburi}@netone.co.jp Aizu-Wakamatsu, Japan Aizu-Wakamatsu, Japan Email:
[email protected] Email: {takafumi,kara}@u-aizu.ac.jp, {miayazki,iwase}@u-aizu.ac.jp
Abstract—This paper introduces an approach to construct SOA (Service Oriented Architecture) systems using the socalled asynchronous messaging network. An asynchronous messaging network (or simply messaging network) refers to an overlay network (over LAN, VPN, Internet, etc.) that allows exchanging well-formatted asynchronous messages (typically in XML) between the service providers and consumers in the system. The proposed approach aims at reducing the operation and maintenance cost of the system by using a messaging network enhanced with the capability to store, inspect and analyze selected portions of the exchanged messages under the strict control of security and privacy. Complexity makes any information system vulnerable to design flaws, operation error, and security problems. The proposed approach facilitates analyzing these problems associated with complex SOA systems through the message-store analysis. We consider that the application of computer forensics to the message store in SOA helps the system administrator to identify and fix various problems. The requirements for the messaging network for SOA systems are also presented. Keywords-SOA, messaging network, forensics, message mediation
I. I NTRODUCTION Service Oriented Architecture (SOA) and related concepts including Enterprise Architecture (EA), Enterprise Service Bus (ESB) and Web Services have become to form a common framework for the development of modern information systems. In an SOA system, the functions of a system are decomposed into loosely-coupled reusable services distributed over public or private networks. SOA requires each service to have a well-defined set of interfaces. Gartner, an IT company and one of the earliest advocates of the SOA approach, suggested that SOA would face two key challenges related to the legacy-system integration [1]–[4]. First, the legacy applications and proprietary Web Services must adapt to the SOA environment by providing location-transparent and (more or less) protocol-independent services. Second, the traditional request-and-reply scheme in the server-client paradigm must integrate with the new Event Driven Architecture (EDA) which is based on the asynchronous message delivery.
SOA facilitates an enterprise to manage ever-changing business requirements in the corporate IT environment. Here an enterprise means a collection of human and informationprocessing activities within an organization or across multiple organizations. In an enterprise, information is exchanged and business resources are shared for the purpose of some common goals. An enterprise often runs multiple independent applications that are built by different vendors using different technologies. SOA is a basis for integrating these multiple applications. In particular, asynchronous messaging plays a central role in separating the design and implementation of an application from the service interface it provides to the service consumers. The advance of information and communication technology (ICT) is changing not only the business practice but also all aspects of human activities involving information exchange and processing. For example, many of the services provided by local governments are already computerized. As the complexity of the data and knowledge stored in such an information system increases, the need for a solid architectural approach such as SOA becomes indispensable in order to efficiently manage secure and reliable e-government operations. In SOA, an information system is deployed as a collection of services. Service consumers can access these services over the Internet through asynchronous messages compliant to a well-defined set of interfaces. II. T HE M ULTI -T ENANT C ONFIGURATION Scalability is always a requirement for information systems. In an SOA environment there are cases where mutually-independent enterprise systems share computer, storage and network resources. It is very difficult to ensure the scalability of such a system (or systems) by conventional single-tenant configurations in which each of the enterprise systems monopolizes its own fixed resources. The multitenant configuration maximizes the chance of effectively sharing the resources across the tenants who reside in a computer, thus providing scalability through flexibility [11]. A multi-tenant system is required to be able to separate the data, memory images, CPU time, communication channels
Publisher
Subscriber
Subscriber
Logical
Pub/Sub Channel
MR
MR MR
MR
Physical LAN/VPN/Internet
Figure 1. Implementation of the Publisher and Subscribers over the Messaging Network
and other resources belonging to different tenants. For that purpose, the controllers of the shared resources (such as OS, device drivers, virtual machine, message routers) must support user authentication, data selection and separation, and often the data-format/protocol conversion between the legacy applications and the SOA message interfaces. Further, the randomness of the traffic in the multi-tenant configuration may cause an unpredictable volume of real-time traffic. The asynchronous messages exchanged between the service consumers and providers can decouple the data source and its destination. But the asynchrony tends to increase the statistical uncertainty of the traffic (and of various resource usages) in a multi-tenant system. These observations lead to a network-centric approach to multitenant system implementations using a scalable messaging infrastructure [12], [13]. The “publish-subscribe channel” is an example of useful network-centric design patterns. It is a message channel (configured over a messaging network) that allows a sender to deliver a message to all interested receivers subscribing to the channel [3]. The senders are not required to maintain the information of the subscribers for the traffic flow control. The sender (publisher) merely publishes a new message to the publish-subscribe channel through a defined interface. If the messaging network is empowered with message routers (e.g. hardware-assisted XML processors) with enough processing capacity, a publishsubscribe channel configured over such a network can provide: 1) a shared subscription management infrastructure, 2) optimal bandwidth management, 3) shortest path content multicasting, and 4) scalability on demand. Figure 1 shows the schematic diagram of the proposed network centric approach for publisher and subscriber in the internet. The use of messaging network simplifies the interface between service consumers and providers. It can also alleviate the traffic-related problems by letting the messaging infrastructure handle the traffic uncertainty. III. C OMPUTER F ORENSICS AND SOA Many SOA systems provide services through message exchanges over the Internet. Consider the following case
related to the messaging security. An authenticated and authorized publisher (source) sends a message to authenticated and authorized subscribers (destinations). The contents of the message must not be falsified. If the datagram carrying the message is tampered in the communication network, various security functions (such as WS-Security and firewalls) available for use with networking components can detect the modified datagram in real-time [10]. However, if an authorized user (who may be actually an intruder into the user account) sends a malicious message, it is difficult for such a mechanical tool (based on a keyed-hash function, etc.) to detect the threat. “Computer Forensics” is an increasingly important area in the investigations of computer crimes [8]. Computer forensics is a professional investigation activity consisting of the phases for collection, examination and analysis of the data and reporting. We extend the application of forensics to the asynchronous messages in SOA. Logging, examination and analysis of the stored messages help the system administrator to identify malicious activity that are difficult to detect using mechanical tools available in the communication protocol layers. The messages exchanged in an SOA system consist of the following: input data to the system, output data from the system, and internal data of the system. The feasibility of the computer forensics applied to SOA depends on the well-defined interfaces, message formats and semantics an SOA system employs in order to exchange the majority data between its service consumers and providers. Therefore the messages exchanged within an SOA system are much more useful and easy to use for the forensics purposes than those exchanged with the legacy systems (for which individual analysis must be made based on proprietary data formats and protocols.) IV. I NFORMATION S ECURITY AND S YSTEMS C OMPLEXITY Information security demands a comprehensive approach. While maintaining the security level required for each part of a system, the system administrator must also address the total security. Many security tools and techniques focus on particular aspects of system security. However, to ensure the total security of an information system, we need to understand the complexity of the information system. It is often the case that the system maintenance cost is a measure of the system complexity. Enterprise Architecture (EA) refers to the analysis and documentation of the current state and the targeted state of an enterprise from an integrated strategy, business, and technology perspective [14]. In order to keep an enterprise secure, the analysis and documentation of the information security with respect to its current and target state are similarly necessary. To understand the complexity of an SOA information system with respect to the security state, we must overcome the following problems: i) The difficulty of identifying, screening, and visualizing
the security-related events in both in the current state of an enterprise and in the target state, and ii) The difficulty of clarifying, in advance, what must be done in response to the security-related events and what must be avoided. As in the ordinary computer and network systems (email servers, Web servers, remote logins, firewalls, etc.), the logging of the messages exchanged between service providers and consumers is helpful (and often the only way) to analyze the security problems of an enterprise SOA system. We use the term “Message Forensics” to refer to a systematic way of logging, examination and analysis of the messages to ensure the comprehensive enterprise SOA security. An SOA system may grow over time with the addition of services, servers, and clients. The number of transmitted messages in the system increases accordingly. The growth of services in an SOA system requires proper management of the logged messages in the system. If the messages are individually logged on each server and client host, the management and correlation of the log data will become so difficult that maintenance (and effective use) of the distributed log data will become almost impossible. Many existing security systems [15] stores the log data in a central server and do not use SOA as the basis of the security management. It still remains to be an open issue how a security management system can employ the advantage of the de-centralized SOA framework. Human error can cause damage (such as data loss) to the information systems. Many security vulnerabilities in an enterprise can also be attributed to human error. In particular, human error in operation and maintenance of an enterprise system can cause serious problems. As a simple example, an XML tag representing an amount of money can be checked for potential abnormality by simply checking the range. This kind of range checking is applicable to XML tags for timestamps, temperatures, amount of medicine, data and time, sensor readings, and so on, if the message content can be examined properly (according to the pre-specified value range). The proper management of maintenance and operation errors by fully utilizing such content-awareness (and its automation, if possible) is substantial for information security. Enterprise Architecture is a key to the successful operation of an enterprise [16]–[18] In a similar spirit, the analysis and documentation of the current state (as well as the target state) of information security is a prerequisite to the operation of an SOA system. Enterprise Security Planning (ESP) refers to this analysis and documentation process [19]–[22]. The “Security Policy” defines a set of constraints and mechanisms to protect the system from internal and external threats. The security policy of an enterprise information system should accurately reflect the current system configurations, operational practice, and limits of the security technology used in the organization. The security policy
must be based on feasible rules and procedures. Information security of an enterprise can be maintained only through feasible rules and procedures. Further, to cope with the uncertainly of the future threats, it is important to develop a medium-term as well as a long-term plans. After the rules and procedures that enforce the security policy are designed, they should be put in action and be evaluated continuously in order to see whether the whole process is practicable and can realize a secure environment. The process should be revised based on the field evaluation results. The continuous improvement of the security policy based on the feedback from the field evaluation results is the basis for the secure operation of enterprise information systems. V. N ETWORK -C ENTRIC P UBLISH /S UBSCRIBE C HANNELS We consider the use of the messaging networks [23]–[25] as the basis for effectively managing the complexity of SOA systems. In particular, a messaging network serves as the primary security-protection mechanism for an SOA system when the sources and the destinations of the messages are authenticated and authorized and the network is equipped with message logging facilities. Those applications characterized by large, dynamic numbers of content producers and consumers are referred to as publish/subscribe applications [3]. The publish/subscribe applications generate an unpredictable volume of real-time traffic. These applications require the messaging network to deliver the data to endpoints having varying interests in various content streams. A sensor network is an example that consists of publish/subscribe applications [23]–[25]. The messaging network enables asynchronous communication among the publish/subscribe applications while fully decoupling the sources (publishers) and the destinations (subscribers.) The functions of messaging network that supports the publish/subscribe-style communication can be categorized into three types of communication channels: subscriber-centric, publisher-centric, and network-centric. In a subscriber-centric channel, the content distribution logic resides at the subscriber side. The publishers do not possess any knowledge of the individual interest of the subscribers. The publishers send data to the channel as needed. The subscribers must choose which data to pick up. The subscribers also need to filter the content and convert the data format for the application if necessary. Such local and distributed content filtering and format conversion incur high processing overhead at the subscriber side and increase the complexity of the system. System complexity results in high maintenance costs. In a publisher-centric channel, the content distribution logic resides at the publisher side. Each subscriber registers with a publisher to receive the publisher’s content, leaving the publisher to manage its local subscription database. The
local management of subscription by the publishers is susceptible to the data integrity problem. Whenever a subscriber joins or leaves the publish-subscribe channel, all publishers that subscriber wants to receive from must update their subscriber lists. It is difficult to ensure the real-time integrity of such local databases in a large scale system. Further, the publisher-centric channel is not bandwidth optimal, because a publisher or some multicast routers in the network must send/copy the same content across the network N times if N subscribers register with identical subscription conditions. The network-centric channel can overcome the problems of the subscriber-centric and publisher-centric channels. In a network-centric architecture [3], the publish/subscribe channel manages the content delivery logic. The subscribers indicate their subscription preferences to the messaging network (not to the individual publishers.) The content delivery mechanism itself is realized as a publish/subscribe system. A publisher does not maintain the information about its subscribers. The messaging network manages the complex flow control requirements needed for the publish/subscribe traffic. Publishers merely send new content to the network whenever it becomes available. A network-centric publish/subscribe architecture provides 1) a shared subscription management infrastructure, 2) bandwidth management, 3) shortest-path content multi-casting, and 4) scalability on demand to accommodate continuous changes of the network. In view of the utility of publish/subscribe channels in modern information systems, we consider the network-centric messaging networking is most suitable as the foundation of SOA systems. VI. M ESSAGING N ETWORKS “Total optimization” (in contrast to local optimization) is recognized as important for information systems [21], [26], [27]. Similarly “total safety/security” is important for information security [21]. While maintaining the security level required for each part of the system, the system administrator must ensure total safety/security. Many security tools and techniques focus on a particular aspect of system security. Our focus is on the use of messaging networks as a foundation of the total safety/security of SOA systems. A messaging network is an overlay network set up on various kinds of communication networks (typically IP networks). Figure 2 shows the layered structure of the messaging network. Message Routing Applications communicate using the data-format and addressing scheme of the messaging network and no longer need to consider the IP networking details such as the IP address and transport-layer protocols. A messaging network consists of message routers and edge gateways. The edge gateways provide mediation interfaces for legacy applications. In addition to the basic message routing function, a messaging network authenticates the users, identifies and
Messaging Patterns (Pub/Sub Channel, etc.)
Message Logging
Message Routing
Mediation Layer
TCP/UDP
IP
Datalink
Figure 2.
Layered Structure of the Messaging Network
categorizes the services and provides message mediation functions between services as well as for interoperability with legacy applications. A messaging network also validates the messages published on a publish/subscribe channel. A messaging network can support a number of message exchange patterns (MEP) [3]. The synchronous messages (the request-response exchanges) and asynchronous messages (one-way messages) are common exchange patterns in SOA. Another important capability of a messaging network is content-based routing. In SOA, standard Web interfaces such as SOAP separate a service from the software implementation of the service. XML is a standard format to describe the content of a message. For interoperability purposes, all services in a messaging network should conform to a standard such as XML whenever possible. The legacy contents must be converted into the standard format when the contents interface with the messaging network. All functions of an SOA system are carried out through message exchanges. The messages are thus the key object to understand the system behaviors. VII. M ESSAGE F ORENSICS Our concept of message forensics of an SOA system derives from the following considerations. Even an authenticated and authorized user might still send malicious messages intentionally. Therefore, the messages should be archived for analysis. An authenticated and authorized user may send an incorrect message by mistake. Therefore, the messages should be logged/recorded in order to identify incorrect messages due to human error. A chain of messages often indicates the activity of a business process (or administrative operations.) Therefore, a series of messages properly recorded can be used to visualize the business process and associated use cases. It is not easy to find malicious messages and incorrect messages sent by an authenticated and authorized user by simply monitoring the messages. A user may intentionally send an incorrect message with malicious intent. Such an
inadequate message may not include those malicious keywords that the signatures of an intrusion detection system (IDS) can pick up. Detection of such messages requires close inspection of the content of the messages. If we rely only on packet monitoring, we have to recover a meaningful message from the captured packets. Sometimes the physical packet size is much larger than the message size contained in the packet, because the packet consists of various information of lower layer protocols. A large packet must be analyzed in order to reconstruct a chain of messages for content inspection. Based on the observation of the above requirement, we introduce “Message-oriented Forensics” (or simply “Message Forensics”) as a primary tool for SOA security. Message forensics requires that all the relevant messages in the system be directly stored in a message store (which may be physically distributed) and the stored messages can be easily examined by the authorized administrators. The messages relevant to any particular message must be able to correlate with each other as a set of messages corresponding to a business activity. Message forensics helps traditional computer forensic systems [8] in analyzing the events in the system. In view of “Web As A Platform” (WaaP) in the Web2.0 (and onward) trend, the Internet is a medium to provide a large number of channels (= software applications) as services. A message history is a kind of log data of the Enterprise Service Bus (ESB). The message history is as important as the log of the bus in a single computer system. As stated above, human errors cannot often be detected mechanically. We need a management scheme to reduce human errors and to cope with the troubles caused by human errors. If the messaging network can record the messages in real-time, typical incorrect messages can be detected and the user can be immediately notified about the incorrect message. The messaging network can classify and store the entire messages into a message store by using message routers. This real-time processing of high-volume messages has become feasible owing to the progress of hardwareassisted message routing technology. For a large SOA system, it is difficult to manage the log data of each application in each system. The log data must be gathered before investigation. The logging system management of a large number of services is also difficult and incurs high costs. The huge number of log data causes information explosion or “info-plosion.” The application software in an SOA system usually exchanges not a single message but a number of associated messages as well. Thus a set of messages can indicate the usage and behavior of the application. Such a set of messages reveals the entire process in which the application is used. Therefore centralized message logging is more useful than logging in individual applications. The (hardware-assisted/high-performance) message routers [12] can effectively collect the messages in
a message store. The collected messages can be organized using an XML database management system (XML-DBMS). An XMLDBMS is a basis to provide the following key services for SOA system management: - Visualization of the business process, - Visualization of the security status and -Identification of typical human errors. The management of such a uniform message database (which may be physically distributed) is much easier than the management of log data stored in individual hosts. VIII. M ESSAGE L OGGER AND M ESSAGE A RCHIVER Message history is a data structure (associated with each message) that maintains the list of all hosts and filters through which the message has passed [25]. Message history is an effective tool in debugging and analyzing messagebased systems [25]. Message history reveals the business process involved through the list of hosts and filters that have processed the message. Information of the filters and hosts is appended to the message history at each point. Once a message is consumed, the message history disappears. In order to save the message and its history, there must be a logging function at each point that can copy, process (if needed) and send the message and associated message history to the message store. We introduce two new functions “Message Archiver” and “Message Logger” together with an extended function “Message Store Grid.” A message archiver exists in all message routers and records the messages themselves. A message archiver adds the information of the message source and destination to each copy of the messages. This information enables the administrator (and message forensics tools) to reconstruct the route a message has taken. Message archives are prepared along message routers. A message archive is a kind of distributed XML database. Many DBMS can maintain the message archives as a distributed XML-DB. It is difficult for a single message store to manage entire message histories. Therefore, a message store grid is necessary to maintain the message histories. The message store grid is a distributed XML database and should be managed by the same XML-DBMS that manages the message archives. The most important function “Message Logging” manages message histories, the message store grid, and message archives. Message logging processes dynamically captures the message-based network and manages the captured information. The proposed approach logs the messages that are transferred in a message network by using publish/subscribe message routing to record the content publishing events, content subscription events, the publishers, and the subscribers. The proposed approach also requires to keep the logs of the XML transformation events. A messaging network
can provide a network-based transformation between XML schemes or between XML and text formats, such as HTML. By utilizing the services Message Logger and Message Archive provide the proposed message forensics can detect malicious content interference or Denial-of-Service (DoS) attacks hidden within XML traffic through event correlation of stored log data based on content-aware analysis of XML tags and valudes. IX. C ONCLUSIONS In this article, we proposed message forensics to manage SOA complexity. We have discussed the management of the message for an SOA-based information system. Messages logging can be used for visualization of the business process and the security associated with the system. The proposed message forensics can enhance the packet-level forensics by enabling the analysis of the correlated messages that consist of a business process activity. R EFERENCES [1] D. Linthicum et al., Enterprise Application Integration. Addison-Wesley Professional, 1999. [2] M. Papazoglou and D. Georgakopoulos, “Service-Oriented Computing,” Communications of the ACM, vol. 46, no. 10, pp. 25–28, 2003. [3] G. Hohpe, B. Woolf, and K. Brown, Enterprise Integration Patterns: designing, building, and deploying messaging solutions. Addison-Wesley Professional, 2003. [4] P. Bernus, T. Williams, and L. Nemes, Architectures for Enterprise Integration. Springer, 1996.
[11] S. Aulbach, T. Grust, D. Jacobs, A. Kemper, and J. Rittinger, “Multi-tenant databases for software as a service: schema-mapping techniques,” in Proceedings of the 2008 ACM SIGMOD international conference on Management of data. ACM, 2008, pp. 1195–1206. [12] Sonoa Systems, “ServiceNet Technical Brief,” 2008. [13] Solace Systems, “Application Multicast,” 2006. [14] M. Lankhorst, Enterprise architecture at work: Modelling, communication and analysis. Springer-Verlag New York Inc, 2009. [15] N. Zuk, “Detection of network security breaches based on analysis of network record logs,” Jan. 29 2008, uS Patent 7,325,002. [16] J. A. Zachman, “A framework for information systems architecture,” IBM Syst. J., vol. 26, no. 3, pp. 2764–292, 1987. [17] M. Javanbakht, M. Pourkamali, and M.-R. Derakhshi, “A new method for enterprise architecture assessment and decisionmaking about improvement or redesign,” in Computing in the Global Information Technology, 2009. ICCGI ’09. Fourth International Multi-Conference on, aug. 2009, pp. 69 –76. [18] Y. Zhao, “Enterprise service oriented architecture (esoa) adoption reference,” in Services Computing, 2006. SCC ’06. IEEE International Conference on, sept. 2006, p. 512. [19] G. Peterson, “Security architecture,” Information Security Bulletin, vol. 10, pp. 325–330, 2005. [20] L. L. DeLooze, “Applying security to an enterprise using the zachman framework,” SANS Institute, Tech. Rep., 2001. [21] T. Hayashi, “Schemes for realizing total security in information systems,” in Proc. of 5th International Conference on ICT and Higher Education, June 2006.
[5] T. Hayashi, H. Fukuhara, M. Hisada, K. Suzuki, T. Yamada, Y. Watanabe, J. Terazono, T. Suzuki, T. Miyazaki, S. Saito, I. Koseda, and J. Iwase, “A network-centric approach to sensor-data and service integration,” in SICE 2011, 9 2011, pp. 2037–2042.
[22] ——, “Zachman framework for realizing information security of local governments,” Journal of Social Informatics, vol. 10, no. 1, pp. 73–82, march 2007.
[6] A. Nakao, R. Ozaki, and Y. Nishida, “CoreLab: an emerging network testbed employing hosted virtual machine monitor,” in Proceedings of the 2008 ACM CoNEXT Conference. ACM, 2008, pp. 1–6.
[23] Y. Choi, J. Terazono, K. Kawauchi, K. Itabashi, H. Fukuhara, I. Koseda, R. Fujita, T. Miyazaki, S. Saito, J. Iwase, and T. Hayashi, “Network–centric mashup for a sensor network that uses a messaging network,” in ICCAS-SICE, 2009, Aug. 2009, pp. 1980 –1983.
[7] M. Kitsuregawa, “Challenge for info-plosion,” in ALT ’07: Proceedings of the 18th international conference on Algorithmic Learning Theory. Berlin, Heidelberg: Springer-Verlag, 2007, pp. 1–8.
[24] T. Hayashi, H. Fukuhara, R. Fujita, T. Miyazaki, and S. Saito, “A messaging network to realize an soa-based system,” in Proc. of CIT2007. IEEE, Oct. 2007, pp. 1083–1088.
[8] B. Schneier and J. Kelsey, “Secure audit logs to support computer forensics,” ACM Transactions on Information and System Security (TISSEC), vol. 2, no. 2, pp. 159–176, 1999.
[25] J.Terazono, H. Fukuhara, R.Fujita, I. Koseda, S. Saito, T. Miyazaki, , and T. Hayashi, “Service oriented architecture realized by a messaging network,” in Network Operations and Management Symposium (NOMS), 2010 IEEE. IEEE/IFIP, Apr. 2010, pp. 934 –937.
[9] F. Curbera, F. Leymann, T. Storey, D. Ferguson, and S. Weerawarana, Web services platform architecture: SOAP, WSDL, WS-policy, WS-addressing, WS-BPEL, WS-reliable messaging and more. Prentice Hall PTR, 2005.
[26] M. Fowler, Patterns of Enterprise Application Architecture. Addison-Wesley, 2002.
[10] R. KANNEGANTI and P. Chodavarapu, SOA Security in Action. Manning Publications, 2007.
[27] J. Ross, P. Weill, and D. Robertson, Enterprise architecture as strategy: Creating a foundation for business execution. Harvard Business Press, 2006.
