PERFORMANCE MONITORING OF JAVA COMPONENT-ORIENTED ...

PERFORMANCE MONITORING OF JAVA COMPONENT-ORIENTED DISTRIBUTED APPLICATIONS Adrian Mos, John Murphy Performance Engineering Lab, Dublin City University Glasnevin, Dublin 9, Ireland Tel: +353 1 700-8762, E-mail: {mosa, murphyj}@eeng.dcu.ie Abstract: We present a framework for monitoring the performance of component oriented distributed applications based on the Enterprise Java Beans specification. The environment leverages EJB architecture to monitor existing applications in real-time and to provide detailed run-time information that help identify performance hotspots at an object-oriented level. It is non-intrusive, portable across all EJB compliant application servers and easily extendable to accommodate new data-acquisition or graphical presentation components. Current status of the work serves as proof of concept and a complete implementation is under development. KEYWORDS: performance, monitoring, EJB, component-oriented, distributed systems

INTRODUCTION Distributed systems have proven to be the solution for building reliable and scalable Internet systems because of their inherent approach in managing computational resources. Mission critical applications such as e-commerce sites, network management and quality of service solutions, can all benefit from the advantages of object oriented distributed technologies. CORBA has established itself as the de facto technology for implementing such systems but since recently, developers can also choose the Enterprise Java Beans [1] framework as an alternative solution. Enterprise Java Beans (EJB) is the newest distributed-objects technology built on top of the high-performance Java RMI-IIOP [2] protocol and its declared purpose is to provide a framework for fast and efficient development of highly scalable and reliable applications. However, just as with CORBA or other distributed technologies, the performance of the implemented applications is not guaranteed by the technology alone, it must be achieved through good design and coding practices. There is a need for tools that aid the developers in designing and implementing distributed applications to perform as expected. We present a framework for monitoring the behaviour of EJB applications with the aim of identifying and correcting scalability problems so that the application can meet its performance requirements. Our work is similar to other approaches to performance monitoring in that it enables gathering of analytical data during the execution of the monitored distributed application. However, we focus exclusively on the EJB technology, which has important distinct architectural issues compared to CORBA (where most of the research has been done). The tool, named EJBMonitor, leverages EJB architecture to monitor existing applications in real-time and to provide detailed run-time information that helps the developer identify performance hotspots. EJBMonitor is not intrusive in that it does not require any modifications to be made to the existing distributed application and it is completely portable across all EJB compatible operating environments.

1. RELATED WORK Most of the research and work in monitoring the performance of distributed systems is in the area of CORBA applications and we present a few relevant projects. OrWell [3] is a monitoring environment for CORBA distributed applications, which uses an event class hierarchy to notify the observers of interaction in a system. It provides detailed analysis of the monitored system, however the authors do not present how the event distribution units EDP are dynamically attached to the existing objects. In addition, it is not clear whether the monitoring environment is portable on different operating systems or not. The main similarity with our work is in the monitoring concepts of using one additional component for each monitored object in order to obtain dynamic run-time information. Wabash [4], [5] is a tool for testing, monitoring and control of CORBA distributed systems. It uses CORBA interceptors to capture run-time information and therefore is similar to EJBMonitor in that it is non-intrusive. Wabash uses geographical information to group monitoring components which is non applicable in EJB environments. For the new Enterprise Java Beans technology there are no generic monitoring frameworks available. A number of application servers provide a certain degree of monitoring but most of them do so at a network/protocol level, giving little help to OO developers who want to understand which component/method is having the scalability problem. To correct this situation, a few third-party plug-ins [6] were developed for a small number of application servers but they are targeted at a specific application server, on a specific platform, offering little flexibility in choosing the development environment. A separate category of performance-related EJB tools is represented by stress testing tools. Products such as Silk Pilot [7] and Bean Test [8] automatically create test clients for each EJB and run scripts with different numbers of simultaneous such clients to see how the EJBs perform. The main disadvantage of such a solution is the fact that it does not gather information from a real-life system but from separated components. Without monitoring the actual deployed system, it is difficult to obtain an accurate performance model for the entire system. Sun Microsystems have launched an EJB performance initiative called ECPerf [9] which is a benchmark designed to measure performance and scalability of EJB environments. It consists of a complex distributed application and the load-generation scripts to test it on any given EJB server. It is useful for evaluating the performance of specific application servers but it cannot help identifying any problems in a distributed application. It does not provide any monitoring framework and it is mainly used by EJB server vendors to showcase their products, giving little help to application developers.

2. BACKGROUND The growing size of Internet systems demands for strong distributed architectures that are difficult to build even for experienced developers. Sun Microsystems proposes Java 2 Enterprise Edition [10] (J2EE), a standard for developing multi-tier enterprise applications, which comprises a number of Java technologies such as Java Servlets, Java Server Pages, Enterprise Java Beans, Java Naming and Directory, Java Database Connectivity, RMI-IIOP, Java IDL, Java Message Service, XML. Enterprise Java Beans framework is the central part of the middle tier in a J2EE application. It defines a model for the development of Java server-side components. These components, called Enterprise Java Beans (EJBs) are Java objects that reside in the logical application server environment. The server can spawn multiple physical machines that together constitute a cohesive computational environment. It is responsible for providing the services that components require, such as database transaction contexts, messaging or security. To benefit from these

services, each component must declare them EJB Container in a deployment descriptor (XML file) that the server uses at deployment time to generate the EJB Home 1 wrapper EJB Object class whose instances the Client Code Java Servlets 2 clients will use in place of the actual Actual bean Java Beans Java App. component instance. This wrapper object EJB Object 3 4 implements the remote interface created by the developer to expose the same methods to the clients that the actual EJB exposes. Note that a client can be any logical entity from a webFigure 1. Accessing an EJB based Java application using RMI/HTTP to a network management node accessing the component through CORBA (i.e. IIOP) since EJB technology uses RMI-IIOP as the transport protocol. Figure 1 illustrates the interaction between a client and an Enterprise Java Bean in a container: 1. The client obtains a reference to the Home Object of the deployed bean (generated automatically by the EJB server at deployment time). This is done using Java Naming and Directory Interface [11] (JNDI). It will then ask the home object to create an EJB Object so that it can access the bean’s ‘public methods’. The public methods are the operations that the Actual Bean will expose to clients. 2. The EJB Home object creates a new EJB Object for the required bean. This object is not the actual bean (created by the application developers) but a server-generated object that will act as a proxy between the client and the ‘real’ enterprise bean. The purpose of this object is to provide additional services such as security or transactions to the EJB component. It will have all the public methods of the enterprise bean and will delegate the requests to the real enterprise beans. 3. After the client receives a reference to the newly created EJB Object, it can begin using the ‘public methods’ of the bean. Since the client doesn’t have direct access to the enterprise bean, it will invoke the desired methods on the server-generated object. 4. The EJB Object forwards all method invocations to the actual bean, which will perform the required operations and return the results to the EJB Object, which in turn will return the results to the client. Since EJB Object provides a number of services to the actual bean, its functionality is more complex than just forwarding method calls. For example, it could check to see if the client has the rights to call a specific method and raise an exception if it doesn’t. Alternatively, it could spawn a transaction and call the required method in the transaction’s context. All this additional processing is performed transparently to the client and for the developer it does not involve writing any code.

3. NON-INTRUSIVE MONITORING 3.1 Overview In order to build a monitoring system that is portable across all J2EE server platforms we need a way to capture and analyse invocation data (method calls that take place inside the running system) with no alterations to the server or the application. Therefore, a number of monitoring components have to be inserted into the application’s environment, which is in fact the EJB container. So, in other words, a monitoring application composed of monitoring EJB components will be deployed in the EJB server so that it can gather information about the already deployed application. An overview of the interaction between application components and monitoring components is presented in Figure 2. The most important parts of the monitoring system are the proxy EJBs and the monitor EJBs.

•

The proxy EJBs correspond on a 1 to 1 basis with the application EJBs. They are automatically generated to implement the same public interface as the application EJBs. Upon deployment, the proxies will use the identities of corresponding application EJBs in order to receive the client invocations. They will forward the invocation to the original recipient after notifying the monitor about it. Figure 2. Using proxy EJBs • The monitor EJBs are responsible for interpreting the invocation notifications, process the results, and send filtered information to registered listeners, such as graphical consoles. For the rest of this Section, we will describe the architecture and functionality of EJBMonitor, a tool that uses this approach for monitoring EJB distributed systems. 3.2 Design and Implementation of EJBMonitor EJBMonitor is a non-intrusive performance monitoring system that can be used to monitor J2EE compatible application servers implementing EJB 1.1 and above specifications. In its current stage, it displays basic statistics about method invocations but it can be extended to display extremely complex analytical graphs and data simply by adding additional invocation handlers to EJBMonitor’s modular architecture. Probably the most important components in the framework are the proxy [12] EJBs. There is one proxy bean for each EJB from the original application. This proxy bean is automatically created by the EJBMonitor’s installer and it implements the same remote interface as the target bean. When the proxy bean will be deployed in the container’s context, it will have the same JNDI name as the target EJB. In this way, the client does not know that the server component has changed. We have identified three possible ways to automatically generate proxy beans that implement a given remote interface: • By reading the XML deployment descriptor of the target EJB, the name of the remote interface can be obtained. Then, the Java source code for the proxy bean can be generated and compiled into the proxy EJB class. Note that the source code for that class is minimal since all the added functionality is inherited or obtained through the adapter [12] mechanism. • By using the new Proxy class from the Java Reflection API (since JDK 1.3), the proxy beans could implement the remote interface of the target EJB at run-time. However, there are a number of issues regarding the use of remote reflection in Java [13] that although have been approached by the research community, still need more attention. Primarily they are related to the way that containers may spawn Java Virtual Machines (JVM) for each deployed component, making it difficult to use the reflection mechanism. • Again by using the Proxy mechanism from the Java Reflection, this is a variation of the second approach. It makes it possible for the proxy component to dynamically implement the public interface of the target component at run-time without the disadvantages presented above. The only condition is that the component’s EJB class has to be physically copied from the original application to the monitoring application so that it is available for the Proxy mechanism.

We are considering all three approaches and the most suitable one will be chosen for development. However, in the current implementation, the proxy beans are generated manually in order to be functional and help demonstrate the system’s abilities. The monitoring components are implemented as stateless session EJBs and are called by the proxy EJBs whenever an event such as a method invocation occurs. All the monitoring components are identical and so they can be cached in a session pool, thus improving the framework’s performance. When a monitoring component receives an event from the proxy components, they send a message to a specific destination using Java Message Service [14] (JMS), a specification implemented by all J2EE compliant application servers. JMS is a Java interface for messaging systems, such as SonicMQ [15], and application servers implementing it must provide a messaging server. The listeners registering for events from the monitoring framework (i.e. graphical consoles) must register themselves as message listeners for that particular destination. This allows the monitoring framework to be completely decoupled from its listeners and to incur minimum overhead on the target application. 3.3 The Functionality of the Monitoring Framework The EJBMonitor system is designed to be completely J2EE compliant, which means that it can be used with any application server that is J2EE compatible. Developers and system integrators would use EJBMonitor during development time (i.e. when they are creating the distributed applications) and also after the deployment of the application when performance problems need to be identified. Figure 3 presents a snapshot of the graphical console in its current implementation. It has two main areas. The upper section shows the registered proxy EJBs with their correspondent target EJBs and the number of instances in the container’s pool for each of them. They are added dynamically when they are first used in the application (i.e. one of their methods is invoked). The lower section Figure 3. EJBMonitor Console displays the methods as they are being invoked, the total number of invocations so far and the average measured execution time in milliseconds. Also present in the snapshot, above and below the console, are real-time graphs generated by the monitoring system. They illustrate the evolution of the execution times for selected methods in the monitored application. These graphs provide an important visual aid for developers to identify unexpected increases in execution time and therefore the methods that do not deliver the expected performance and scalability parameters. 3.4 Advanced Monitoring: Automating the Monitoring Process As presented above, the monitoring framework uses a messaging system to notify the graphical consoles of important events. We consider the use of time-stamps, which are automatically provided by the messaging infrastructure, for marking the events. Therefore, events can be ordered in a logical structure and transactions can be structured in a tree format. An example of a transaction would be obtaining information about a Service Level Agreement (SLA) and allocate the needed network resources for providing the contracted service. The top-level

components/methods would be the first components/methods along the invocation path, and lower-level components/methods would be at the end of the transaction invocation path. Having the graphical tree structure, developers can use a ‘divide and conquer’ approach to identify the faulty method or component by browsing down in the transaction tree. This is a feature that will be added in the near future to the EJBMonitor framework. The tree representation of a transaction and the browsing process are illustrated in Figure 4. Automating the browsing EJB OK process in the transaction tree is M1 M2 Mk Mn EJB OK EJB OK M1 M2 Mk Mn M1 M2 Mk Mn a desirable feature that we are EJB ALERT EJB ALERT M1 M2 Mk Mn taking into consideration. The M1 M2 Mk Mn EJB ALERT EJB OK M1 M2 Mk Mn M1 M2 Mk Mn EJB ALERT system would then be able to EJB OK M1 M2 Mk Mn ALERT EJB detect the components that are M1 M2 Mk Mn M1 M2 Mk Mn performing poorly (i.e. are showing severe degradation of Figure 4. Browsing down the invocation tree performance parameters when usage increases). Under these conditions, when one faulty component is identified in the transaction invocation tree, the system would browse further down onto that component’s subtree to eventually identify the particular component(s) that are individually responsible for the degradation of the transaction’s entire performance loss.

4. EXPERIMENTAL RESULTS 4.1 Testing Environment A number of tests have been carried out in order to measure the impact of the monitoring system on the target application. In addition, we have performed tests that measure the accuracy of the monitoring process and how monitoring can help identify performance hotspots in the application design. For testing purposes, a small application consisting of a HTML page, a Java Servlet and two Enterprise Java Beans has been built. Users use a web browser to load in the HTML page, which has a simple form. After completing the form, the user submits it to the Java Servlet object, which in turn calls all the methods of the two EJBs. The J2EE application server used in testing was BEA WebLogic 6.1 [16]. The tool used in the recording and replication of user interaction was OpenSTA [17]. The hardware consisted of a dedicated application server with 4 Intel Pentium III Xeon processors and 512 MB of RAM and one load-generation server with 2 Intel Pentium III Xeon processors and 512 MB of RAM, interconnected in a 100Mbit LAN.

Figure 5. Monitoring overhead

Figure 6. Measurement accuracy

4.2 Monitoring Overhead We have performed tests in order to measure the overhead of the monitoring system on the global application response time. The results are depicted in Figure 5. The dotted line describes the response time when monitoring is enabled. Tests show that even with a large increase in the number of simultaneous users, the monitoring process does not affect the application response time significantly. This is largely due to the efficient use of the messaging infrastructure that enables the monitoring component to induce minimum overhead when notifying the listeners (i.e. graphical consoles). 4.3 Measurement Accuracy In Figure 6, the dotted lines represent the average measured execution time reported by the monitor while the solid lines represent the known execution time. The results show that the measurement process is accurate even as the number of simultaneous users increases. 4.4 Detecting Performance Hotspots We envisage that a developer team would use the monitoring framework to identify the causes of the increase in execution time presented in Figure 5. They would know that a number of factors influence the growth in execution time (i.e. web-server, bandwidth) but they would like to find out the exact issues (if any) are present in the application server. Using the EJBMonitor, they would get the graphs shown in Figure 3 that clearly show the methods that are not scalable enough to efficiently handle the load. By adding the features presented in Section 3.4, we believe that EJBMonitor could dramatically reduce the time needed to identify faulty components in the application.

5. CONCLUSIONS AND FURTHER WORK Building large distributed systems is a complex task and tools are needed to help developers create high performance such applications. Enterprise Java Beans technology addresses a number of issues of generic distributed systems by simplifying the development of large component oriented applications. However, by itself, it does not guarantee that they will perform as expected under heavy loads. We propose a monitoring environment for Enterprise Java Beans technology that helps identify performance problems at an object-oriented level thus helping developers understand the nature of the problem. The tool, named EJBMonitor uses the EJB architecture to dynamically generate proxy objects that capture all incoming method calls and other events in the EJB application. They notify monitors of these events and the monitors are responsible for sending processed data to graphical display consoles and other listeners. We compared our work with other related projects and pointed out the differences and similarities. The EJBMonitor environment is still under construction and there are a number of components, especially in the static section that have to be implemented yet. We also plan to collect more information from the running system and add features such as time-stamps for ordering the events and provide functionality to track an entire invocation path from the client to the system’s backend. A number of graphical views that display charts and graphs are also to be added in the future in order to help the developers gain a better understanding of the running system. We believe that the presented framework can provide an important aid in identifying and correcting performance issues in complex distributed systems that are based on the Enterprise Java Beans technology, and the fact that it is completely portable would prove particularly useful when these systems are deployed in heterogeneous environments.

REFERENCES [1] Sun Microsystems, “Enterprise Java beans Technology”, http://java.sun.com/products/ejb/. [2] Matjaz B. Juric, Ivan Rozman, Simon Nash “Java 2 Distributed Object Middleware Performance Analysis and Optimization”, ACM SIGPLAN Notices, August 2000, No. 8, pages 31-40. [3] R. Weinreich, W. Kurschl, “Dynamic Analysis of Distributed Object-Oriented Applications”, Proceedings of the Hawaii International Conference On System Sciences, January 6-9, 1997, Kona, Hawaii. [4] B. Sridharan, B. Dasarathy and A. P. Mathur, “On Building Non-intrusive Performance Instrumentation Blocks for CORBA-based Distributed Systems”, 4th IEEE International Computer Performance and Dependability Symposium, Chicago March 2000. [5] B. Sridharan, S. Mundkur and A. P. Mathur, “ Non-intrusive Testing, Monitoring and Control of Distributed CORBA Objects”, TOOLS Europe 2000, St. Malo, France, June 2000. [6] “Precise/Indepth for the J2EE platform”, http://www.precise.com/products/indepth.asp. [7] Segue, “SilkPilot”, http://www.segue.com/html/s_solutions/s_corba/s_spilot_overview.htm. [8] Empirix, “Bean Test”, www.empirix.com/empirix/web+test+monitoring/products/. [9] Sun Microsystems, “ECPerf Version 1.0”, http://java.sun.com/j2ee/ecperf/. [10] Sun Microsystems, “Java 2 Platform Enterprise Edition”, http://java.sun.com/j2ee/. [11] Sun Microsystems, “Java Naming and Directory Interface (JNDI)”, http://java.sun.com/products/jndi/. [12] Mark Grand, “Patterns in Java”, Wiley Computer Publishing, 1998. [13] Richmond, M. & Noble, J., “Reflections on Remote Reflection”, Australasian Computer Science Conference (ACSC) 2001, Brisbane, Jan 2001. [14] Sun Microsystems, “Java Message Service (JMS) API", http://java.sun.com/products/jms/. [15] Sonic Software, “SonicMQ Message Broker”, http://www.sonicsoftware.com/products/index.htm. [16] BEA Systems, “BEA WebLogic Server”, http://www.bea.com/products/weblogic/server/index.shtml. [17] OpenSTA, “OpenSTA (Open System Testing Architecture)”, http://www.opensta.org/.