State Capture and Resource Control for Java: The Design ... - CiteSeerX

State Capture and Resource Control for Java: The Design and Implementation of the Aroma Virtual Machine1 Niranjan Suri, Jeffrey M. Bradshaw, Maggie R. Breedy, Kenneth M. Ford, Paul T. Groth, Gregory A. Hill, and Raul Saavedra Institute for Human & Machine Cognition, University of West Florida {nsuri,jbradshaw,mbreedy,kford,pgroth,ghill,rsaavedra}@ai.uwf.edu Abstract Current versions of Java fail to address many of the unique challenges posed by the new generation of distributed applications and mobile agent systems. Aroma is a Java-compatible Virtual Machine (VM) that provides unique features such as the ability to capture the full execution state of the VM or individual thread execution state, and the ability to dynamically control disk, network, and CPU resources. This paper describes the design and implementation of the Aroma VM focusing on the features of the VM that support state capture and resource control. 1.

Introduction

Although Java is currently riding a rising wave of popularity, current versions fail to address many of the unique challenges posed by the new generation of distributed applications. In particular the advent of peer-to-peer computing models [1] and the proliferation of software agents [2,3] motivates various requirements that go beyond the capabilities of current Java Virtual Machines: •

Full state capture. To support checkpointing and load balancing, the Virtual Machine (VM) must be able to capture its complete state including all threads, objects, and classes in the heap. To support requirements for strong “anytime” mobility and forced migration (such as when a host is about to go offline), the VM must be able to support asynchronous requests to capture execution state for a thread or thread group.

•

Dynamic access and resource control. The security model in Java 2 [4] provides a fairly comprehensive model for access control but does not allow for dynamic permission revocation. Once permission is granted to a process, that permission is in effect for the lifetime of that process. Furthermore, there is no way of specifying the amount of a resource that is granted, assuring a specific quality of service for each process. For example, one would like to be able to limit the quantity of hard disk, network, or CPU usage that is available to a given process or to determine the rate at which the resource may be used. Denial-of-service conditions on a host or network resulting from code that is poorly programmed, malicious, or has been tampered with are impossible to detect and interrupt without dynamic monitoring and control mechanisms for individual processes.

•

Resource accounting. Tracking of resource use, based on resource control mechanisms, enables accounting and billing mechanisms that hosts can use to calculate charges for resident programs. The same mechanisms can be used to detect patterns of resource abuse.

The Aroma VM is a Java-compatible VM that provides unique capabilities such as thread and VM state capture and dynamic, fine-grained resource control and accounting. Aroma was developed as part of a research project on mobile agent systems and distributed systems. The overall goal was to develop a VM for research use that would be simple, flexible, and portable. Therefore, Aroma tries to minimize dependence on operating systems features and avoids platform-specific assembly code. Aroma is currently being used as part of the NOMADS mobile agent system [2] and in the WYA (While You’re Away) distributed system for load balancing and utilizing idle workstations (see section 5).

1

This research is supported in part by DARPA’s Control of Agent-Based Systems (CoABS) program and the National Technology Alliance (NTA) / National Imagery and Mapping Agency (NIMA)

In section 2 we describe the overall design of the Aroma VM followed by a description of the implementation in section 3. Section 4 summarizes performance results and section 5 describes the NOMADS and WYA applications. After a discussion of related work (section 6), we conclude with a summary (section 7). 2.

Design of Aroma

The Aroma VM is designed as a set of C++ classes. Figure 1 shows the major components of the Aroma VM. The primary data structure is the class JavaHeap, which is the heap of objects. The heap may store many types of objects such as simple Java objects (i.e., instances of Java classes), arrays of primitive data types, arrays of other objects, classes, and threads. Simple Java objects are represented by the class JavaObject. Each such object contains a field table that provides storage for member fields, a pointer to the class that defines the object, and an optional pointer to a monitor that is used for synchronization. Loaded Java classes are represented by the C++ class JavaDefinedClass. Class objects (i.e., instances of JavaDefinedClass) are subclasses of regular objects (i.e., instances of JavaObject) and hence inherit their structure. In addition, class objects contain data structures that are loaded from class files such as the constant pool, method descriptors, code attributes, and so forth. Class objects also contain field tables for static fields defined by classes. Java threads are represented as instances of the C++ class JavaThread. Each thread object contains a method stack that contains stack frames for methods invoked by the thread. Each stack frame contains an operand stack, a set of local variables, a program counter, and a pointer to the method corresponding to the stack frame. Method stacks in Aroma can also contain special stack frames known as VM stack frames that enable finer granularity in state capture. These special stack frames are described in section 2.2. Virtual Machine JavaThread Threads

ExecutionUnit

Heap Java Object

JavaStackFrame

JavaLangThread Class JavaObject JavaThread JavaPrimitive ArrayClass JavaThread JavaPrimitive Array JavaReferece ArrayClass JavaPrimitive Class JavaReference Array JavaDefined Class

LoadClassVM StackFrame JavaStack Frame JavaStack Frame InitClassVM StackFrame KickerVM StackFrame

Operand Stack

Method Stack

JavaExecutionUnit

Local Variables

Program Counter

Native Thread

Operating System Services

Java Object

JavaDefinedClass System ClassLoader

ConstantPool

FieldDefinitions

SuperClasses

StaticFields

DefiningClass

Interfaces

MethodDefinitions

FieldTable

Attributes

MethodTable

JavaObject

MonitorHandler Monitor

Monitor

Monitor

Monitor

Figure 1: Structure of the Aroma Virtual Machine

2

NativeMethod Invoker

AVMNative DLL

Other DLLs

Resource Usage Limits

Each Java thread object has an associated native operating system thread. The go() function in the class JavaThread contains the main interpreter loop that executes Java bytecode instructions. Therefore, a native thread calls the go() function in order to execute the corresponding Java thread. There are advantages and disadvantages to mapping each Java thread to a separate native thread. On the one hand, without the use of native threads the VM would have to emulate the multitasking of Java threads and convert all blocking operations such as I/O, waiting to enter a monitor, or performing a wait on a condition variable to nonblocking operations. Doing so would require either a significant amount of platform-specific code to perform thread switching or a user-level thread library that might not be available on all platforms. Another advantage of using native threads is the ability for a program to easily take advantage of multiple processors. The primary disadvantage of using native threads is the increased difficulty in capturing the state of Java threads. In particular, asynchronous requests to capture thread state are harder to support because threads may be in one of many states (such as running, blocked, waiting, sleeping, or suspended) when the state capture is requested. We have designed the necessary features in Aroma to overcome this problem (see sections 2.1-2.3). Another major component of the VM is a dynamically loaded library (avmnative) that implements native methods in the Java Platform API [5]. The avmnative library is also responsible for enforcing resource limits and performing accounting for disk and network operations. Other components worth mentioning include the execution unit that implements the Java bytecode instructions [6] and the monitor handler that contains monitors for threads to synchronize in objects. The rest of this section describes Aroma-unique design features that support state capture and resource control. 2.1 Decoupling of Java Threads and Native Threads Decoupling Java threads from their corresponding native threads was one of the key design features of the Aroma VM. As a Java thread executes bytecodes that invoke and return from methods, the method stack in the corresponding thread object continues to change. However, in the normal mode of operation, the stack of the native thread (which executes C++ code) remains fixed across the execution of bytecodes. Moreover, the Java thread’s state information (such as whether it is running, or blocked trying to enter a monitor, or waiting on a condition variable) is stored within the thread object and not as part of the native thread. Therefore, the native thread that is currently executing a Java thread may be terminated without any loss of the Java thread’s state. A new native thread may be created at a later point in time (potentially on a different system) to continue the execution of the Java thread. Figure 2 gives a concrete illustration of the difference between a Java thread stack and the corresponding native thread stack. While the Java thread stack changes as the Java methods main(), doX(), doY(), and doZ() are invoked, the native thread stack remains unchanged. Therefore, if the original native thread is terminated and a new Test::doZ()

Test::main()

Test::doY()

Test::doY()

Test::doX()

Test::doX()

Test::doX()

Test::main()

Test::main()

Test::main()

Java thread stack changes as methods are invoked

JavaThread::go() VirtualMachine::run() main()

Native thread stack does not change Figure 2: Comparison Between Java Thread Stack and Native Thread Stack

3

one created at some point later, the new native thread merely calls the go() function in JavaThread and the execution of the original Java thread resumes where it left off. 2.2 Stack-Oriented Execution The second key design feature of Aroma is a stack-oriented execution mechanism to avoid changes to the native thread stack. As Figure 2 shows, during normal Java method invocation, the Java thread stack changes but the native thread stack remains constant. However, operations like loading, initializing, or instantiating classes require that native code and Java code interleave, which makes it difficult to maintain a fixed native thread stack. Consider the example of instantiating a class that has not yet been loaded by the VM. Figure 3 shows the sequence of native and Java code steps that would occur. Figure 4 shows the call stack of the native thread at the point when the Java bytecodes for the class constructor () are being executed. If the method requests a state capture (or if a second thread requests a state capture while the first thread is executing ), the VM would not be able to satisfy the request since the state of the native functions cannot be captured. Note that it is quite possible for the constructor of the class to instantiate another class (which might itself need to be loaded and initialized) requiring further nesting of the load/initialize/instantiate operations. Also, initializing a class may require super classes and interfaces to be loaded and initialized, causing further nesting of the load/initialize operations. In order to solve this problem, Aroma uses special stack frames (called VM stack frames) that are placed on the Java method stack. These VM stack frames obviate the need for recursive calls to the interpreter’s main loop inside the go() function. The design is based on the observation that several of the native functions (e.g., initializing a class or loading classes) need to carry out a sequence of steps before they are completed. In addition, intervening steps may be required such as executing Java bytecode. Therefore, we decided to convert the native functions that could potentially make nested calls to Java code into finite state machines (FSM). These FSMs are located inside VM stack frames. These VM stack frames may be located on the Java thread’s method stack intermixed with regular Java stack frames. When the main loop of the interpreter comes across a VM stack frame, the interpreter executes a special invoke() function which allows the FSM to examine its current state (saved in the VM stack frame) and take any necessary steps. These steps may either move the FSM to a completed state or alter the method stack by pushing other stack frames. In either case, the VM stack frame always returns back to the main loop without making a nested call to the new instruction load class initialize class and invoke method instantiate object invoke method

(Java) (Native) (Native) (Java) (Native) (Native) (Java)

Figure 3: Interleaving of Java and Native Code When Instantiating a Class main() Original Loop VirtualMachine::run() JavaThread::go() JavaExecutionUnit::new() ClassLoader::loadClass() JavaClass::initialize() JavaThread::executeMethod() JavaThread::go() JavaClass::instantiate() JavaThread::executeMethod() JavaThread:go()

Loop for Loop for

Figure 4: Problem with Changing Native Thread Stack When Instantiating a Class

4

interpreter. Since the main loop always processes the topmost stack frame, if a VM stack frame pushed other stack frames on top to carry out intermediate tasks (such as executing Java bytecode), the main loop will handle the new stack frames before calling invoke() on the original VM stack frame. Consider the case of a class that needs to be loaded and initialized. After native code loads the class, the native code executes the class initializer method () if one is found. Instead of recursively calling the go() function, the native code stores its current state into a special VM stack frame, pushes a new stack frame for the Java method , and finally returns to the original go() function. When the go() function finishes the new Java method, it recognizes the presence of the VM stack frame and returns control to the original native code. Figure 5 shows the state machine for the VM stack frame used to initialize a class. Note that between any of the transitions in the state machine, the VM stack frame could choose to execute Java code by pushing a stack frame for the Java method and returning back to the original go() method. Also, in the Initialize Dependent Classes state, several new VM stack frames could be created to initialize super classes and interfaces. Figure 6 shows the changes in the Java thread’s method stack as a previously unused class is loaded, initialized, and instantiated. The leftmost illustration shows the state of the stack when the class initializer method of the class being instantiated is invoked. The middle illustration shows the state of the stack when the first class initializer tries to instantiate a second notyet-loaded class. The rightmost illustration shows the state of the stack when the constructor for the second class is being invoked. 2.3 Abortable Monitors Monitors in Aroma were also carefully designed to support state capture. The important requirement was to be able to capture the state of a Java thread that might be blocked trying to enter a monitor or waiting on a condition

Begin

Load Class

Initialize Dependent Classes

Class Loaded

Initialize Class

Invoke Class Initializer

Begin

Figure 5: Finite State Machine Inside the VM Stack Frame to Initialize a Java Class

Java Stack Frame Load Class Stack Frame Init Class Stack Frame

Java Stack Frame

Inst. Class Stack Frame

Inst. Class Stack Frame

Java Stack Frame

Java Stack Frame

Java Stack Frame

Load Class Stack Frame

Load Class Stack Frame

Load Class Stack Frame

Init Class Stack Frame

Init Class Stack Frame

Init Class Stack Frame

Inst. Class Stack Frame

Inst. Class Stack Frame

Inst. Class Stack Frame

Java Stack Frame

Java Stack Frame

Java Stack Frame

Java Method Stack

Java Method Stack

Figure 6: Changes in Method Stack in Java Thread as a Java Class is Instantiated

5

Java Method Stack

variable in the monitor. For example, suppose that a Java thread tries to enter a monitor that is busy. The Java thread will have to block. If the state of the thread is captured at that moment, the state will indicate that the thread is currently blocked. If the thread’s state is restored at a subsequent point, the restored thread must resume its blocked position in the monitor. The mapping of Java threads to native threads complicates capturing and restoring blocked Java threads. In order for the Java thread to block, the corresponding native thread also has to block on some synchronization primitive (e.g., a semaphore). When a blocked Java thread’s state is restored, a new native thread must resume in the same blocked position as the original native thread. To provide this capability, monitors in Aroma allow blocked threads to be aborted. When a native thread is aborted, the monitor still maintains information about the thread (such as the thread’s position in a blocked threads queue). The aborted native thread may then be terminated without any effect on the corresponding Java thread. A (potentially new) native thread can call resume on the monitor object to resume the original native thread’s blocked position. 2.4 Thread Pausing Capturing the state of a Java thread is straightforward if the request originates from the same thread. However, capturing the state of another asynchronously running thread is more difficult. A thread must stop executing bytecodes before its state may be captured by another thread. In order to support this capability, Java threads in Aroma have been designed to allow asynchronous requests to pause the threads. When a Java thread pauses, the associated native thread blocks on a synchronization primitive so that it is safe to capture the Java thread’s state. When a paused thread is resumed, the native thread is unblocked and resumes execution of the Java thread. When a Java thread is paused, the associated native thread may also be terminated and a new native thread started later to resume a paused Java thread. The thread pausing feature is an integral part of state capture. The decoupling of Java threads and native threads and the stack-oriented execution helps guarantee that when a Java thread is paused, the execution stack of the native thread is always fixed and well known. Therefore, the native thread may be terminated and replaced with a new thread. Similarly, abortable monitors allow threads that are blocked in monitors to be paused. Figure 7 shows the functions provided by Aroma to support thread pausing. The basic pauseThread() function is used to asynchronously pause a Java thread. If the optional timeout is specified, the function will fail and return an error code if the thread could not be paused within the specified time. Once a thread has been paused, the associated native thread may be terminated by calling terminatePausedThread() function. Alternatively, a paused thread may be resumed by calling the continuePausedThread() function. The pauseAllThreads() function pauses all of the Java threads currently in the VM. Again, an optional timeout may be specified. This function is meant to be invoked by a non-Java thread and is currently used in three scenarios. The first scenario is running a garbage collection cycle (Aroma does not yet provide a concurrent garbage collector). The second scenario is an externally triggered checkpoint, where the complete state of the VM needs to be captured and saved. The third scenario is process migration, where an external event requires that the complete state of the VM be captured and moved to another host. The pauseAllOtherThreads() function is similar to pauseAllThreads() except that it is invoked by a currently running Java thread. In this case, the Java thread that invoked the operation is not paused. This function is pauseThread (JavaThread *pThread, int64 i64MaxWaitTime); terminatePausedThread (JavaThread *pThread); continuePausedThread (JavaThread *pThread); pauseAllThreads (int64 i64MaxWaitTime); pauseAllOtherThreads (JavaThread *pThread, int64 i64MaxWaitTime); terminateAllPausedThreads (void); continueAllPausedThreads (void); Figure 7: Virtual Machine Functions Relating to Pausing Threads

6

used to implement operations where a Java thread directly (e.g. a request to checkpoint) or indirectly (e.g., a request to move from one host to another) requests a state capture operation. The terminateAllPausedThreads() is similar to terminatePausedThread() except that it terminates the native threads associated with all of the currently paused Java threads. Finally, the continueAllPausedThreads() function resumes execution of all Java threads that are currently paused. 2.5 Resource Control Aroma currently provides a comprehensive set of resource controls for CPU, disk, and network. The resource control mechanisms allow limits to be placed on both the rate and quantity of resources used by Java threads. Rate limits include CPU usage, disk read rate, disk write rate, network read rate and network write rate. Rate limits for I/O are specified in bytes/millisecond. Quantity limits include disk space, total bytes written to disk, total bytes read from the disk, total bytes written to the network, and total bytes read from the network. Quantity limits are specified in bytes. CPU resource control was designed to support two alternative means of expressing the resource limits. The first alternative is to express the limit in terms of bytecodes executed per millisecond. The advantage of expressing a limit in terms of bytecodes per unit time is that given the processing requirements of a thread, the thread’s execution time (or time to complete a task) may be predicted. Another advantage of expressing limits in terms of bytecodes per unit time is that the limit is system and architecture independent. The second alternative is to express the limit in terms of some percentage of CPU time, expressed as a number between 0 and 100. Expressing limits as a percentage of overall CPU time on a host provides better control over resource consumption on that particular host. Rate limits for disk and network are expressed in terms of bytes read or written per millisecond. If a rate limit is in effect, then I/O operations are transparently delayed if necessary until such time that allowing the operation would not exceed the limit. Threads performing I/O operations will not be aware of any resource limits in place unless they choose to query the VM. Quantity limits for disk and network are expressed in terms of bytes. If a quantity limit is in effect, then the VM throws an exception when a thread requests an I/O operation that would result in the limit being exceeded. 2.6 State Capture Limitations Java threads may invoke native code by invoking native methods. In addition, it is possible for native methods to use the Java Native Interface (JNI) to invoke Java methods. In both cases, the Java thread’s method stack will contain a stack frame for the native method. If a Java thread is asked to pause in either of these two situations, the thread will pause when the native method completes. Therefore, a request to capture the state of the Java thread will be deferred until the native method completes and returns control back to Java code and there are no further native stack frames on the method stack. 3.

Implementation Description

The Aroma VM implementation is based on the Java Virtual Machine Specification [6] and does not use any source code from other licensed VM implementations. Therefore, Aroma may be distributed without any licensing constraints. Currently, Aroma is distributed in binary form as bundled with the NOMADS mobile agent system. NOMADS may be downloaded and used free of charge for non-commercial purposes from http://www.coginst.uwf.edu/nomads. We also plan to release the Aroma VM in the form of an object library that may be embedded inside other applications. The current implementation of Aroma uses about 40,000 lines of C++ code. The VM has been ported to Win32 (on x86), Solaris (on SPARC), and Linux (on x86) platforms. A port to Mac OSX is currently underway. Ports to other platforms should be possible as long as either POSIX-style or Win32-style threads and POSIX-style I/O libraries are supported. Aroma is currently JDK 1.2.2 “compatible” with some missing features such as support for AWT and Swing. Also, Aroma currently works with the Sun implementation of the Java Platform API (as distributed in the Java Runtime Environment). In the future, we also plan to support other API implementations such as the GNU Classpath project [7].

7

The rest of this section describes additional aspects of the implementation and elaborates on the design features described previously. 3.1 Execution Cycle Structure Figure 8 shows the structure of the VM interpreter’s main execution cycle (contained in the go() function inside the JavaThread class). The cycle is structured as a double loop with the inner loop executing bytecodes until there is a reason to stop (detected by testing a boolean at the beginning of the loop). Before each bytecode is executed, the VM invokes a function to perform CPU resource control and accounting (described in section 3.3). If there is a reason to stop executing bytecodes, the inner loop terminates. Reasons include blocking to enter a monitor, waiting on a condition variable, sleeping, terminating, or satisfying a request to suspend or pause. The primary reason for the structure of the execution cycle is to make sure that whenever a native thread has to stop execution, it does so in the go() function in the outer loop. When the Java thread’s state needs to be captured, the VM can simply terminate the native thread. When a new native thread starts to execute the outer loop, the thread will check all of the conditions and resume the state of the previous native thread. 3.2 Disk and Network Resource Control The native code library (avmnative) is responsible for implementing the enforcement of the disk and network resource limits. Resource limits may be grouped into two categories: rate limits and quantity limits. To enforce the quantity limits, the native code library maintains four counters for the number of bytes read and written to the network and the disk. For every read or write operation, the library checks whether performing the operation would allow the agent to exceed a limit. If so, the library returns an exception to the agent. Otherwise, the appropriate counter is incremented and the operation is allowed to proceed. To enforce the disk space limit, the library performs a similar computation except that seek operations and file deletions are taken into consideration. Again, if an operation would allow the agent to exceed the disk space limit, the library returns an exception and does not complete the operation. To enforce the rate limits, the library maintains four additional counters for the number of bytes read and written to the network and the disk and four time variables, which record the time when the first operation was performed. Before an operation is allowed, the library divides the number of bytes by the elapsed time to check if the agent is above the rate limit. If so, the library puts the thread to sleep until such time that the agent is within the rate limit. Then, the library computes how many bytes may be read or written by the agent in a 100ms interval. If the operation requested by the agent is less than what is allowed in a 100ms interval, the library simply completes the operation and returns (after updating the counter). Otherwise, the library divides the operation into sub-operations and performs them in each interval. After an operation is performed, the library sleeps until the interval finishes. go() { while (1) { check to enter monitor check to wait in monitor check to pause check to sleep check to suspend check to terminate while (no reason to stop) { check and enforce CPU resource limits execute bytecode instruction } } } Figure 8: Structure of the VM’s Main Interpreter Loop

8

3.3 CPU Resource Control Figure 9 shows the basic algorithm used to enforce the CPU resource control. This algorithm is executed in each iteration of the inner execution loop. The algorithm first computes the number of bytecodes that should be allowed in a 100-millisecond interval. Then, each time the number of bytecodes allowed per interval has been executed, the algorithm checks to see how much of the 100 milliseconds have elapsed. If there is any remaining time, the algorithm puts the thread to sleep so that no more bytecodes are executed until the start of the next interval. Note that for efficiency the algorithm uses intervals of 100 milliseconds in enforcing the CPU rate limit. This implies that within 100 milliseconds, the algorithm does not care about the distribution of the load on the CPU. However, the algorithm guarantees that the average number of bytecodes per 100 milliseconds will be limited correctly. This interval may be decreased if a more even distribution of the CPU load is desired although decreasing the interval will result in an increase in overhead. 4.

Performance

In initial performance tests, we compared Aroma against Sun’s Java 1.3 and IBM’s Java 1.3. The benchmarks chosen were: Linpack [8], Tak [9], and UCSD [10]. These benchmarks were executed on a 600 MHz Intel Pentium III with 256 MB of RAM running Windows 2000. The results are shown in Table 1. All times are reported in seconds. The results show that the performance of the Aroma VM is significantly slower than all the other VMs. Aroma is between 4.8 to 11.3 times slower when compared to the Sun and IBM VM implementations without a JIT. The performance difference is to be expected given that we have not yet optimized the VM code. We also measured the performance of the state capture and restore operations. These measurements were performed in the context of an agent move operation in the NOMADS mobile agent system. The move operation involves capturing, transferring, and restoring the execution state. The results in Table 2 show that the performance of state capture and restore is fairly fast. Capturing the state of an individual thread requires an average of 38 ms whereas capturing the complete state of the VM requires an average of 332 ms. The size of the state information for individual threads is about 4 KB and the size of the state information for the full VM is about 1.5 MB. Note that the // State Variables intervalStartTime bytecodeCount bytecodesPerInterval

// Start time of the current 100 MS interval // Number of bytecodes executed in the interval // Number of bytecodes allowed in one interval

bytecodeCount++ if (bytecodeCount == bytecodesPerInterval) currentTime = elapsedTime = currentTime - intervalStartTime if (elapsedTime < 100) // 100 milliseconds timeToSleep = 100 – elapsedTime sleep (timeToSleep) intervalStartTime = currentTime bytecodeCount = 0 Figure 9: Algorithm to Check and Enforce CPU Rate Limit Aroma 0.11 Linpack TAK UCSD

Sun JDK 1.3 Sun JDK No JIT W ith JIT 0.68 0.14 165.34 19.36 43.55 4.20

1.3

IBM JDK 1.3 IBM JDK No JIT W ith JIT 0.02 0.14 2.29 25.01 2.00 3.84

1.3 0.07 1.86 2.27

Table 1: Performance Comparison between Aroma, Sun, and IBM Virtual Machines (Times in Seconds).

9

size of the full VM state is large because it includes all of the standard Java system and I/O classes that are loaded during VM initialization time. Finally, we measured the performance of the resource control mechanisms. Table 3 shows the performance results for I/O resource control and Table 4 shows the performance results for CPU resource control. On a 450 MHz Intel Pentium II system with 256 MB of RAM running Windows 2000, the results show that the overhead introduced by the disk I/O resource control is minimal. The results for the network resource control are similar. The CPU resource control benchmarks show an overhead of 6.6% to 6.9%. 5.

Applications

5.1 NOMADS NOMADS is a mobile agent system that provides strong mobility and fine-grained resource control. Unlike applets which generally only travel a single hop on demand from server to client, mobile agents can travel over networks under their own power with their own itinerary. Strong mobility requires that the execution state of an agent (i.e., the thread state) be captured and moved from one host to another across a network so that the agent can in principle take up execution after a move right where it left off. NOMADS relies on Aroma’s state capture capabilities to provide strong mobility. Since this capability is not available in the standard Java VMs, most mobile agent systems have resorted to providing only weak mobility. The few that provide strong mobility do so by either modifying an existing Java VM (thereby limiting the ability to redistribute the system) or using a preprocessor to add state-saving code (which does not work well with multiple threads). NOMADS also supports forced mobility, which allows the system to move agents between hosts potentially even without the agents being aware of the relocation. Such forced mobility of agents is very useful for load-balancing and evacuation of agents from hosts pending shutdown. Forced mobility relies on Aroma’s support for asynchronous requests for state capture. Finally, NOMADS provides secure execution environments to handle untrusted and potentially buggy or malicious agents. Executing mobile code securely depends not only on access control but resource control as well. NOMADS relies on the resource control capabilities of Aroma to limit the disk, network, and CPU resources that may be consumed by mobile agents. Resource controls are particularly essential for mobile code and mobile agents so that systems (and networks) are not open to denial of service attacks. Time 38 sec 332 sec

Individual Thread State Full VM State

Size 4 KB 1.5 MB

Table 2: State Capture Performance of Aroma

Sun JDK 1.3 (With JIT) A roma VM (with no resource control code) A roma VM (with code but no limit) A roma VM (with code but very high limit)

10 KB 9532 ms 8772 ms 8746 ms 8702 ms

I/O Block Size 64 KB 9903 ms 9650 ms 9656 ms 9655 ms

Table 3: Disk I/O Resource Control Performance of Aroma

Linpack Tak UCSD

W ithout Resource Control Code 0.86 4.43 59.86

W ith Resource Control Code (But No Limit) 0.92 4.42 63.83

Table 4: CPU Resource Control Performance of Aroma (Times in Seconds)

10

5.2 WYA WYA (While You’re Away) is a distributed system that takes advantage of the processing power of idle workstations. WYA supports the notion of roaming computations – running programs that are moved around on a network to run on idle systems. The system consists of a centralized coordinator and a set of user workstations. Users may submit jobs to the coordinator, which are then sent to idle workstations. WYA provides several unique capabilities: Architecture Independence: Using a VM allows roaming computations to be moved across systems of different architectures. Therefore, it is no longer necessary to have identical workstations for load-balancing. Workstation Availability: When an end-user wishes to use his or her personal workstation, any roaming computations on that workstation are immediately moved back to the server. This feature relies on Aroma’s asynchronous state capture capabilities. Workstation Security: Using the resource control mechanisms of Aroma, WYA guarantees that the resources of user workstations are not abused. User Incentive: WYA uses Aroma’s resource accounting mechanisms to keep track of the resources used by roaming computations. As a user’s workstation gets used, the user accumulates points that could conceivably be used for a variety of purposes. The capabilities that WYA demonstrates are ideally suited for any kind of peer-to-peer computing application [1]. 6.

Related Work

Much work has been done in the area of resource control and state capture – though no other effort of which we are aware offers them both as part of the same system. Therefore, we first describe work related to state capture followed by work related to resource control. 6.1 Related Work in State Capture An early attempt at state capture within Sun’s Java VM was the persistent Java project at Sun Labs [11,12]. We have been told that this work has been discontinued. Sumatra [13] and Ara [14,15] are mobile agent systems that use a modified version of the JDK 1.0.2 VM to provide state capture. However, Sumatra did not support asynchronous requests or capturing the state of multiple threads. Also, JDK 1.0.2 does not use native threads, which simplifies capturing state. Another prototype system that Plank [16] developed uses state capture to provide checkpointing of Java processes. Plank’s system is based on a modified version of the Kaffe [17] Virtual Machine. Merpati [18] is a more recent system that provides checkpointing. Merpati is based on a modified JDK 1.1.7 VM. Unlike Sumatra and Ara, Merpati can checkpoint multiple threads but cannot handle threads that are blocked in monitors. Finally, we know of three approaches that use a preprocessor to capture execution state. In all three cases, a preprocessor examines code to find locations where state capture may be requested and wraps those locations with extra code to save and restore state. WASP [19] uses a preprocessor to instrument Java source code whereas JavaGO [20] and Correlate [21] use a preprocessor to instrument Java bytecode. These systems work within a standard Java VM but have difficulty with multiple threads and with asynchronous requests to capture state. 6.2 Related Work in Resource Control JRes [22] was the first system to provide resource controls for Java. JRes uses a preprocessor to instrument code allowing it to keep track of memory allocation and the creation of Java threads. A JRes system thread uses this information to query the operating system about the resource consumption of each Java thread. JRes also supported quantity limits on network I/O. Unlike JRes, code running inside Aroma does not require a preprocessor. Also, JRes relies on adjusting a thread’s priority in order to enforce a limit. This is not as accurate as the mechanism

11

implemented in the Aroma VM. Finally, Aroma offers a more comprehensive set of resource limits including limits on rates of resource usage. SOMA [23] is another mobile agent system that provides resource accounting and some limited control. SOMA relies on the Java VM Profiler Interface [24] and native libraries to perform resource accounting. The native libraries query the underlying operating system to determine resources consumed by the Java VM. Controlling resource consumption in SOMA is limited to lowering the priority of Java threads, suspending Java threads, or terminating Java threads. Lal and Pandey’s system [25] uses a modified Java VM to provide CPU resource control and scheduling. Their emphasis is on scheduling mobile programs taking into account requirements expressed by programs as well as constraints expressed by hosts. One important difference is that Aroma allows the limits (and eventually requirements) to be expressed in terms of bytecodes per unit time, which is platform independent. The Real-Time Java [26] specification also addresses CPU resource control. In particular, the specification calls for installable thread schedulers that can implement custom scheduling policies which could take CPU resource control into account. 7.

Summary

The Aroma VM is a Java compatible VM developed for research use. Aroma provides unique features such as execution state capture and CPU, disk, and network resource controls. Aroma has been ported to Win 32 on x86, Solaris on SPARC, and Linux on x86. The NOMADS mobile agent system and the WYA (While You’re Away) distributed system currently use the unique capabilities of Aroma. In fact, we believe that the widespread proliferation of such applications is contingent on the kind of transparency, security, and resource control that Aroma provides. The performance of Aroma is significantly slower than Virtual Machines from Sun or IBM. However, Aroma offers good state capture performance with minimal overhead for disk and network resource controls. CPU resource control introduces an overhead of 6.6% to 6.9%. Aroma is currently distributed in binary form as part of the NOMADS and may be downloaded and used free of charge for non-commercial use from http://www.coginst.uwf.edu/nomads. References 1. 2.

3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13.

Peer-to-Peer Working Group Welcome Presentation. http://www.peer-to-peerwg.org/meetings/00-10-12/ collateral/Welcome.ppt. Suri, N., Bradshaw, J.M., Breedy, M.R., Groth, P.T., Hill, G.A., and Jeffers, R. Strong Mobility and FineGrained Resource Control in NOMADS. Proceedings of the Second International Symposium on Agent Systems and Applications and the Fourth International Symposium on Mobile Agents (ASA/MA'2000). Zurich, Switzerland. Springer-Verlag. Bradshaw, J.M. Ed. Software Agents. AAAI Press/MIT Press. Gong, L. Inside Java 2 Platform Security: Architecture, API Design, and Implementation. Addison-Wesley. Gosling, J. and Yellin, F. The Java Application Programmming Interface, Vols. 1-2. Addison-Wesley. Lindholm, T. and Yellin, F. The Java Virtual Machine Specification. 2nd ed. Addison-Wesley. GNU Classpath Project. http://www.classpath.org. Hardwick, J. Optimized Java Linpack Benchmark. http://www.cs.cmu.edu/~jch/java/linpack.html. Harold, E. The Tak Benchmark Applet. http://www.ibiblio.org/javagaq/newsletter/TakApplet.html. Griswold, B. and Phillips, P. UCSD Benchmarks for Java. http://www-cse.uscd.edu/users/wgg/ JavaProf/javaprof.html. Atkinson, M.P. and Morrison, R. Orthogonally Persistent Object Systems. In VLDB J., Vol. 4, No. 3, 1995, pp. 319-401. Jordan, M. and Atkinson, M.P. Orthogonal Persistence for Java – A Mid-Term Report. Sun Microsystems Laboratories, 1998. Acharya, A., Ragnganathan, M., & Saltz, J. Sumatra: A language for resource-aware mobile programs. In J. Vitek & C. Tschudin (Ed.), Mobile Object Systems. Springer-Verlag.

12

14. Peine, H., & Stolpmann, T. The architecture of the Ara platform fro mobile agents. In K. Rothernel & R. Popescu-Zeletin (Ed.), Proceedings of the First International Workshop on Mobile Agents (MA 97). SpringerVerlag. 15. Maurer, J. Porting the Java runtime system to the Ara platform for mobile agents. Diploma Thesis, University of Kaiserslautern. 16. Plank, J.S. An Overview of Checkpointing in Uniprocessor and Distributed Systems Focusing on Implementation and Performance. Technical Report UT-CS-97-372, Dept. of Computer Science, Univ. of Tennessee, Knoxville, Tenn., 1997. 17. Kaffe Project. http://www.kaffe.org. 18. Suezawa, T. Persistent Execution State of a Java Virtual Machine. Proceedings of the 2000 ACM Java GrandeConference. San Francisco, CA. ACM Press. 19. Fünfrocken, S. Transparent migration of Java-based mobile agents: Capturing and reestablishing the state of Java programs. In K. Rothermel & F. Hohl (Ed.), Mobile Agents: Proceedings of the Second International Workshop (MA 98). Springer-Verlag. 20. Sakamoto, T., Sekiguchi, T., and Yonezawa, A. Bytecode Transformation for Portable Thread Migration in Java. Proceedings of the Second International Symposium on Agent Systems and Applications and the Fourth International Symposium on Mobile Agents (ASA/MA'2000). Zurich, Switzerland. Springer-Verlag. 21. Truyen, E., Robben, B., Vanhaute, B., Coninx, T., Joosen, W., and Verbaeten, P. Portable Support for Transparent Thread Migration In Java. Proceedings of the Second International Symposium on Agent Systems and Applications and the Fourth International Symposium on Mobile Agents (ASA/MA'2000). Zurich, Switzerland. Springer-Verlag. 22. Czajkowki, G., & von Eicken, T. JRes: A resource accounting interface for Java. Proceedings of the 1998 ACM OOPSLA Conference. Vancouver, B.C., Canada. 23. Bellavista, P., Corradi, A. and Stefanelli, C. Monitor and Control of Mobile Agent Applications. In OOPSLA 2000 Workshop on Experiences with Autonomous Mobile Objects and Agent Based Systems. http://www.cs.umn.edu/Ajanta/oopsla-workshop/stefanelli.pdf 24. Sun Microsystems. Java Virtual Machine Profiler Interface (JVMPI). http://java.sun.com/products/jdk/1.3/ docs/guide/jvmpi/jvmpi.html. 25. Lal, M. & Pandey, R. CPU Resource Control for Mobile Programs. Proceedings of the First International Symposium on Agent Systems and Applications and the Third International Symposium on Mobile Agents (ASA/MA’99). Palm Springs, CA. IEEE Computer Society Press. 26. Sun Microsystems. Real-Time Specification for Java. http://java.sun.com/aboutJava/communityprocess/jsr/ jsr_001_real_time.html.

13