PVM Emulation in the Harness Metacomputing System: a Plug-in Based Approach 1
1
Mauro Migliardi , Vaidy Sunderam 1
Emory University , Dept. Of Math & Computer Science Atlanta, GA, 30322, USA
[email protected]
Abstract. Metacomputing frameworks have received renewed attention of late, fueled both by advances in hardware and networking, and by novel concepts such as computational grids. Harness is an experimental metacomputing system based upon the principle of dynamic reconfigurability not only in terms of the computers and networks that comprise the virtual machine, but also in the capabilities of the VM itself. These characteristics may be modified under user control via a "plug-in" mechanism that is the central feature of the system. In this paper we describe our preliminary experience in the design of PVM emulation by means of a set of plug-in.
1 Introduction Harness [1] is a metacomputing framework that is based upon several experimental concepts, including dynamic reconfigurability and fluid, extensible, virtual machines. Harness is a joint project between Emory University, Oak Ridge National Lab, and the University of Tennessee, and is a follow on to PVM [2], a popular network-based distributed computing platform of the 1990’s. The underlying motivation behind Harness is to develop a metacomputing platform for the next generation, incorporating the inherent capability to integrate new technologies as they evolve. The first motivation is an outcome of the perceived need in metacomputing systems to provide more functionality, flexibility, and performance, while the second is based upon a desire to allow the framework to respond rapidly to advances in hardware, networks, system software, and applications. Both motivations are, in some part, derived from our experiences with the PVM system, whose monolithic design implies that substantial re-engineering is required to extend its capabilities or to adapt it to new network or machine architectures. Harness attempts to overcome the limited flexibility of traditional software systems by defining a simple but powerful architectural model based on the concept of a software backplane. The Harness model is one that consists primarily of a kernel that is configured, according to user or application requirements, by attaching “plug-in” modules that provide various services. Some plug-ins are provided as part of the Harness system, while others might be developed by individual users for special
situations, while yet other plug-ins might be obtained from third-party repositories. By configuring a Harness virtual machine using a suite of plug-ins appropriate to the particular hardware platform being used, the application being executed, and resource and time constraints, users are able to obtain functionality and performance that is well suited to their specific circumstances. Furthermore, since the Harness architecture is modular, plug-ins may be developed incrementally for emerging technologies such as faster networks or switches, new data compression algorithms or visualization methods, or resource allocation schemes – and these may be incorporated into the Harness system without requiring a major re-engineering effort. The Harness project is in its initial stages, and we are in the process of defining the backplane architecture and developing the core framework. Simultaneously, we are also experimenting with a prototype kernel implementation as well as with plug-ins to realize different types of services. One major effort involves the creation of plug-ins that will emulate the PVM concurrent computing system within Harness. From the point of view of Harness research, this exercise will determine the viability of constructing new metacomputing facilities or even complete environments through the use of plug-ins. From a practical perspective, a suite of PVM plug-ins for Harness will provide a convenient and effective transition path for existing users and applications. In this paper, we describe the preliminary design of PVM emulation in Harness. We begin with an overview of the Harness model, describe an initial implementation and experiences with its use, and outline the proposed design for developing PVM plug-ins. A discussion of the status of Harness and the project outlook concludes the paper.
2 Architectural Overview of Harness The fundamental abstraction in the Harness metacomputing framework is the Distributed Virtual Machine (DVM) (see figure 1, level 1). Any DVM is associated with a symbolic name that is unique in the Harness name space, but has no physical entities connected to it. Heterogeneous Computational Resources may enroll into a DVM (see figure 1, level 2) at any time, however at this level the DVM is not ready yet to accept requests from users. To get ready to interact with users and applications the heterogeneous computational resources enrolled in a DVM need to load plug-ins (see figure 1, level 3). A plug-in is a software component implementing a specific service. By loading plug-ins a DVM can build a consistent service baseline (see figure 1, level 4). Users may reconfigure the DVM at any time (see figure 1, level 4) both in terms of computational resources enrolled by having them join or leave the DVM and in terms of services available by loading and unloading plug-ins. The main goal of the Harness metacomputing framework is to achieve the capability to enroll heterogeneous computational resources into a DVM and make them capable of delivering a consistent service baseline to users. This goal require the programs building up the framework to be as portable as possible over an as large as possible selection of systems. The availability of services to heterogeneous computational resources derives from two different properties of the framework: the portability of
Figure 1 Abstract model of a Harness Distributed Virtual Machine
plug-ins and the presence of multiple searchable plug-in repositories. Harness implements these properties mainly leveraging two different features of Java technology. These features are the capability to layer a homogeneous architecture such as the Java Virtual Machine (JVM) [3] over a large set of heterogeneous computational resources, and the capability to customize the mechanism adopted to load and link new objects and libraries. The adoption of the Java language has also given us the capability to tune the tradeoff between portability and efficiency for the different components of the framework. This capability is extremely important, in fact, although portability at large is needed in all the components of the framework, it is possible to distinguish three different categories of components that requires different level of portability. The first category is represented by the components implementing the capability to manage the DVM status and load and unload services. We call these components kernel level services. These services require the highest achievable degree of portability, as a matter of fact they are necessary to enroll a computational resource into a DVM. The second category is represented by very commonly used services (e.g. a general, network independent, message-passing service or a generic event notification mechanism). We call these services basic services. Basic services should be generally available, but it is conceivable for some computational resources based on specialized architecture to lack them. The last category is represented by highly architecture specific services. These services include all those services that are inherently dependent on the specific characteristics of a computational resource (e.g. a low-level image processing service exploiting a SIMD co-processor, a message-passing service exploiting a specific network interface or any service that need architecture dependent optimization). We call these services specialized services. For this last category portability is a goal to strive for, but it is acceptable that they will be available only on small subsets of the available computational resources. These different requirements for portability and
efficiency can optimally leverage the capability to link together Java byte code and system dependent native code enabled by the Java Native Interface (JNI) [4]. The JNI allows to develop the parts of the framework that are most critical to efficient application execution in ANSI C language and to introduce into them the desired level of architecture dependent optimization at the cost of increased development effort. The use of native code requires a different implementation of a service for each type of heterogeneous computational resource enrolled in the DVM. This fact implies a larger development effort. However, if a version of the plug-in for a specific architecture is available, the Harness metacomputing framework is able to fetch and load it in a user transparent fashion, thus users are screened from the necessity to control the set of architectures their application is currently running on. To achieve this result Harness leverages the capability of the JVM to let users redefine the mechanism used to retrieve and load both Java classes bytecode and native shared libraries. In fact, each DVM in the framework is able to search a set of plug-ins repositories for the desired library. This set of repositories is dynamically reconfigurable at run-time, users can add or delete repositories at any time.
3 The Design of a PVM plug-in for a Harness DVM Our initial approach to the problem of providing PVM compatibility in the Harness system has been guided by three objectives: • requiring no changes into PVM applications to run in the new environment; • minimizing the amount of changes to be inserted in the application side PVM library; • achieving a modular design for the services provided by the PVM daemon. We plan to achieve these goals by designing a set of plug-ins able to understand the original PVM library to PVM daemon protocol and to duplicate the services provided by the original PVM daemon. This approach allows us to provide complete compatibility with PVM legacy code while requiring only two changes in the PVM library on the application side: • the adoption of internet domain sockets for the communication channel between the library and the daemon; • the insertion of a Harness startup function. Thus to run a legacy PVM application in the Harness PVM environment it is only necessary to link the original object code with the modified version of the library. The services provided by the PVM daemons have been divided into five groups, namely process control, information, signaling, user-level message-passing and group operations. The services in each of these group are provided by a dedicated plug-in, namely the spawner plug-in, the database plug-in, the signal plug-in, the messagepassing plug-in and the group plug-in. These plug-ins are loaded in the Harness kernels by the main PVMD plug-in when the Harness PVM environment is started or when an add-host command enrolls a new host. The main PVMD plug-in is also responsible of the loading of the plug-in that provides the message-passing service
Figure 2 Sequence of events at Harness PVM startup
between the daemons. In figure 2 we show the actual sequence of the events in the Harness PVM startup, while figure 3 shows the chain of events serving an add_host request. At PVM startup the PVM application (usually the console) starts up the PVM demon by issuing to the Harness kernel the command to load the main PVMD plugin. This plug-in takes care to request the Harness kernel to load the services that are required to provide full PVM compatibility. When a task requests an add-host operation the local PVMD plug-in translates it into a request for the remote Harness kernel to load the main PVMD plug-in which then takes care of requesting the loading of the other needed plug-ins. The first release of these plug-ins will provide only the services required to emulate PVM daemon’s capabilities. However, the modularity of the design will easily let us substitute any plug-in with new versions in order to provide an enhanced
Figure 3 Sequence of events performed to service an add_host request.
version of the service. As an example, it will be easy to load a new version of the group plug-in in order to provide an extended set of group operations or a new database plug-in to provide an extended system querying capability. Besides, the Harness capability to hot swap services will allow run-time tuning of services to the set of hosts enrolled in the virtual machines, e.g. a specific version of messagepassing plug-in could be loaded at run-time if a new communication fabric becomes available. Besides fulfilling our primary goals this design has other advantages. The first one derives from the fact that the message-passing service provided by the messagepassing plug-in needs only to peek at the destination field of a message in order to route it and does not need to know anything about the actual content of the message. This is beneficial for two reasons: • the marshalling and un-marshalling of the data types is performed inside the Harness PVM library in C or Fortran thus we don’t incur in the typical marshalling inefficiency due to the strong typedness of Java; • it is extremely easy to substitute the message passing plug-in with another plug-in optimized to a specific communication fabric (be it a proprietary local network or an unreliable Internet connection) because they only need to move arrays of bytes. Another benefit of our design is the fact that PVM applications can rely on the Harness capability to soft-install applications to move executable and libraries to the hosts in the VM. Thus it is not necessary to install the application and PVM itself on all the hosts of the VM, the Harness loader will do it as long as they are available on any host in the Harness DVM or on any one of the enlisted repositories. A third, very important benefit of our design is the removal of the single point of failure represented by the master PVM daemon. In fact, providing PVM compatibility on top of the Harness system by means of a set of cooperating plug-ins, allowed us to exploit the Harness event subscription/notification service to implement a distributed control algorithm in the information management plug-ins. This algorithm is capable of reconstructing a consistent, up-to-date version of the PVM status after the crash of any daemon.
4 Related Works Metacomputing frameworks have been popular for nearly a decade, when the advent of high-end workstations and ubiquitous networking enabled high performance concurrent computing in networked environments. PVM was one of the earliest systems to formulate the metacomputing concept in concrete virtual machine and programming-environment terms. PVM however, is inflexible in many respects that can be constraining to the next generation of metacomputing and collaborative applications. The Harness “plug-in” paradigm effectively alleviates these drawbacks while providing greatly expanded scope and substantial protection against both rigidity and obsolescence. Jpvm [5] is a pure Java implementation of the PVM system. However it is not
compatible with any traditional PVM application. Legion [6] is a metacomputing system that began as an extension of the Mentat project. Legion can accommodate a heterogeneous mix of geographically distributed high-performance machines and workstations. Legion is an object-oriented system where the focus is on providing transparent access to an enterprise-wide distributed computing framework. As such, it does not attempt to cater to changing needs and it is static in the types of computing models it supports as well as in implementation. Globus [7] is a metacomputing infrastructure which is built upon the “Nexus” [8] multi-language communication framework. The Globus system is designed around the concept of a toolkit that consists of the pre-defined modules pertaining to communication, resource allocation, data, etc. However the assembly of these modules is not supposed to happen dynamically at run-time as in Harness. Besides, the modularity of Globus is at the metacomputing system level in the sense that modules affect the global composition of the metacomputing substrate. Sun Microsystems Jini project [9] presents a model where a federation of Java enabled objects connected through a network can freely interact and deliver services to each other and to end users. In principle Jini shares with Harness many keywords and goals, such as services as building blocks and hetereogeneity of service providers. However, Jini focuses on the capability to build up a world of plug-and-play consumer devices and, to cope with such a goal, increase the resolution of the computational resources that can be enrolled in a Jini federation. In fact in the Jini model these resources range from complete computational systems down to devices such as disks, printers, TVs and VCRs. The above projects envision a model in which very high performance bricks are statically connected to build a larger system. One of the main idea of the Harness project is to trade some efficiency to gain enhanced global availability, upgradability and resilience to failures by dynamically connecting, disconnecting and reconfiguring heterogeneous components. Harness is also seen as a research tool for exploring pluggability and dynamic adaptability within DVMs as the design of the PVMD plugin shows.
5 Conclusions In this paper we have described our plug-in based design for providing PVM emulation in the Harness metacomputing system together with its rationale and advantages. Our design allows achieving object level compatibility with PVM legacy application while giving PVM user access to the new features provided by the Harness system. Besides it will provide to PVM users a smooth transition path to a programming environment that allows combining different programming paradigms. Harness it’s still in its initial stage, however our prototype is capable of: • defining services in term of abstract Java interfaces in order to have them updated in a user transparent manner; • adapting to changing user needs by adding new services to heterogeneous computational resources via the plug-in mechanism;
The design of our Harness PVM emulation shows that it is feasible to define distributed computing environment in terms of plug-ins. This approach allows the incorporation of new features and technological enhancements into existing systems such as PVM without loosing compatibility with legacy applications.
References 1
2
3 4 5 6
7 8 9
M. Migliardi, V. Sunderam, A. Geist, J. Dongarra, Dynamic Reconfiguration and Virtual Machine Management in the Harness Metacomputing System, Proc. of ISCOPE98, pp. 127-134, Santa Fe', New Mexico (USA), December 8-11, 1998. A. Geist, A. Beguelin, J. Dongarra, W. Jiang, B. Mancheck and V. Sunderam, PVM: Parallel Virtual Machine a User’s Guide and Tutorial for Networked Parallel Computing, MIT Press, Cambridge, MA, 1994. T. Lindholm and F. Yellin, The Java Virtual Machine Specification, Addison Wesley, 1997. S. Liang, The Java Native Interface: Programming Guide and Reference, Addison Wesley, 1998. A. J. Ferrari, Jpvm, Technical report, http://www.cs.virginia.edu/ajf2j/jpvm A. Grimshaw, W. Wulf, J. French, A. Weaver and P. Reynolds. Legion: the next logical step toward a nationwide virtual computer, Technical Report CS-94-21, University of Virginia, 1994. I. Foster and C. Kesselman, Globus: a Metacomputing Infrastructure Toolkit, International Journal of Supercomputing Application, May 1997. I. Foster, C. Kesselman and S. Tuecke, The Nexus Approach to Integrating Multithreading and Communication, Journal of Parallel and Distributed Computing, 37:70-82, 1996 Sun Microsystems, Jini Architecture Overview, available on line at http://java.sun.com/products/jini/whitepapers/architectureoverview.pdf, 1998.
