Computer Architecture and Operating Systems Group (CAOS) ... support to the transparent management/use of remote heterogeneous distributed (and parallel) ...
Web Remote Services Oriented Architecture for Cluster Management* Josep Jorba, Rafael Bustos, Ángel Casquero, Tomàs Margalef, and Emilio Luque Computer Architecture and Operating Systems Group (CAOS) Escola Tecnica Superior de Enginyeria Universidad Autonoma de Bellaterra (UAB) 08193 Bellaterra, Spain {josep.jorba,tomas.margalef,emilio.luque}@uab.es
Abstract. PCs and Workstations clusters are becoming more popular everyday. In many cases these systems are considered as part of a wider system for meta or GRID computing purposes. The management, administration and use of this systems present several difficulties due to the need to access different systems located at remote sites. A new architecture for managing a MultiUser MultiCluster environment has been developed. A cluster management environment prototype called ION Web (Interoperable Object Network Web) has been developed. This prototype is based on well-known standards (Web, CORBA, Java and XML). It works with PCs or Workstation Clusters but the architecture is not final environment dependent, and can be migrated or adapted to any cluster, like for example, PVM or MPI Clusters, or Grid systems using their native services.
1
Introduction
The use of parallel/distributed systems must be considered from different points of view depending on the kind of user involved. System administrators must take care of the system configuration, access control, application installation and so on. Application developers carry out the task of designing and building parallel/distributed applications, using a reduced set of practical APIs, like PVM[1], MPI[2], BSP, etcetera. Finally, end user runs applications, providing the input data and getting the output results. In the classical approach each one of these kinds of users have a complete different view of the system with a different user interface. Moreover, there are no uniform interfaces and each application must redefine its own interface. This fact causes the rebuilding of application runtime environments for data management, execution control and results publishing, for each system involved. *
This work has been supported by the MCyT (Spain) under contract TIC2001-2592 and partially supported by Generalitat de Catalunya – G. de Recerca consolidat 2001SGR-00218.
D. Kranzlmüller et al. (Eds.): Euro PVM/MPI 2002, LNCS 2474, pp. 368-375, 2002. Springer-Verlag Berlin Heidelberg 2002
Web Remote Services Oriented Architecture for Cluster Management
369
Our proposal is to define an architectural framework, n-thier based, that provides support to the transparent management/use of remote heterogeneous distributed (and parallel) systems. This proposal is based on providing an API that offers a set of basic remote services for the cluster system that can be used from any type of device (PCs, cellular phones, PDAs) or software clients. The proposed architecture needs to provide different points of view of the Cluster/Parallel system, concerning the offered model to the architecture clients: •
End user needs a functional model of the system utilisation, which includes an abstract view of the services provided by the target system to the user. Administrator requires services for system configuration from a structural and functional point of view, and services to control the interactions of the users with the system.
•
In the literature several approaches have been described to design computing portals (WebFlow [3], Cactus[4], M3C[5]). However, these systems mainly care about the user interface, but not about offering some generic services since they take some existing platform to offer its services (for example, Grid systems [6]). Our approach uses a n-their model that separates the system modules in a reduced set of layers. A prototype called IONweb (Interoperable Object Network Web) based on well know object oriented technologies uses the layered architecture. Section 2 describes the requirements of the system and the multilayer architecture proposed. Section 3 analyses the three main layers. Section 4 presents the implementation of a prototype called IONweb. Section 5 shows a case use. Finally, section 6 presents some conclusions and several possibilities of future development.
2
Web Remote Services Oriented Architecture
As it has been mentioned above, when considering a cluster system, there are three different kind of user with different degree of expertise, different purposes and, therefore, different requirements. These three types of “user” are system administrators, application developers and end users. So the first step to define the system architecture is to analyse the requirements of each one of these “users”. From an end user perspective the system must provide the following facilities: -
-
Transparent access to the system. The user does not need to know the system features, their heterogeneities or problematic (if his application does not need to know them). Remote management of computing distributed/parallel systems (like clusters) for launching applications. End user can ask for: 1) Run installed applications; 2) Run applications with a personal configuration of the user (or predefined configuration); 3) Upload the application to be run (and the set of data and parameters needed); ACLs (Access Control Lists), The correct management of the computing system requires the creation of different user profiles with different capabilities. Therefore, it is necessary to establish Access Control Lists (ACL).
370 -
Josep Jorba et al.
End users can decide to run their applications in different modes: 1) Interactive, the user can be connected for launching and waiting for the task termination (in short tasks), or it may disconnect and after reconnect for consulting their tasks status. 2) Batch, launching in disconnected mode (in time or priority queue), one task or a pool of tasks. 3) Parameterised tasks, multiple runs of a task with variation of parameters (or data used).
Considering developers of applications the system should also provide the following facilities: -
Developers or applications testers need compiling native tasks. Remote terminal access for specialized tasks (like reconfiguration, sources or data edition).
environment
From a system administration perspective the facilities should also include: -
-
-
Users Profiles: The system needs to manage different categories of user, for a restricted system view: information access permissions, resource allocation, services call restrictions... Virtual systems management: Creation of virtual environment of execution, partioning and mapping of pseudo virtual machines available to different profiles of users. Management of Tasks Pools.
There is an additional type of user involved in the system. It can be called a system integrator (usually can be the system administrator). System integrators are interested in including new nodes or clusters in the distributed system. These requirements involve different level of abstractions, some refer to final user interface, others to internal management, and others deal with services provided by the distributed computing systems. The proposed architecture involves multiple layers: User Layer (UL): functional interface with users. Remote Service Interface Layer (RSIL): Interface interconnecting users to cluster computing services. System Controller Layer (SCL): Interface controlling communication with cluster, defining final services provided by the cluster system. System Layer (SL): The cluster systems. Remote Service Interface Layer Service Controller Layer
User Layer I Application Server
N
Remote Object 1
Distributed System 1
Remote Object 2
Distributed System 2
T E
Servlet
R
Web
N
Server
User
...
E T Remote Object N
Data Base
Fig. 1. General view of the multilayer architecture
Distributed System N
Web Remote Services Oriented Architecture for Cluster Management
371
Next section describes the three main layers of the proposed architecture: User Layer (UL), Remote Service Interface Layer (RSIL) and Service Controller Layer (SCL).
3
Architectural Layers
3.1
User Layer: Multiple Device Interface
Internet is providing multiple interconnection forms to the information systems. In this evolving interconnected world, users adapt their work to the available devices at every moment. Therefore, flexibility becomes an important feature of current systems. Most applications running in a cluster system require some common management tasks such as configuring input data, configuring the execution parameters, launching the application, checking the execution status, obtaining data of application execution (results), etc. These management tasks do not require great computing power, but only a specification how to carry out them and the data themselves. These tasks can be done by the actual power of more simplified devices, like for example cellular phones with enabled Internet capabilities (WAP phones), PDAs, or any other device of mobile computing. So, these devices can be used for these purposes and the user can be liberated from working on the target system itself and just accessing it through internet. The User Layer (UL) provides the required independence and offers the interface facilities for treating all these devices. Most of these mobile devices have some enabled web browser, and the information pages (html pages) needed for management the system can be adapted and translated to display in these devices. For example, WML pages (for mobile phones) can be adapted from their corresponding HTML pages to manage the same operation. Web and XML [7] technologies provide data independent representation, and data independent visualization on multiple devices. 3.2
Remote System Interface Layer
The Remote System Interface Layer (RSIL) acts as intermediate layer between final user interface (UL) and the gateway to cluster systems (SCL). It provides the management logic to the operations available to the users (figure 2), mainly by means of remote cluster services. Depending on the complexity of each operation, the translation can be direct (one operation to a service call), chained direct (a set of basic services provides one operation), or can require internal data management, for example when the cluster system does not provide such facilities (for example, user profile management, job/tasks/users status historic management). This internal data management requires a storage data system, to integrate data information flows in the architecture. This data storage system must store the user initiated operations or in course, partial status information of operations, monitoring information, or system statistics.
372
Josep Jorba et al.
Remote Service Interfase Layer User Layer
Manager Server
Request Data
Request
Users Validation
Users Manager
Users Register
Virtual Machine Config
Users monitoring
Remote Sys. Manag.
Operations Repository
Profiles validation
Profiles Manager
Operations Register
Execution Manager
Operations Monitoring
Data
System Controller Layer
Data Manager
Storage System
Fig. 2. RSIL management processes
3.3
Service Controller Layer: Cluster Remote Services Access
The Service controller layer (SCL) incorporates a gateway to access the services provided by the Cluster system from external device/system/Web Portal clients, isolating the cluster system from the external management system. This architecture provides a clear separation between the layers of user presentation, system logic and the final system. The cluster system and the provided services are independent of the hardware/software client platform. The gateway is formulated as an API specification of a set of services offered to the clients. The gateway clients need a set of operations in the RSIL. The services are the basic indivisible operations, that can be done in the cluster. These services are used for operations such as Job configuration, Job Monitoring, File Upload/download, Virtual system configuration, System Status, User tasks status, System status, etc. These services are provided in a standard form including I/O data and their formats. However, the services are not only static, but they can be specified dynamically. An static services API offers a reduced set of services with very generic interface that can be easily adapted to any target system. But the final system can provide particular added services, that must be offered to the client. This fact forces to open the possibility of specifying new services. In our cluster management system, this new added services specification can be done without recompiling or regenerating the gateway using XML service description.
Web Remote Services Oriented Architecture for Cluster Management
4
373
Implementation of the IONweb Prototype
In our prototype IONweb, we have used well known standard technologies for developing the architecture layers. Figure 3 shows the technical solutions involved and their interconnections. In the final interface to the user (UL), we were looking for independence from visualization system (PC, cellular phone, PDA). The use of XML technologies provides independence of data representation, and data visualization, by readapting data to view formats. The combination of XML data representation with XSL [8] transformations generates final HTML pages to different destinations. In our prototype, we have tested the generation of interface to Web browsers in standard form by HTTP protocol and HTML pages, and to WAP interface for mobile phones with WML. Other specialized devices (like PDAs), could be included by applying the necessary XSL styles to adapt de data outputs. Using our interface, the user can connect to the system using multiple devices to run, test or monitoring his/her tasks & applications. Users Requests
Data User Layer
HTML XML
HTTPS
HTTPS
WML
HTTPS
WML Secured Comunications
I N T E R N E T
HTTPS HTTPS
HTTPS HTML XML
HTTPS WML
WML
Remote Service Interfase Layer
Apache Web Server HTTP
Jakarta Tomcat Application Server
CORBA messages
XML / HTML / WML
XML
11
Servlets Logic
CORBA XML messages
XSL JDBC
XML
MySQL Database
CORBA messages
XML
Service Controller Layer Corba Server 1
API
XML
Distributed System 1
Corba Server 2
API
...
XML
Distributed System 2
Corba Server N
API
...
XML
Distributed System N
Fig. 3. IONweb prototype architecture
374
Josep Jorba et al.
In the interface RSIL, we provide a web server based solution, with open standards. It includes a secured Apache Web server, with HTTPS protocol for users access to clusters. The layer logic is implemented by a combination of Java servlets and database access. The servlets encapsulate functionality of accessing to remote cluster services, and maintaining state information in a database: state of the cluster sessions, cluster tasks, users state, and control users rights (ACLs, access control list) in their profile for access the final services. The servlet runtime engine used is Jakarta Tomcat, and Java JDBC drivers for accessing the database system (MySQL). In the SCL we deal with the final cluster system providing an external standard accessible services. In our prototype we implement this using CORBA [9] middleware technology. In CORBA, the server objects can be remotely activated and response to service requests. The services implemented are language, platform, and network independent, and can be accessed by means of a standard runtime called ORB (Object Request Broker), that manages server access and client requests. The used ORB implementation has been Iona ORBACUS. In our prototype, we are working with PVM Clusters, and the SCL is implemented as a CORBA server dealing with communication to PVM daemons. Our Server can be initiated on a cluster node called representative node (or entry point). This CORBA server represents the cluster, and have a persistent internal configuration (written in XML), with information of cluster nodes and their capabilities.
5
Use of IONweb
In this section we will show a case use of an end user that wants to run a PVM application on a particular cluster. The end user connects to a web site and logs in using the interface offered by the UL. The RSIL validates the user, gets his/her user profile from the database and provides information to UL so that the possibilities available to the user are shown in a new page. The user selects, the cluster, the application, the data files and submits the application to the system. The UL transfer the request to the RSIL and this layer connects to the services interface in SCL to start up and launch the application on the cluster system. Now the application is running and the user can decide to logout and reconnect afterwards to check the status of his/her job, or to get the results.
6
Conclusions and Future Work
The system described provides an easy way to use and manage cluster systems from remote clients. The architecture provides different views of the target system to users, adapting services of the clusters to application users, application developers, cluster administrators or cluster systems integrators. The architecture is organised in three main layers, concerning user interface, management logic and cluster services. The use of independent layers simplifies the adaptation to the evolving user interfaces provided by different client devices (PCs, mobile phones, or PDAs). They are many ways to improve actual architecture, concerning new services in the system layer:
Web Remote Services Oriented Architecture for Cluster Management
• •
•
375
Remote console service for clusters. This provides a web remote interface for accessing the cluster, improving typical user operations related to the processes of edition, compilation and customisation of an application. Remote visualization service. Data visualization is a very useful and essential for the application users. This can be integrated in the system using standard file downloading services, and plug-in browser technology for adapting them to known formats or creating a new viewer for the application. Service access protocol. Some new technologies, like SOAP protocol [10], could provide better performance results in the interlayer communications for accessing the cluster services. A more standard access API offering a new level of abstraction can be obtained using Web Services [11] paradigm. WSDL (Web Services Description Language) and UDDI (Universal Description Discovery and Integration) can be used for offering the architecture services in standard form, and facilitating the description and discovery of new cluster services.
References [1]
[2] [3] [4] [5] [6] [7] [8] [9] [10] [11]
Geist, Al; Beguelin, Adam; Dongarra, Jack; Jiang, W., Mancheck, R. and Sunderman, V., “PVM : Parallel Virtual Machine – A User’s Guide and Tutorial for Networked Parallel Computing”, The MIT Press, Cambridge, (1994). Gropp, W., Lusk, E., Doss, N., Skjellum, A. “A high performance, portable implementation of the MPI standard”. Parallel Computing, 22 (6): 789-828, 1996. Akarsu, E., Fox, G.C., Furmanski, W., Haupt, T. “WebFlow – High level programming environment and visual authoring toolkit for high performance distributed computing”. In proceeding of Supercomputing 98, (1998). Allen G., Benger W., Goodale, T., Hege, H., Lanfermann, G., Merzky, A., Radke, T., Seidel, E., Shalf, J. “Cactus Tools for Grid Applications”. Cluster Computing 4, 179-188, (2001). Geist, A., Schwidder, J. “M3C - An Architecture for Monitoring and Managing Multiple Clusters”, 12th IASTED Internatinal Conference on Parallel and Distributed Computing and Systems, Las Vegas, Nevada (2000). Foster and C. Kesselman (eds). The Grid: Blueprint for a Future Computing Infrastructure, Morgan Kaufman Publishers, (1999). Extensible Markup Language (XML), available at http://www.w3.org/XML/ Extensible Stylesheet Language (XSL), available at http://www.w3.org/Style/XSL/ The Common Object Request Broker: Architecture and Specification, available at http://www.omg.org Simple Object Access Protocol (SOAP), available at http://www.w3.org/TR/soap12-part0/ Web Services Description Language (WSDL), at http://www.w3.org/TR/wsdl
