web service-based architecture of new arc components - TNC2009

WEB SERVICE-BASED ARCHITECTURE OF NEW ARC COMPONENTS Thomas Frågåt, Farid Ould-Saada, Weizhong Quiang, Alex Read, Adrian Taga, Alexander Konstantinov University of Oslo, University of Oslo, Department of Physics, P.O.Box 1048, Blindern, N-0316, Oslo, Norway

Balázs Kónya, Oxana Smirnova Experimental High Energy Physics, Institute of Physics, Lund University, Box 118, SE-22100 Lund, Sweden NDGF, Kastruplundsgade 22, DK-2770 Kastrup, Denmark

Mattias Ellert, Markus Nordén Department of Physics and Astronomy, Division of Nuclear and Particle Physics, Uppsala University, Box 535, SE-75121 Uppsala, Sweden

Iván Márton, Zsombor Nagy, Péter Stefán National Information and Infrastructure Development Institute NIIF/HUNGARNET, Victor Hugo 18-22, H-1132 Budapest, Hungary

Henning Müller University and Hospitals of Geneva, Service of Medical Informatics, 24 Rue Micheli-du-Crest, CH-1211 Geneva 14, Switzerland

Steffen Möller University of Lübeck, Institute of Neuro- and Bioinformatics, Ratzeburger Allee 160, D-23538~Lübeck, Germany

Jozef Cernak Pavol Jozef Safárik University, Faculty of Science, Jesenná 5, SK-04000 Kosice, Slovak Republic

Abstract The Advanced Resource Connector (ARC) has proven to be a reliable and high performing grid middleware solution since the first release in 2002. Originated by the NorduGrid collaboration, it is now used and developed by a number of projects including KnowARC, a European Commission funded initiative. It is used in several scientific domains, from high-energy physics and climate research to bioinformatics and clinical decision support. It is well known for its simplicity and non-invasive approach. The current development of ARC aims at providing a powerful and reliable next generation middleware based on evolving grid standards and a service oriented architecture. The middleware will be of industrial strength and quality, with enhanced security. Further, work is carried out on promoting interoperability with other grid solutions, such as Unicore and gLite. In this paper we describe the architecture of the new ARC components.

Keywords Grid, Distributed computing, Middleware, e-Infrastructure, standards

Introduction This document describes the service-oriented architecture of the next generation grid middleware based on the NorduGrid [1] Advanced Resource Connector [2] (ARC). The new architecture builds upon design principles very similar to those followed by the pre-web-service ARC middleware. Hence, it provides a natural continuation of the ARC middleware line. This middleware first emerged in 2002 as the NorduGrid toolkit. This evolved into ARC, which has proven to be a lightweight, robust, reliable, scalable, and production quality grid middleware for Europe. It is widely used in numerous scientific fields, such as the high-energy physics, bioinformatics, climate research, material sciences, clinical support and bioinformatics. The main effort in ARC development is currently provided by the European-Commission funded KnowARC project [3]. KnowARC includes a team much broader than the Nordic origins of ARC, and includes partners from countries such as Hungary, Germany, Slovakia, Switzerland, and the United Kingdom. KnowARC, which

runs from June 2006 to November 2009, is intended to strengthen ARC, improving its conformance to standards and usability, and promoting it to a broad range of users. This paper will first outline the general design principles of ARC, then proceed to describe the next generation ARC components in light of these principles.

Next Generation Advanced Resource Connector Design principles The architecture principles of the next generation ARC components build upon some fundamental design considerations of the pre-web-service middleware. These principles can be outlined as [2]: • • • • •

A truly decentralized system without bottlenecks, single point of failure of the need for centralized management. A robust and fault-tolerant system, capable of providing stable round-the clock services for years. Non-intrusive and easily portable grid tools and utilities. No extra manpower besides the staff of data centres should be needed to maintain and utilize the grid layer. Tools and utilities respect local resource owner policies; in particular those related to security.

Further, the design is also based on a survey of architectures and implementations in other relevant grid systems and projects [4]. This is crucial as KnowARC aims to learn from the work of other projects, incorporating their components where appropriate rather than repeating good work done by others. ARC will provide services for most of the capabilities identified by the Open Grid Service Architecture [5] (OGSA), of which execution management, information and data capabilities will be the main supported functions. Furthermore, while data capability in general targets data organized both in files and databases, the scope of ARC in the near future is restricted to providing support for file-based data services. Currently, there are no plans for intrinsic support for database access. ARC services enabling OGSA capabilities will be implemented within a uniform hosting environment that is a central component of our architecture (see Section 2.4). Some of the capabilities (such as security and the information capabilities) will be partially provided by the hosting environment itself. The KnowARC project has produced a detailed inventory of relevant Grid standards and our conformance to them [6], which should lead to improved interoperability with other middleware solutions that are also following these standards.

General Architecture Principles In addition to the general design principles and features outlined above, next production ARC releases are required to implement a number of identified OGSA capabilities [5]. Grid systems integrate, virtualize and manage physical resources and services within distributed, heterogeneous, dynamic Virtual Organizations (VOs) across traditional administrative and organizational domains. We define VOs as consisting of a set of individuals or institutions having direct access to resources. Physical resources, including computing power, data storage, networking capability and specialized instruments, are exposed and managed by a set of services. Services are loosely coupled peers that, either on their own or as part of an interacting group of services, provide a capability: a function useful in the grid context. Services are networkcapable units of software that are accessible through well-defined interfaces, implement logics, manage states, communicate via messages and are governed by policies [8]. ARC aims to facilitate the seamless use and management of distributed, heterogeneous resources via implementing a Service Oriented Architecture (SOA) [9]. The SOA concept embraces the idea of building modular, reliable distributed systems that deliver functionality through loosely coupled services. SOA significantly increases the abstraction level for code reuse and provides a clean model in which to integrate software systems both within and across organizational boundaries. The ARC services will be accompanied by ARC clients, elements that use and integrate services of the capabilities above. ARC clients will be developed using the client libraries while third-party solutions can directly interface to the ARC services through their Web Service interfaces (see Figure 1).

Figure 1: An overview architecture diagram of the internal structure of ARC client and server side middleware components. On the client side are clients that use services by means of the arclib1 client libraries. Communication with the server side of ARC is done through a low-level arclib1 communication layer. Communication with ARC Classic and external middleware solutions is made through specific adaptors. On the server side, there is a hosting environment that receives all requests from the client side and passes them on to appropriate services. Some services rely on local resources that can be accessed directly or through external service modules. The new ARC middleware releases will come with new service interfaces as a consequence of the planned SOA transition and the switch to conformance with a range of emerging standards. To provide transitional backward compatibility with pre-web-service ARC, the new system should be able to seamlessly co-exist with the old services deployed over the same resource. Running both the new and old services on the same resource for a transitional period will facilitate backward compatibility. Additionally, this allows ARC releases to incrementally add the new components as they become available rather than make a single massively revised release. This both gives ARC users access to new components as quickly as possible and spreads the load of getting to grips with the new components over a period of time rather than forcing users to switch all at once.

Internal Structure of ARC Components An overview of the internal structure of ARC server and client side middleware components is presented in Figure 1. On the server side, there is a hosting environment that receives all requests from the client side and passes them on to the appropriate services. Some services rely on local resources (e.g. batch systems or database systems, usually provided by a third-party) that can be accessed directly or through external service modules. For example, there may be several external service modules available to an execution service for interfacing with different batch systems. Other services (such as schedulers) do not use any local resource themselves. Still, there may be several external service modules available that can provide different strategies for the scheduler. On the client side, there are command line interfaces (CLI) and a client with a graphical user interface. There are also special-purpose third party user applications that users can interact with. Those tools use services by means of client libraries, which communicate with server side components through a communication library. In the integrated client library there will be an API with methods for performing operations on different services. The API will also contain higher-level methods that perform operations involving several services, e.g. job submission. This is the main reason why clients should not use the communication library directly to communicate with services. The communication library will contain components such as SOAP messages, HTTP clients, and TCP clients that are organized in layers, making it possible to construct complete communication chains.

Hosting Environment The Hosting Environment provides residential space for services and a collection of programming libraries, message chain components (MCC) and daemon components. The internal structure of the Hosting Environment Daemon (HED) (also showing the security framework) is shown in Figure 2.

Figure 2: The internal structure of the Hosting Environment Daemon (HED). Messages are routed through Message Chain Components (MCCs) between the appropriate service and the outside world. One of the main functions of this environment is to route messages between the services and the outside world and to provide efficient inter-service communication for services residing in the same hosting environment. The hosting environment supports different levels and forms of communication mechanisms: from simple UNIX sockets to HTTPS/SOAP messaging. The motivation for this design is to separate all the communication related functions from the service logic itself. To provide a flexible solution for this communication, the hosting environment includes dynamically loadable MCCs, organized into message chains. The hosting environment contains libraries for other common functions such as reading and parsing configuration files, loading modules, logging errors and events, and accessing authorization services. Together they constitute the hosting environment internal API that allows simple and efficient service development.

Security of the Hosting Environment Since the HED component will provide the basic functionalities for various services, it should also provide basic functionality for security. KnowARC prepared a report that gives a comprehensive review of ARC security [10]. Communication security for ARC covers both transport and message levels. In the transport level, the SSL v3/TLS v1 protocol is supported to guarantee transport confidentiality and authentication. In the message level, the WS-Security standard will be supported to provide protection through message integrity, message confidentiality, and single message authentication for SOAP messages. The authentication mechanism is based on X.509 public key infrastructure, supporting authentication for transport and message levels. Authorization includes a policy information point (PIP) that collects attribute information about users, and a policy decision point (PDP) that retrieves policy information and makes authorization decisions. The authorization module supports plugins and can be customized for any ARC service. The PDP provides extensibility for different policies and request expressions. It supports a specific XML policy file, and will support eXtensible Access Control Markup Language [11] (XACML) as well. The Security Assertion Markup Language [12] (SAML) standard will also be used for supporting attribute and authorization assertions. VO management, which is related to authorization, will be supported through Virtual Organization Management Service (VOMS). Single sign on (SSO) is implemented by using the certificate delegation mechanism. Furthermore, the SAML attribute assertion generated by the identity provider (which will authenticate the user and acquire user attributes from an attribute authority of the VO, i.e. VOMS) will also extend the identity-based

SSO to an attribute-based SSO.

Services Identification of the necessary services resulted from an extensive requirement analysis study, a survey of current technologies and the ARC team’s wealth of experience accumulated during the development of different grid middleware solutions, such as pre-web-service ARC. Information System. All the ARC Information System components will be implemented in the general ARC Hosting Environment framework and will make use of the ARC security libraries. ARC aims to keep local information locally, close to the place and service where the information was generated. ARC services will provide a complete description of themselves, including the resources behind the service through the native Service Interface of the corresponding service. The local service and resource description will provide characterization of grid resources by specifying static, semi-static and dynamic properties, including service information, its status and detailed resource specification. The information model to be used for service and resource description is the GLUE 2 schema [13], a standard to which KnowARC has significantly contributed. The Service and Resource Description component will support sophisticated queries, relying on ARC info backends being able to provide functionality for caching, searching and collecting information. Individual grid resources are connected to an "information mesh" by dynamic registration processes performed by Info Registration Services and utilizing peer-to-peer-like indexing technologies. The indexing mechanism itself will be an essential Information Service, the Information Indexing System. The Indexing System itself will not provide detailed service or resource information. Nevertheless, some small subset of static resource and service properties are foreseen to be stored in the Indexing System together with the essential service contact information. Grid clients performing resource discovery will contact the Indexing Service in order to find available resources. Once a resource is discovered the client can perform a query directly to the actual service in order to obtain up-to-date, detailed information. The MARS accounting service will collect, manage and present current usage and historical information about grid resources and jobs. Furthermore, it will provide an interface for customers to query these metrics for, say, reporting or accounting purposes. In addition to a single metric query, the query functionality will also enable detailed and aggregated reports per individual user, VO or a resource. Execution Capability and the A-REX service. The services behind the Execution Management capability deal with initiating and managing up to completion units of work (grid jobs). The architecture design identifies the ARC Resource-coupled EXecution service (A-REX), selection services and job management services as the most important services for realizing this capability. The A-REX service can be considered as an "atomic unit" or a core of the Execution Management capability in ARC, while the rest of the services represent higher-level execution management services. The A-REX service is the final endpoint of a Grid job and it provides a link between services and clients of the grid layer and the computational job management software, which covers one particular computer cluster. The current implementation of the A-REX service has the Grid Manager [2] component of pre-web-service ARC as its core. Hence, it inherits the powerful Grid job management functionality of the former releases. The core is extended with a WS interface that implements the widely accepted Basic Execution Service [14] (BES) WS interface for job management of acceptable Grid jobs being described using the Job Submission Description Language [15] (JSDL). Both the BES interface and the JSDL are extended to provide functionality specific to the A-REX service. Still, A-REX maintains backward compatibility for third-party clients without support for these extensions.The Web Services Resource Framework [16] (WSRF) WS-Resource Properties (WS-RP) interface is also present, which provides information about the service state and capabilities as well as extended information about managed grid jobs. Information is formatted according to the ARC schema inherited from pre-web-service ARC by using a proprietary LDAP Data Interchange Format (LDIF) to XML converter. The GLUE 2 schema will also be adopted as soon as the specifications are released.

Data Management Capability. The Data Catalogues will be based on a consistent distributed hash table that will make it possible to eliminate all kinds of single points of failure. This distributed hash table will be a general-purpose, property-value store that may later be used as a stand-alone solution. Together, the Data Catalogues and the ARC Storage Elements (SEs) will be able to manage replication of files. The SEs will have logic to register themselves to the Catalogues and to send periodic heartbeat messages with information about changes in the state of the stored files. This will allow the system to notice broken replicas when they break and not only when a service attempts to read them. The Storage Managers will provide a UNIX-like file system interface with methods such as list, copy, move, create collection, etc. Here, a collection is very similar to the directory concept of a normal local file system. These methods work on Logical Names that are very similar to paths and file names in a local file system. All these functionalities create a global file hierarchical system namespace where VOs and users may have a home collection to which they have all permissions. There will also be a sophisticated access control mechanism that gives users fine-grained permission control over their files and collections. Storage Gateway services will provide a way to integrate existing data on third-party storages to ARC global namespace in next generation ARC. If a third-party storage has its own namespace there will be a way to mount that namespace into the ARC global namespace and to use ARC Storage Managers to access the files on the third-party storage by using exactly the same methods as for files stored on ARC SEs. Resource Management Capability. General aims for resource management in ARC comprise: • • •

Automated installation of software on computing resources and avoidance of manual interference with this process. Presentation of information on available resources (or their installation status). Automated de-installation of software on computing resources.

These aims will be achieved by the means of dynamic Runtime Environments (RE) and deployment of virtualization techniques. An RE specifies an application software environment required to allow the execution of a task. Descriptions of dynamic and manually installed REs are stored in a machine-readable format and offered through RE Catalogues. A Catalogue has a public interface to retrieve the formal descriptions of REs and to search REs. The Janitor service [17, 18] makes use of the Catalogue in order to deploy a dynamic RE on demand. The Janitor also supports dynamic RE management operations, such as verification, removal, lifetime management and modification of access control. The Janitor can deploy virtual disk images for the Virtual Machine Management service (VMMS). Virtualization will help to separate running jobs from each other, providing higher security level or separated jobs execution environments. VMMS will make it possible to start, stop and control different kind of virtual machines on the resource side. Self-Management Capability. The orchestration of ARC services will provide a certain level of selfmanageability of the grid system: • • •

self-configuring, self-healing, self-optimizing.

These mechanisms heavily rely on the information capability services. The job manager services, using the dynamic RE services of ARC, can self-configure and so dynamically adapt the grid to changes in the system by using policies provided by VO managers or resource administrators. Self-healing mechanisms that detect improper operations of services and initiate policy-based corrective action without disrupting the grid environment are critical for achieving reliability. These mechanisms will be integrated into infrastructure-level services, such as the Information Indices and Data Catalogues. The self-healing mechanisms will also be used in the storage system. Self-optimizing mechanisms are able to self-tune the system to meet end user or business needs (e.g. they will address the problem of transferring Grid

jobs waiting in slow queues to faster queues, and to move very long running jobs from resource to resource by migration). Self-optimization makes use of self-configuration in its implementation. Clients. In general, there will be several components that can be categorized as clients: a generic multifunctional CLI, a GUI, a Web portal, a workflow management tool, a monitor, an RE management client and even hybrid client servers, such as e.g. gateways. ARC will offer a convenient client library wrapping around the WS interface of the ARC services. Based on these, there will be an end-user client package that can be used out of the box, providing interfaces to all the relevant services. The change of the core service interfaces implies that clients of pre-web-service ARC will not be able to access resources offered through the new service interfaces. From the client perspective, backward compatibility with old servers will be integrated into the new releases of ARC client utilities for a transitional period so that new clients will be able to talk to both the new and the old service interfaces. The client will have the following functionalities: grid credentials handling, single job submission and subsequent manipulation (e.g.: status query, deletion, hold/release, retrieval, re-scheduling, retry, and attribute change), single file manipulation, bulk operations on jobs and files, and general information discovery and also a test suite.

Conclusions While ARC has been in production use since 2002, the revised ARC architecture and new components available further raises the quality of the middleware, increasing its efficiency, usability and flexibility. Through the KnowARC project’s significant contributions to the Grid standards community and the implementation of these standards in a real production-scale Grid infrastructure, ARC is able to interoperate with other standards-driven middleware and infrastructures. These and other new features of ARC are made available to the ARC user community as they become available, such that users can always have access to the most advanced ARC system possible. This is possible because of the modular nature of ARC. While introducing many new abilities and features, all the changes to ARC are in line with its basic goal: to make a simple, lightweight and platform-agnostic middleware solution appropriate for an extremely wide range of users, applications and situations.

Acknowledgments This work was supported in part by the Information Society and Technologies Activity of the European Commission through the work of the KnowARC project (Contract No.: 032691).

References [1] Ould-Saada, F.: The NorduGrid Collaboration. SWITCHjournal 1 (2004) 23-24 [2] Ellert, M. et al.: Advanced Resource Connector middleware for lightweight computational Grids. Future Generation Comp. Syst. 23 (2) (2007) 219-240 [3] Grid-enabled Know-how Sharing Technology Based on ARC Services and Open Standards (KnowARC), Web site. http://www.knowarc.eu [4] KnowARC Deliverable D1.1-1 - Design Document. URL http://www.knowarc.eu/documents/Knowarc_D1.1-1_07.pdf (2007) [5] The Open Grid Services Architecture, Version 1.5 (2006). URL http://www.gridforum.org/documents/GFD.80.pdf [6] KnowARC Deliverable D3.3-1 - KnowARC Standards Conformance Roadmap (2006). URL http://www.knowarc.eu/documents/Knowarc_D3.3-1_06.pdf [7] Foster, I., Kesselman, C.: Globus: A Metacomputing Infrastructure Toolkit, International Journal of Supercomputer Applications 11 (2) (1997) 115-128 [8] Open Grid Services Architecture Glossary of Terms Version 1.6, URL

http://www.gridforum.org/documents/GFD.120.pdf [9] OASIS Reference Model for Service Oriented Architecture V 1.0. URL www.oasisopen.org/committees/download.php/19679/soa-rm-cs.pdf [10] KnowARC Deliverable D1.6-1 { KnowARC Security Review (2007). URL http: //www.knowarc.eu/documents/Knowarc_D1.6-1_07.pdf [11] OASIS eXtensible Access Control Markup Language (XACML) TC. URL http: //www.oasisopen.org/committees/xacml/ [12] Use of SAML for OGSI authorization. URL http://www.ogf.org/documents/GFD.66.pdf [13] OGF Glue Schema Working Group, Web site. URL http://forge.ogf.org/sf/projects/glue-wg/ [14] OGSATMBasic Execution Service Version 1.0, URL http://www.ogf.org/documents/GFD.108.pdf [15] Job Submission Description Language (JSDL) Specification v1.0, URL http://www.gridforum.org/documents/GFD.56.pdf [16] OASIS Web Services Resource Framework (WSRF) TC, Web site. URL http: //www.oasisopen.org/committees/tc_home.php?wg_abbrev=wsrf [17] KnowARC Deliverable D2.5-1 { RDF Based Semantic Runtime Environment (RE) Description And Dynamic RE Management Framework Including Creating Proof Of Concept Bioinformatics REs. URL http://www.knowarc.eu/documents/Knowarc_D2.5-1_07.pdf (2007) [18] D. Bayer, T. Bhimdi, G. Oechsler, F. Orellana, A. Waananen, B. Konya, S. Moller: Dynamic Runtime Environments for Grid Computing, submitted to Cracow Grid Workshop 2007 (CGW07)