use also in the matchmaking of resources and users. 1. INTRODUCTION. The needs of e-science and e-business present several inter- sections. The problem ...
An OWL-S Based Approach to Express Grid Services Coordination Laura Bocchi, Paolo Ciancarini, Rocco Moretti, Valentina Presutti, Davide Rossi Dept. of Computer Science University of Bologna Mura Anteo Zamboni 7 Bologna, Italy {
bocchi | cianca | moretti | presutti | rossi }@cs.unibo.it
ABSTRACT E-science and e-business share the need of a middleware that integrates dynamic services running on distributed, heterogeneous platforms. The Grid community is addressing this problem by moving towards Web services as it is proposed in the Open Grid Service Architecture (OGSA). In this paper we discuss the benefits of using an ontology driven approach to Grid services in order to enhance their composability thus enabling the submission of multiple interoperating jobs. Specifically we introduce an OWL-S extension to describe relations among Grid jobs in order to express coordination-related aspects. This enables to expose coordination requirements in an orthogonal way with respect to single jobs management. As a proof of concept we use this extension to express relations among jobs providing for their use also in the matchmaking of resources and users.
1.
INTRODUCTION
The needs of e-science and e-business present several intersections. The problem, in its more general significance, is the need of a middleware that integrates dynamic services running on distributed, heterogeneous platforms. In the forthcoming scenario, solutions to e-science and e-business needs are likely to converge, at least partially. In the e-business scenario, this problem is being addressed by Web services related technologies. The Grid community is also moving towards Web services: the Global Grid Forum refers to Web services in the Grid architecture since 2002 [22]. A good reason for e-science applications to deal with Web services is to leverage from a promising technology and to participate to a shared set of standards, thus simplifying the engineering of new applications. On the other hand, the Grid would possibly evolve toward a direction enabling the usage of its resources and services to a wider range of users, not just the scientific community.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. SAC ’05 Santa FE, New Mexico USA Copyright 200X ACM X-XXXXX-XX-X/XX/XX ...$5.00.
We propose to express the convergence described above at the level of ontology. The World Wide Web Consortium (W3C) recommends the use of the Web Ontology Language (OWL) [12] for the definition of ontologies on the Web. OWL-S [7] is an ontology of services that has been developed as part of the DARPA Agent Markup Language Program (DAML) [2]. Our proposal is to extend the OWL-S ontology with the Grid related concepts of Grid Service, Job, and Job Submission Description (JSD). Informally, a Grid Service is a Web service with specific features: stateful interactions, life-cycle management, externally accessible state and the possibility to subscribe for notification of events. A Job is an activity being executed by a Grid service, and a JSD is a unit of work describing a Job. A user submits a JSD to the Grid for (typically) having the job executed. The integration of these concepts with the OWL-S allows us to identify, with a rigorous method, the relationships between the basic concepts and technologies related to Web services and the Grid. Further evolutions in one of the two areas will be reflected in the overall ontology, such that the overall evolution would be possibly observable through a homogeneous process. One of the aspects emerging from the extended ontology is related to the coordination between jobs. Statefulness is an important requirement in many ebusiness scenarios. It typically implies a complex coordination of the activities performed by each single service. In the Web service scenario, Orchestration and Choreography are aimed to manage coordination as an orthogonal dimension with respect to the management of each single activity (i.e., the application code). Orchestration and Choreography define the coordination of activities through the execution and the management of a Business Processes. The ServiceModel of OWL-S describes these aspects of coordination by focusing on the features of (business) processes. In a number of Grid implementations, such as Datagrid [4, 3], just a minimal level of coordination between jobs is offered by Condor DAGMan [5]. It allows to submit multiple jobs and to specify, by means of a Direct Acyclic Graphs (DAG), a causal relation between their execution. The capabilities provided by DAGMan are easily extensible with the concepts of ServiceModel (i.e., business process): coordinating jobs with a business process would make it possible to express more complex dependencies between jobs, and to achieve a major degree of automatization in jobs management. The Business Process Execution Language for Web
WEB SERVICES: THE STATE OF THE ART
In general, a Service Oriented Architecture (SOA) is ‘a set of components which can be invoked, and whose interface descriptions can be published and discovered ’ [14]. A SOA is composed by network addressable services with a well defined interface. Services have stateless connections and use standard communication protocols. The Web Architecture is an instance of SOA where services are Web servers, the interfaces are those of Web pages and the final user is typically a human. The Web architecture is currently evolving toward a Web Service Architecture (WSA) [15]. The WSA is also an instance of SOA where services have well defined interfaces and are accessible to human, other services, and software components in general. In the Web Service Architecture all the Web services have to provide an interface that is expressed using the Web Service Definition Language (WSDL) [17]. Web services are based on a very minimal set of concepts: service, XML document, address and envelope [33]. The messages contained in the envelope and the WSDL interfaces are XML based; this has a crucial role in the automation of Web services interactions. The WSA does not provide statefulness of interactions as a built in feature: all the data for a request must be in the request. Behind a WSDL interface there might be a variety of entities: a stateless service, a subservient object, a human or an autonomous agent with its own life cycle.
APPL
2.
UIMS DBMS
The section 2 presents the essential terms and concepts in the context of coordination of Web Services. In the section 3 we discuss the convergence between the Grid and Web services and discuss about this integration at the level of ontology. The section 4 considers a motivating scenario and an example discussing about how non functional aspects can be used by a Grid service in order to choose among several available resources. Finally, the section 5 presents our conclusions.
OS 1970’s
APPL
WFMS
Structure of the Paper
OS 1960’s
APPL
DBMS
1.1
UIMS APPL
DBMS
Services (BPEL4WS) [18] is currently being considered as an alternative by Condor [1] developers but it is not clear if this is the best alternative. BPEL4WS is a language to define business processes in the context of Web service; it has been proposed by a consortium enclosing BEA, IBM and Microsoft. Finally, the representation of typical Grid concepts with OWL-S allows to describe how such concepts (i.e., jobs) can be merged and enhanced by the issues of Web service coordination that are already being discussed by the community. We refer to the possibility of supporting, through the coordination of single jobs, the submission of sets of possibly interoperable jobs, not just single or batch ones. The choice between many available services for a single service request, typically involves three steps: determining services that respect minimal requirements expressed by the requestor (matchmaking), ordering the suitable services up to some preferences (ranking), choosing the service (selection). When more than one interoperable service has to be selected, the choice should not ignore the relations between single services and the relation with the overall coordinating process. We discuss this possibility in an example where matchmaking is done on the basis of information like the distance between highly interoperable jobs.
OS
OS
1980’s
1990’s
Figure 1: Progressive separation of different computational aspects
Figure 2: Coordination of distributed parties with no centralized control
2.1
Web Services Coordination
Many business-to-business scenarios need a stateful, collaborative behavior thus implying complex coordination. It is thus crucial to specify a precise order and causality of the service operations (for example: a purchase should happen after a payment). In the Web service scenario, nothing prevents from coordinating business processes at the level of application code. Anyway, the shared interest of both industry and academia about workflow management is supporting the orthogonality between computation and coordination. In this scenario, the general equation first discussed by Carriero and Gelernter [16]: application = computation + coordination
(1)
can be rewritten as [27]: workflow = activities + processes
(2)
where the basic entities involved in a workflow are activities performing working units while processes are employed for coordinating such activities. Considering Web services, activities can be processes themselves that are invoked by means of standard messages along the Internet. Figure 1 represents the progressive separation of different computational aspects from the application code. Firstly, in the 70’s the delegation of data management to DBMS; then in the 80’s the introduction of User Interface Management Systems (UIMS); finally, in the 90’s the separation of the business logic from the application code is expressed by a centralized Workflow Management System (WFMS). At present, this evolution is going towards a decentralized interoperation (see fig. 2) where Web services (WS) provide an interface to applications, and WFMS’s have two coordination components: orchestration and choreography. The separation of concerns between activities and processes acquires particular relevance when our perspective involves distribution across many logical boundaries, for example Virtual Organizations (VO’s) in the Grid scenario. In fact, distribution makes interoperability more complex (e.g., managing the evolution of a system and updating the inter-
and provides; • is DescribedBy a service model through the concepts defined in the ServiceModel ontology; • supports a way to be accessed. This is specified through the concepts defined in the ServiceGrounding ontology. This ontology of services provides to us a way to model the integration of the Web service domain and the Grid domain. Figure 3: Main classes of the OWL-based Web service ontology nal/business logics, heterogeneous resources integration, security, etc). Current work, under the general headings of orchestration and choreography, attempts to standardize the possibility of service composition. Although it does not exist a universally accepted definition. The two terms are somehow overlapping, and somehow referring to complementary perspectives of the composition problem [30]. In fig. 2, the dashed box encompasses choreography overview of the interaction; that is the order of exchanged messages. Note that a choreography could involve an arbitrary number of parties. Two business processes have been defined, thus two perspectives and two orchestrations exist. Let us consider a multi parties interaction involving distributed Web Services. Orchestration characterizes executable business processes that embody the perspective of a particular party. Choreography describes the interaction pattern between different distributed business processes. Service composition is currently addressed by two approaches that in [20] are called: Composable Service Assurances and Service Composition. Composable Service Assurances comprises a set of standards that are defined by means of specific information inside the SOAP message headers; it intends to extend base Web service properties considering quality of service issues such as security (e.g., WS-Security [11]), reliability of messaging upon unreliable connections (e.g., WS-ReliableMessaging [13]) and support for business transactions (e.g., WS-Coordination and WS-Transaction). Service Composition refers to orchestration and choreography standards. Orchestration languages are for instance BPML [19], XLANG [32] and WSFL [26]. BPEL4WS [18] has been proposed as a merge of XLANG and WSFL, therefore it presents analogous features. In this sense it could be defined as an orchestration language. It is usually referred to as a choreography language since it also allows to specify some roles, partners, and relations for a complex distributed interaction. The Oracle WS-CDL [25] is a choreography standard.
2.2
An Ontology for Web Services
We recall that OWL-S is a OWL-based ontology of Web service which provides a set of constructs for describing the properties and capabilities of Web services in unambiguous, computer interpretable form. OWL-S defines the concept of Service which provides the reference class for the instantiation of any kind of Web service. The OWL-S ontology is structured in three main parts which are shown in figure 3. In particular, a Service: • presents a profile through the ServiceProfile ontology. The profile describes both what the service requires
3. 3.1
INTEGRATING THE GRID WITH WEB SERVICES The Grid Toward Web Services
The coordination of Grid services has to take into account the lack of centralized control in a scenario comprehending multi-institutional Virtual Organizations (VO’s) [22] spanning software boundaries, multiple locations and different machine architectures. The current production systems (e.g., LHC Computing Grid [6], INFN-GRID [21]) are based on the de-facto standard Globus Toolkit 2. It relies on a layered architecture where the downmost layer is the Fabric providing shared access mediated by Grid protocols to operations on local resources. The Connectivity layer defines core communication protocols to enable the exchange of data between Fabric layer resources. Further, it defines authentication protocols built on communication services in order to provide secure mechanisms for verifying the identity of users and resources. The Resource layer defines protocols for the secure negotiation, initiation, monitoring, control, accounting, and payment of sharing operations on individual resources. It does not consider issues concerning global state and atomic actions across distributed collections. The Collection layer contains protocols and services that are global in nature. It captures interactions across collections of resources. Finally, the upmost layer is the Application comprising the user applications that operate within a VO environment. We pointed out in the introduction the trend of the Grid community to move towards Web services. Since 2002, the Global Grid Forum introduced a reference for Web services in the Grid architecture. We refer to the Open Grid Service Architecture (OGSA) [22]. OGSA is an evolution towards Grid systems based on Web services concepts and technologies. The OGSA basic layer is provided by Web services standards, but this infrastructure does not satisfy all requirements common to Grid services such as, life-cycle management, externally accessible state, and stateful interactions. Life-cycle management refers to the capability for a client to create and destroy new instances of a service. The instance can be destroyed both directly and for soft-state expiration [24] (a service is destroyed for timer expiration if clients do not explicitly extend its life period). For externally accessible state, we mean the notification system of a Grid service. Stateful interaction addresses the need of session abstraction and correlated message between services. We mentioned above that Web services should do not assume any particular underlying programming paradigm: Web Services are not distributed objects [33]. In order to satisfy also these requirements, the Open Grid Service Infrastructure (OGSI) [9] has been defined and implemented by the Grid community. Several OGSI implementations have been developed for different platforms (among them the
Globus Toolkit 3 by the Globus Alliance [8]). A number of criticisms have been made to OGSI in this sense: (1) OGSI assumes an object oriented protocol paradigm; (2) OGSI requirements can be expressed more generally by means of correlated messages using existing Web services specifications (i.e., composable service assurance). Such criticisms led to a further binding between Web services and Grid. In fact, OGSI has been supplanted by a new set of specifications announced on January 2004: the WS-Resource Framework (WSRF) [23]. It is a refactoring and extension of OGSI that is closer to the funding concepts of Web services. The WSResource Framework will be implemented in Globus Toolkit 4.0 expected by September 2004.
3.2
Extending OWL-S with Grid Jobs: a Proposed Ontology
E-science and e-business share the need of a middleware that integrates dynamic services running on distributed, heterogeneous platforms. The Grid community is addressing this problem moving towards Web services as it is the proposed in the of OGSA. The aim of this paper is to discuss how this convergence can be represented with an extension of OWL-S that involves the most relevant concepts of the Grid. Benefits of this approach are discussed in section 1 and are illustrated in the example presented in section 4. In particular, the possibility of enhancing the interactivity and interoperability of Grid services, and how to use coordination aspects described in the ontology, for the matchmaking of resources and requestors. The main aspects of the ontology extension concern the definition of the GridService, Job and the JobSubmissionDescription classes. The figure 4 shows the OWL-S structure after the introduction of the Grid related classes. First of all we define the concept of Grid service as a specialization of the class Service having specific properties, according to the Grid services specification. It inherits the three main relations (i.e., the properties) presents, DescribedBy and supports between Service and respectively ServiceProfile, ServiceModel and ServiceGrounding. It also inherits the fact of possibly being provided by a resource. We then introduce the Grid related concept of Job, a basic activity in a (business) process. By analyzing the elements that constitute the ServiceModel ontology we can observe the absence of a concept with which a Job Submission Description (JSD) can be represented, that is the description of an activity (i.e., a job, a basic unit for a business process). In fact, in the OWL-S view the basic unit that is modeled is the process (i.e., the AtomicProcess). The JobSubmissionDescription is submitted by a resource and describes a Job. The description of a job is meaningful for several reasons. First of all, a JSD is typically submitted by the user and it contains the code to execute, the necessary data for the execution, and information about the requirements and constraints useful for the matchmaking. ¿From our point of view the submission of a JSD is a part of a process but is not a process itself, on the other hand the execution of a job (i.e. the job) can be seen as a type of process. In OWL-S three types of processes are defined: Atomic Processes (AtomicProcess), Simple Processes (SimpleProcess) and Composite Processes (CompositeProcess). An atomic process is a basic unit of implementation, accessible through, at most, an input request and an output reply. A simple process is an
Figure 4: An extension to the basic set of OWL-S classes
Figure 5: Processes and jobs in the OWL-S extension abstraction of either an atomic or a composite process and it is not directly invocable. A composite process is built by coordinating, trough the ControlConstructs (split-join, sequence, if-then-else,...) any kind of processes. As shown in figure 5, we model the Job in the ServiceModel ontology as a possible component of a generic process. The Job class is a sub-class of the AtomicProcess that can be executed by an instance of a GridService.
4.
MOTIVATING SCENARIO
Our aim is to illustrate an example where the information embodied in an ontology describing a service, concerns the coordination of multiple jobs in an overall process. For each single job, a resource has to be chosen in a range of several available ones. In current Grid systems, the choice typically depends on the requirements of the single job and on the status of the resources. Information about the other jobs that compose the overall process are not taken into account. In contrast we show how the information about relationships between different jobs can be used in the choice. In our case, the information is enclosed in the ontology representing the process. Resources that offer the same functional characteristics can be very different among them and the choice of one in place of another should be made taking care of compositional and non-functional aspects. The example that follows merges simple non-functional aspects (i.e. closeness of resources) and simple compositional aspects (i.e. parallel execution of potentially interoperable jobs).
4.1
An Example
A
B
Figure 6: A is a storage of meteorological data and B is a the computing element controlling irrigation. The blue boxes represent different localities, the green dashed box represents localities with a high degree of interoperability In the Grid area, core services are related to computing, storage and network services. A computing service is a uniquely identified service providing computing power in a certain execution environment. A storage service manages storage spaces. A network service is a uniquely identified Grid service offering unidirectional communication capabilities between network domains that are meant as sets of services sharing the same connectivity. Such services can be dynamically composed on the basis of user requirements in order to set up and provide complex services. Therefore, it is essential to be able to describe user requirements and service properties for deciding if a profitable interaction can take place. We take as an example the closeness between computing elements executing interoperable jobs. We consider a workflow that represents the controller of field irrigation (see fig. 6). The workflow involves two jobs. The first one is a storage service that memorizes meteorological information (A). The second one is a computing service that is able to process meteorological data and to manage the fields irrigation (B). The overall activity has to optimize the watering activity using meteorological forecast (i.e., nor too few and not too much irrigation according to precipitations). These information suggest to choose well connected (close) resources. The notion of closeness can be instantiated in many ways. For example, A and B have to be well connected since the computing service B has to access many data residing in the meteorological sensor A. In fact, many scenarios imply a notion of connectivity that involves latency and bandwidth. Also, A and B have to be physically close so that information held by A are meaningful for B. We split the overall information about resources selection in two separated areas: the information about requirements/preferences of multiple jobs compositions and the information about the requirements/preferences of single jobs. The former kind of information is here described using OWL-S; in particular we express in this example the request of closeness between resources executing parallel, interoperable jobs. The latter kind of information can be described by any language for resource description. We model the requirements for A and B with a widely employed language for describing Grid services and service agreement proposals: the Classified Advertisement (ClassAd) language [31]. It follows a brief discussion on how ClassAd notation is used by computing systems (e.g., Condor [1]) to implement the actual process of discovery and allocation of resources. In general, a Grid user can submit to the broker/matchmaker service a Job Submission Description (JSD) document describing the expected complex Grid service. Jobs require-
ments are described within JSD using a ClassAd based syntax. The broker/matchmaker executes a matchmaking between the JSD and the available Grid services, also expressed by the ClassAd language. One solution (i.e., a resource or a set of resources) is selected according to a ranking of all matching solutions by the broker/matchmaker. The overall workflow described above (see fig. 6) can be represented as follows. Let us see first how the requirements of the workflow composed by jobs A and B can be described with OWL-S. A and B are concurrent jobs. The parallel composition of a set of jobs is expressed in OWL-S by the ControlConstruct representing the split. The split construct is associated to a set of components, defined as a ProcessComponentBag, that in our example is called ‘jobs’ and contains A and B.
We then define the ProcessComponentBag class ‘jobs’. We express the two requirements related to physical and virtual closeness between the jobs enclosed in the bag. In particular, it is required that the physical distance between the components has to be less than 10 and that the bandwidth between components has to be more than 200. Single requirements description for A and B with ClassAd are presented later. 10 200
The properties expressed above represent necessary requirements that the requestor asks to services. In the same way, it is possible to specify a ranking function that expresses the preference of the requestor. The ranking function characterizes an order of suitability of all the services that respect the requirements. This ranking function affects the selection of resources. The function expressed below states that: if A and B are in the same URI then the ranking is 1M , otherwise it is proportional to the available bandwidth and the inverse of the physical distance. ClassAd (A.DomainURI== B.DomainURI?1M:Bandwidth + (1/PhysicalDistance)*1M)
Let us see how the requirements of single jobs A and B can be described by ClassAd. ClassAd expresses the possible values of attributes of interest with equations on the form ‘attribute=value’. A JSD is defined by a set of these equations where attributes model properties like: the path name of the user job (Executable), the set of input files for the job (FileStageIn) and the set of output files for the job (FileStageOut). The features of each component service are described by ports including the name of the service, constraints and ranking expressions (Name, Constraint, Rank). Constraints are boolean expressions while the ranking consists in defining an arithmetic expression synthesizing values used for sorting the services.
[
(1/ns.PhysicalDistance)*1M); Type = "job submission description"; Executable = "/usr/local/CaptureMeteoData"; FileStageOut = {"MeteoData.xml"}; Ports = { [ Name = A; Constraint = A.Type == "computing service" && A.OSType == "Debian Linux" && A.TotalCPUConsumption >= 2000; && A.Antenna = "yes" Rank = A.TotalCPUConsumption; ], }
]
[ Type = "job submission description"; Executable = "/usr/local/IrrigationManager"; FileStageIn = {"MeteoData.xml"}; FileStageOut = {"Result.xml"}; Ports = { [ Name = B; Constraint = B.Type == "computing service" && A.OSType == "Debian Linux" && cs.TotalCPUConsumption >= 2000; Rank = cs.TotalCPUConsumption; ], }dif ]
The two orthogonal aspects of the example showed above, that is the description of coordination between jobs and the description of single jobs, can be both modeled using the ClassAd language. However this approach provides a low degree of modularity in the description of complex jobs. The foregoing example is expressed below by a single JSD, enclosing the description of A and B. Coordination related properties (i.e., constraints and the ranking function concerning the closeness of A and B with respect to the physical distance and the bandwidth) are also expressed in the same JSD as requirements for network services. [ Type = "job submission description"; Executable = "/usr/local/myApplication"; Arguments = "--config /home/ui/my.cfg"; FileStageIn = {"gsiftp://host37/Data.tpl"}; FileStageOut = {"Result.xml"}; Ports = { [ Name = A; Constraint = A.Type == "computing service" && A.OSType == "Debian Linux" && A.TotalCPUConsumption >= 2000; && A.Antenna = "yes" Rank = A.TotalCPUConsumption; ], [ Name = B; Constraint = B.Type == "computing service" && A.OSType == "Debian Linux" && cs.TotalCPUConsumption >= 2000; Rank = cs.TotalCPUConsumption; ], [ Name = ns; Constraint = (A.DomainURI == B.DomainURI?true: ns.Type == "network service" && ns.SourceDomainURI == A.DomainURI && ns.DestinationDomainURI == B.DomainURI && ns.Bandwidth >= 200) && ns.PhysicalDistance
