Composing and Deploying Distributed Applications Using ... - CiteSeerX

Composing and Deploying Distributed Applications Using a Semantic Service Discovery Mechanism James J. Nolan George Mason University Center for Image Analysis Department of Computer Science Fairfax, VA USA

Arun K. Sood George Mason University Center for Image Analysis Department of Computer Science Fairfax, VA USA

Robert Simon George Mason University Center for Image Analysis Department of Computer Science Fairfax, VA USA

Abstract We present semantic approach to discovering, composing and deploying distributed service-based applications. This approach allows users to advertise services on the network in agent form as well as compose meta-agents from existing agents. Agent advertisement is done through a rich RDF representation, enabling a semantic agent discovery layer that sits on top of traditional middleware. We present our prototype system that has been implemented in Java using Jini as the middleware. One hundred multimedia agents providing imagery, geospatial, video, and textual processing are currently implemented in the prototype system. We compare our service discovery mechanism to Jini and discuss results. Keywords: service discovery, service composition, multi-agent systems, distributed multimedia systems.

1

Introduction

Distributed computing potentially offers developers and system users several advantages over serial computing. Some of these advantages include: the ability to take advantage of idle cycles, reduce

clock time, and emulate supercomputers by networking commodity workstations. However, this does not come without cost as distributed computing remains much more difficult than traditional, serial computing. The programming of distributed applications is highly complex, decisions on when and where to parallelize and distribute tasks and data are difficult, and perhaps most important, the development and maintenance of distributed programs can be prohibitively high. As a result, developing programs for parallel and distributed computing still remains a highly specialized skill, effectively reducing the availability of the technology for organizations that find the cost prohibitive. In this paper, we present a semantic middleware layer that abstracts away many of these difficulties, even allowing a user to assemble distributed services into a deployable application using a graphical interface. This semantic layer has been developed to support a larger agent architecture for distributed multimedia computing [1]. The agents on the network advertise themselves through a rich textual description, which is then fully searchable, allowing a client to discover and compose agents into meta-agents to assist in highlevel problem solving.

An abstract class used as the top-level class for processing services A human readable description of the processing service The input data type that an agent requires. A parameter used by the agent. A Service that the agent is dependant upon prior to processing. . . .

Figure 1. Example RDF Schema for Agent Descriptions

Commercial middleware based on CORBA Trader [2] and Jini [3] as well as emerging middleware platforms such as Condor-G [4] and NimrodG [5] all support service description, discovery, and composition. However, these approaches rely on implementation specific, semantically closed service descriptions and search mechanisms with a low resolving power. This environment is unsuitable as the number of services available on the network and their heterogeneity increase in a superlinear fashion. Our approach introduces a querydriven, information retrieval-based approach to service discovery using implementation independent service descriptions. In summary, our approach has a higher resolving power as it performs similarity matching between a user query and the set of agents available on the network.

use the underlying discovery mechanism to compose and deploy service-based applications. Finally, Section 5 concludes and discusses future work.

In Section 2 we present our agent description and discovery mechanism. Section 3 presents our results and compares them to traditional service discovery mechanisms. Section 4 presents our prototype multimedia application, where users can

2.1 Agent Description

2

Agent Description and Discovery

In this section we present the focus of our research: developing an approach to discover agents in a distributed system to support user-driven composition of applications. This research is centered on two areas: 1) how agents describe themselves on the network, and 2) how that information is leveraged to support semantic search. By providing these two mechanisms, we develop the fundamentals necessary to support the composition of distributed services into applications.

In order for agents to interoperate, collaborate, and share information they need a language that allows them to do that. This language is

based on an ontology. An ontology provides a shared and common understanding of a domain that can be communicated between people and across application systems [6]. Well-defined ontologies are critical to both agent-based systems and distributed multimedia processing systems. The automatic processing of information and data in each case requires a machine readable representation of agent communication sessions, data to be processed, and algorithms that will perform the processing. Ontologies are highly domain dependent, as such we have developed an ontology for distributed multimedia systems which supports our approach [7]. The ontology is represented using the Resource Description Framework (RDF) [8] and RDF Schema [9] encoded in the eXtensible Markup Language (XML) [10] and is called I-XML. Others [11] have shown RDF encoded in XML to be a suitable mechanism to represent ontological information in agent-based systems and we build on the approach. A sample of the agent description ontology is shown in Figure 1. This multimedia processing ontology is based on a set of commercial processing suites. Image processing service descriptions are based on the Java Advanced Imaging API [12]. Geospatial processing descriptions are based on the OpenMap API (openmap.bbn.com). Video and audio processing descriptions are based on the Java Media Framework [13]. For textual processing, we use the Grok natural language processing API (grok.sourceforge.net) descriptions. These packages have been integrated into our architecture, and the individual processing components have been identified by using the descriptors above. Our complete ontology for multimedia agents can be found at http://aiga.cs.gmu. edu/ontology/aiga-services.rdfs.

2.2 Agent Discovery Our approach to service representation enables our service discovery and composition mechanism. This is accomplished through the use of a Collaboration Agent as shown in Figure 2. This agent assists agents in the discovery of agents for processing by adding an information retrieval layer

Figure 2. Collaboration Agent Semantic Layer

on top of a middleware implementation such as Jini or CORBA. Specifically, as agents come onto the network the Collaboration Agent stores its rich I-XML description and makes it available for a semantic search. The terms in the description are weighted according to their inverse document frequency. When the Collaboration Agent receives a user query, it parses it into words, compares those words to the keywords and descriptions of all agents on the network using Salton’s [14] similarity algorithm, and returns the most similar agents to the user that exceed a certain threshold. We have chosen the Salton model partly for its simplicity of measuring similarity through keyword matching between a query and the set of agent descriptions. Salton’s vector model information retrieval algorithm works as follows: sim(sj , q) =

Sj • Q | Sj | ∗Q

where sim(sj , q) represents the similarity of service j to query q, S represents a vector of service descriptions, and Q represents a vector of the terms from query q. This formula states that the similarity of the capabilities of an service s(j) to a particular query q can be calculated by taking the cosine of the angle between the vectors S (the terms of the service description) and Q (the terms of the query). The weights of the vectors Sj and Q are determined by taking the product of the

Table 1. Comparison of Service Discovery Mechanism Assumptions Jini and CORBA Assume Discovery based on templates Explicit knowledge of the service’s class Explicit knowledge of the service attribute type and names All services are Java services Boolean match on services

AIGA Assumes Discovery based on query parsing No knowledge of the service’s class No knowledge of the service’s attribute name, assumes types are Strings No underlying implementation assumptions Relevancy-based match on services

normalized frequency and the inverse document frequency: T T F/M T F ∗ log(N/ni ) where T T F is the total term frequency of a term in the service description or query, M T F is the maximum frequency of any term in the service description or query, N is the number of services on the network, and ni is the number of service descriptions in which the term i is found. Our approach is distinct from traditional middleware-bases discovery in several ways as shown in Table 1. First, our approach allows for open-ended queries with little prior knowledge necessary. Second, because our approach requires no prior knowledge, it will grow with the system as new types of services are added. In summary our approach has the advantage of discovering a service even when an incomplete description is provided.

3

Results

To test our service discovery mechanism, we developed an experiment searching for services using our information retrieval approach and compared them to a traditional middleware implementation. The comparison was done with the Jini Lookup Service, however, we expect the results would be similar to a comparison with the CORBA Trader

Figure 3. Time to discover one agent as compared to the number of agents on the network

Figure 4. Mean and normalized time to discover multiple agents as compared to the number of agents on the network

Service. While each have different implementations of service discovery, their underlying concepts are similar. In our case (I-XML), we formulated a query that we knew would result in positive results: ”Is there an agent that can perform the Add function?”. There is no direct way to convert this into a Jini discovery template, so we manually mapped the query into a Jini Entry that would guarantee the same set of results. In this mapping, we created public fields of type String that encompassed the I-XML description of a service, this included ”name”, ”description” and ”keywords” (ar-

Figure 5. Standard deviation discover one agent as compared to the number of agents on the network

ray of Strings) fields. These fields were populated with the appropriate values. We then submitted the query to the I-XML Collaboration Agent and the Entry to the Jini Lookup service and measured results. Our prototype system currently has 100 unique imagery, video, textual, and geospatial processing agents. We tested up to 500 agents by permuting these original 100 to guarantee a new set of 100 unique agents. At each increment of 100, we tested the time to discover one service and multiple services. Each increment of additional services added another service similar to the known service, so at 200 agents, 2 services were returned, 300, 3 services, etc. We ran 1000 queries each time, 10 iterations of 100, and took the average round trip time to return an answer, or service(s) that can assist in answering the query. The experiment was conducted on a network of Sun Ultra 60 workstations with the Solaris 8 Operating System, Java Development Kit 1.3, and Jini version 1.2. The results of our experiments can be seen in Figures 3-5. Figure 3 shows the mean time for discovery of one agent in the Jini and I-XML case. These results show that our approach is scalable, the search time is linear with respect to the number of agents on the network. Figure 4 shows the mean time for multiple agent discovery as well as the normalized time for multiple service discovery

(mean time taken / number of agents discovered). We note that as the number of services returned increases, the difference in discovery time between I-XML and Jini increases. On the other hand, the two lower graphs in Figure 4 show the difference between the normalized times for I-XML and Jini are constant. Figure 5 shows the standard deviation in the time taken to discover one agent in the Jini and I-XML case. In both cases the standard deviation is relatively high, our reasoning behind testing 1000 queries at each increment. These results show that our approach is comparable to traditional service discovery techniques with the added benefit of unstructured query processing. Since our service discovery layer is built on top of Jini, these curves show the additional overhead incurred to provide the capability for service discovery under partial specification.

4

Prototype System

We have implemented the described discovery mechanism as part of a larger agent-based multimedia system on a network of workstations. Jini serves as the middleware, providing a mechanism for services to register on the network, and discover other services on the network through traditional means (e.g., by specifying a class or interfaces type). The Collaboration Agent adds another layer to the lookup and discovery mechanism in Jini, parsing queries and matching them with appropriate agents. Users take the results from this agent and compose larger meta-agents to be deployed on the network. The system is used to deploy agents to process large multimedia streams, remotely sensed images on the order of 2 GB, video streams of several GB, audio streams of several MB, and textual reports on the order of several hundred KB. We have created over 100 master multimedia agents from several packages. For image processing, Sun’s Java Advanced Imaging API provides 96 low-level image processing functions. For video and audio processing, we use the Java Media Framework represented in agent form. For textual processing, we use the Grok natural language processing API. For processing geospatial data sets,

Figure 6. Client Screenshot

we use BBN’s OpenMap API which provides lowlevel geospatial analysis functions. We have built wrappers that translate queries into the I-XML representation and search for service that perform the processing. A screenshot of the client portion system can be seen in Figure 6. In the upper left quadrant of the screenshot is the Agent List Panel. This panel gives a list of all agents available on the network, or a user can search for a specific agent by typing in a textual description or just drag and drop the agent of interest. In the lower left quadrant is the Knowledge List Panel. This panel allows the user to search the knowledge repository and see how and what agents have been used to solve particular queries. Users can view the agent composition, even down to the detail of the parameters and their setting for each agent composition. In the upper right quadrant of the screenshot is the Agent Canvas.

From the Agent List Panel, users can drag agents onto this canvas, link them together, and create a new meta-agent for processing. Once on this panel, the attributes of each agent can be modified for a particular task, then deployed on the network as a new meta-agent, or agent of agents. This agent now behaves like any other agent on the network, and may be discovered and used by other clients or agents.

5

Conclusions and Future Work

We have presented an approach for discovering, composing, and deploying agent-based multimedia services to solve query-driven problems. The approach is query driven, allowing a user to pose a query and compose a set of distributed agents to assist in obtaining an answer. We introduce a novel approach for discovering distributed agents on the network through the use of a rich

agent description language combined with traditional information retrieval techniques. Our approach does add additional overhead as compared to traditional middleware discovery mechanisms, however the benefit is the additional flexibility allowing for discovery through unstructured queries. For future work, we plan to expand this approach in two directions. First, we are developing an approach to decentralize the repository of agent descriptions over a network of peers. This network will be organized by service type into a hierarchy, providing a more efficient search space. Second, we plan to develop a dynamic scheduling mechanism to optimize the processing graph of agents created by the user. This mechanism will optimize the schedule based on the number and state of computing resources available on the network.

References [1] Robert Simon, James J. Nolan, and Arun K. Sood. A light-weight agent approach for collaborative multimedia systems. Information Sciences, 140(1/2):53–84, January 2002. [2] OMG. The trading object service specification, version 1.0. Technical report, The Object Management Group, May 2000. [3] Ken Arnold, Brian O’Sullivan, Robert W. Scheifler, Jim Waldo, and Ann Wollrath. The Jini(TM) Specification. Addison-Wesley, Reading, MA, 1999. [4] J. Frey, T. Tannenbaum, M. Livny, I. Foster, and S Tuecke. Condor-g: A computation management agent for multi-institutional grids. In IEEE Conference on High Performance Distributed Computing, 2001. [5] D. Abramson, Giddy J., and L. Kotler. High performance parametric modeling with nimrod/g: Killer application for the global grid? In International Parallel and Distributed Processing Symposium, 2000. [6] E. Mena, V. Kashyap, A. Illarramendi, and A. Sheth. Formal Ontology in Information

Systems, chapter Domain Specific Ontologies for Semantic Information Brokering on the Global Information Infrastructure. IOS Press, Amsterdam, Netherlands, 1998. [7] James J. Nolan, Robert Simon, and Arun K. Sood. Developing an ontology and acl in an agent-based gis. In Ontologies in Agent Systems (OAS 2001), pages 99–101, Montreal, Canada, May 2001. [8] World-wide Web Consortium, Cambridge, MA. Resource Description Framework (RDF) Model and Syntax Specification, February 1999. [9] World-wide Web Consortium, Cambridge, MA. Resource Description Framework (RDF) Schema Specification 1.0, March 2000. [10] World-wide Web Consortium, Cambridge, MA. Extensible Markup Language (XML) 1.0, February 1998. [11] Steffen Staab, Michael Erdmann, Alexander Mdche, and Stefan Decker. An extensible approach for modeling ontologies in rdf(s). In Proceedings of the ECDL-2000 Workshop Semantic Web: Models, Architectures and Management, September 2000. [12] Sun Microsystems. Java Advanced Imaging Application Programmers Interface, 1999. [13] Sun Microsystems. Java Media Framework Application Programmers Interface, 2001. [14] G. Salton. The SMART Retrieval System. Prentice Hall, Englewood Cliffs, NJ, 1971.