The OSIRIS Process Support Middleware and the ISIS ... - CiteSeerX

The OSIRIS Process Support Middleware and the ISIS Process-Based Digital Library Application Gert Brettlecker1 , Michael Springmann1 , Laura Voicu2 , Heiko Schuldt1,2 , Hans-J¨org Schek1 1

University for Health Sciences, Medical Informatics and Technology (UMIT), Austria

Email: {gert.brettlecker, michael.springmann, heiko.schuldt, hans-joerg.schek}@umit.at 2

University of Basel, Switzerland

Email: {laura.voicu, heiko.schuldt}@unibas.ch

Management Summary This paper presents the OSIRIS middleware (Open Service Infrastructure for Reliable and Integrated process Support) which allows combining different distributed services into processes, thereby implementing flexible digital library applications. In OSIRIS, we distinguish between system services and application services. System services are used internally for coordinating the execution of processes in a distributed way, without relying on a central execution engine/scheduler. In addition, the distributed architecture of OSIRIS allows for reliable peer-to-peer process management and sophisticated failure handling according to the transactional process model and is able to balance the load among several providers of the same service. For application services, we distinguish between loosely coupled and tightly coupled services. Usually, a part of the distributed OSIRIS middleware (called OSIRIS layer) runs on each host providing application services. These application services are considered as tightly coupled since, in case information on their transactional properties like compensation or re-invocation (retriability) is available, dedicated transactional guarantees for processes can be provided. Loosely-coupled application services are those that have to be called remotely, without a local ORISIS layer available. The integration/invocation is done via WSDL for service description and SOAP for invocation. The core of OSIRIS has been developed at ETH Z¨ urich. Currently, it is being extended (e.g., for processing continuous streams of data) jointly at UMIT and at the University of Basel. There are two versions of OSIRIS: one is implemented in C++ and runs on Microsoft platforms, the other version is implemented in Java. Second, this paper also introduces the ISIS application (Interactive SImilarity Search) that has been built on top of the OSIRIS platform. ISIS consists of a set of pre-defined processes and several application services (like feature extraction, index management, index access, relevance feedback, etc.) which are used within these processes. The paper presents how the ISIS application has been built and the application services it relies on. Finally, it also gives detailed information on how new digital library applications can be built on the basis of existing services.

1

1

What is OSIRIS?

OSIRIS (Open Service Infrastructure for Reliable and Integrated process Support) [19, 20] is a platform that allows combining different distributed services into processes. The OSIRIS platform itself does not provide any application functionality but, by combining specialized application services, supports the definition and reliable execution of dedicated processes (this is also known as “programming-in-the-large”). When different specialized digital library application services are made available to the OSIRIS platform, users can define and run powerful digital library processes by making use of these services. OSIRIS processes themselves are wrapped by a service interface. Therefore, a process can be invoked just like any other service (and used in other processes as well). Following the model of transactional processes [18], processes in OSIRIS contain two orders on their constituent services: a (partial) precedence order specifies regular execution while the precedence order is defined for failure handling purposes (alternative executions). Data flow between services of a process can be defined independently of control flow. Activities in a process are invocations of application services. Ideally, the transactional behavior of each application service is known. This transactional behavior includes information on compensation (how can the effects of a service execution be semantically undone; this is needed for compensation purposes in case a failure in a process execution exists) and on whether a failed service can be re-invoked (retriability). In addition to transactional guarantees and reliability, OSIRIS focuses on scalability of process execution. The decentralized peer-to-peer approach for process execution in OSIRIS, which is realized by sophisticated replication mechanisms for control flow dependencies, avoids any single point of failure during process execution and provides a high degree of scalability. Peer-to-peer process execution also incorporates sophisticated load balancing in order to distribute process load among available, suitable peers. Finally, OSIRIS is equipped with the O’GRAPE (OSIRIS GRAphical Process Editor) [26] user interface for process definition. It allows for easy creation of process descriptions without programming skills. In addition, O’GRAPE supports the integration of existing application services by leveraging existing Web service standards like SOAP and WSDL.

2

What is ISIS?

ISIS stands for Interactice SImilarity Search and is a prototype application for information retrieval in multimedia collections built at ETH Z¨ urich [15]. It supports content-based retrieval of images, audio and video content, and the combination of any of these media types with sophisticated text retrieval [21]. The screenshots in Figure 1 show a combined search for flowers. Starting point is the query frontend, depicted in Figure 1(a), where keyword and reference image can be specified. Figure 1(b) then shows the query results for this combined query. One of the main considerations in designing ISIS was to ensure high scalability and flexibility. Therefore, instead of implementing one monolithic application, ISIS consists of a set of specialized application services for similarity search which are combined by the OSIRIS middleware. The ISIS services can be easily distributed among several nodes in a network [23]. The query presented in Figure 1 is therefore implemented as a process. It is important to note that the process specification just contains the details of all application services it encompasses (WSDL description) and the orders within the process. The actual service providers where the service is invoked are determined at run-time. Therefore, information on the location of these providers is not part of the process

2

(a) Query

(b) Result

Figure 1: Combined Text and Images Similarity Search description. Hence, each step of the process can be executed by any node providing the required service. After issuing the query a first time, a user can refine and re-issue her query. The query process (including user feedback) consists of the steps Query Reformulation (based on relevance feedback the user has issued), Query Execution (index access), and Result Filtering (which may again take user feedback into account). In Figure 2, this process is shown in the design view of the O’GRAPE tool. Any content-based retrieval system is commonly exposed to heavy load under two distinct circumstances: (1) Content-based queries, e.g., for similar images, are based on comparison of features of the object like color histograms or texture. Because these features form high-dimensional retrieval spaces, determining the similarity for ranking the results is computationally expensive. One approach to reduce the system load would use efficient data structures as indexes, e.g., as described in [25]. Another approach would replicate data on several nodes to serve more requests in parallel, employ load-balancing, or try to handle parts of the request on several nodes [3]. ISIS follows both approaches. (2) While replication helps to cope with query load, it increases complexity of modifying a collection by inserting, deleting, or updating objects since the updates of all indexes have to be coordinated to ensure consistency. In ISIS, this is done by appropriate system processes, i.e., processes that have been designed by system administrator and which run automatically to guarantee consistency over several replicas of the index. The extraction of features itself can be a time-consuming task, therefore monitoring constantly changing collections and providing access can be challenging as well. If the insertion of multimedia objects can be divided in several sub-tasks and those can be executed on different nodes while using an infrastructure ensuring correctness of the distributed execution, this can improve the performance significantly [24]. Figure 3 shows a simplified version of the process used in ISIS to insert new objects which follows this idea of distributing effort among different nodes in the system. The first service (each service is illustrated by a different shape) stores the multimedia object, i.e., the location and available meta information. Depending on the

3

Figure 2: Design View of an ISIS Search Process (Encompassing Relevance Feedback) in O’GRAPE media type further information is extracted. In case of a web document, the object will not only contain an image, but also some text surrounding this image on the page. Later on, this text is used to determine textual descriptions related to the image. Independent of the image context, the feature extraction service uses raw pixel information of the image. Finally, the store features service hands all derived object information over to a metadata service, which makes it available for indexing and search in a suitable way. Worth mentioning in this context, ISIS is efficiently searching and maintaining a collection of more than 600.000 images used within the ETHWorld project, the virtual campus of ETH Z¨ urich. The images have been extracted from websites of the university and all its institutes.

3

OSIRIS Architecture

A process encompasses a set of activities. Each activity corresponds to an invocation of a service. This can either be a basic (web) service or again a process. On the set of activities, two different orders are defined. The partial precedence order specifies the regular order in which activities are executed (i.e., in which the associated services are invoked). An activity can only be executed when all its pre-ordered activities have successfully finished and when the conditions on its execution are fulfilled. Since the precedence order is a partial order, intra-process parallelization can be realized by parallel branches (fork/join). Data flow is defined as mappings from a data space in the process instance, called whiteboard, to the service request parameters, and back from the response values to the whiteboard after execution. In addition to the precedence order, the preference order specifies alternative executions that can be chosen when an execution path fails. OSIRIS follows the model of transactional processes [18] that aims at providing transactional execution guarantees for processes. Essentially, it considers the termination semantics of individual activities. Each activity is either compensatable, retriable, or pivot. The effects of compensatable activities can be semantically undone after the corresponding service invocation has successfully 4

Store Object

Separate Media Types

Extract Features

Analyze Context

Store Features

Figure 3: Sample Process Insert Multimedia Object returned. An activity is pivot when it is not compensatable. Retriable activities are guaranteed to terminate correctly, even if they have to be invoked repeatedly. Based on these termination guarantees and the precedence and preference orders, it can be verified already at built-time whether or not a process can be executed correctly. This is the case when all failures that may occur during execution can be resolved either by complete compensation or by partial compensation until a point is reached from where an alternative execution path can be followed. The architecture of OSIRIS is driven by the goal of implementing a true peer-to-peer process execution engine. Therefore, process execution involves only those machines of the network that offer a service required by the process definition. Conceptually, OSIRIS is a service-oriented architecture (SoA), where service inter-communication is done by passing messages through queues or pipelines which are part of the OSIRS middleware. There is no central peer that controls the execution of process instances. As a consequence, distributed process execution necessitates efficient metadata management and replication in order to be able to make local decisions on how a process instance has to be routed. OSIRIS distinguishes between application-specific services, called application services, and services providing general purpose functions needed by all applications, called system services. Examples of OSIRIS system services are storage services, replication services, and process execution services, to name just the most important. An example of an application service is the feature extraction service as it is used in ISIS, where a specific color histogram is derived from a given input image. While system services are part of the OSIRIS midleware, application services can be any callable services provided by third parties. For system services, there is an additional distinction on where they run. Firstly, each peer of the network providing application services runs an OSIRIS layer that provides local functionality for process instance navigation and routing. This is the standard way of integrating application services into OSIRIS (such application services are called tightly coupled). Secondly, OSIRIS runs a number of additional system services (not deployed at all service providers) which contain repositories for metadata management. These repositories hold information about process definitions, subscription lists, service providers, load information, etc. In addition, a dedicated system service exists for 5

Process Invocation

Process Execution Service Invocation Metadata Replication

Global Repositories

OSIRIS layer

Services

Service Provider

Load Repository

C E F P R

Registry

… Process Programs

Services

…

OSIRIS layer

R

… OSIRIS layer OSIRIS layer

P

Services

Services

Figure 4: Overview of P2P Process Execution in OSIRIS the remote invocation of Web services (loosely coupled application services) by means of SOAP. The lcoal OSIRIS layers that actually execute processes should not have to query meta information from the global repositories. Rather, a push mechanism replicates those parts of meta information towards the local OSIRIS layers that they require to fulfill their tasks. For instance, if a peer is involved in executing process P1 , the definition and any changes to that definition are pushed from the process repository to the local OSIRIS layer. This reduces dramatically the probability that this information have to be requested at runtime. From another perspective, the local OSIRIS layers perform their tasks based mainly on local versions of the global meta information. With this information, the process execution is decoupled from the metadata management and allows for efficient distributed process execution. Moreover, it is often sufficient to hold approximate versions of metadata: for instance, load information of peers is required by the local OSIRIS layer to balance the process workload among the available service providers. Since this is an optimization, it is sufficient if local load information is only approximately accurate. Note that this would not lead to incorrect execution. In worst case, the load distribution among all service providers is just not optimal. On the other hand, changes on process definitions have to be propagated immediately. However, even this information can be replicated in a lazy way, since process instances are explicitly bound to a dedicated version of a process definition. For application services, we distinguish between loosely coupled and tightly coupled services. Each node in the system providing a service which is tightly coupled to the OSIRIS network runs a local OSIRIS layer that is responsible for the invocation of requested services, execution and routing of process instances, and failure handling. For all application services and their service providers which are only loosely coupled, i.e., for which no local OSIRIS layer exists, there exists a

6

dedicated system service (that can be deployed in multiple instances in the OSIRIS network) which remotely invokes a service. A requirement for the invocation of a loosely coupled service from an OSIRIS process is that the service is implemented as a web service, thus provides a WSDL service description and is callable by using SOAP. Furthermore, existing loosely coupled services have to be registered in order to be accessible by OSIRIS. For this purpose, a global repository called UDDI registry is collecting available service descriptions. Figure 3 illustrates the distributed process execution in OSIRIS. System services are depicted in the middle (global repositories) and, in addition, each node in the network is equipped with a local OSIRIS layer (i.e., all application services –which are illustrated by different shapes– are tightly coupled). It can be seen that process execution (solid, bold arcs) is decentralied and that all services are invoked locally (bold dashed arcs). Meta data replication from global repositories to local OSIRIS layers is decoupled from process execution (unbold dashed arcs). OSIRIS comes together with O’GRAPE [26], a Java-based process modeling tool that supports a designer in defining and validating transactional processes.

3.1

System Services and their Functionality

Among the system services, the global repositories are important since they store all necessary information for process execution. The OSIRIS layer of each node offers a replication service with is in charge of replicating sufficient information in order to allow for decentralized and decoupled process execution. Additional system services support tasks like process management, replication, messaging, routing, and load balancing. 3.1.1

Global Repository Services

Process Repository holds the global definitions of all processes. Process definitions are decomposed into pairs of subsequent activities within the control flow, called execution units. Execution units are tuples containing process ID, ID of current activity, ID of subsequent activity/activities in control flow, and event type. On termination of the current activity, the OSIRIS layer produces an event to indicate the status of current activity execution. For example, if the current activity has not terminated gracefully, an error event is produced. The event type allows for different control flows for correct and failed execution. Additionally, one activity may have multiple direct successors in the control flow for the same event type, which corresponds to a split in the process execution. Only locally needed execution units are replicated. In other words, only execution units with current activities a provider is able to perform are replicated from this repository. Subscription Repository records a list of all available (tightly coupled) services, offered by all providers in the OSIRIS network. Providers in the OSIRIS network only need replicated information about services which are subsequent activities of execution units they are involved in. Load Repository is storing information of the current load situation of providers in the OSIRIS network. This information is replicated in a similar way as information from the subscription repository. Additionally, freshness parameters on subscriptions are used to avoid unnecessary propagation of minor load updates. UDDI Registry is in charge of keeping a registry of loosely coupled services together with their WSDL descriptions, since loosely coupled services do not register themselves at the subscription

7

Upload of sample process

C C

Global Repositories

Process Repository

B B A

Load Repository

A B

Service Repository

K

C ...

...

...

...

Publish OSIRIS Layer

Services

Services

OSIRIS Layer

Local OSIRIS Layers

OSIRIS Layer

OSIRIS Layer

...

Services

Services Service Providers

A

B

C

K

Figure 5: OSIRIS Architecture repository. This registry is necessary to keep the OSIRIS infrastructure informed about available loosely coupled services. All these repositories do not require a centralized service implementation. If the number of nodes in the OSIRIS network is increasing, additional repositories/services of same kind can reduce the load (i.e., by applying horizontal partitioning). Figure 5 illustrates the OSIRIS architecture. It shows the upload of a sample process to the process repository. After the upload, the process description is divided into execution units. These execution units are replicated to the local OSIRIS layers of nodes providing participating tightly coupled services. Also necessary load and available service information is replicated from load and subscription repositories. Finally, after replication, enough process and service meta information is locally available to allow for peer-to-peer process execution. 3.1.2

Functionality of the Local OSIRIS Layer

Messaging The local OSIRIS layer allows for a reliable exchange of messages between all nodes of the OSIRIS network. In addition to TCP, the peer-to-peer messaging framework JXTA [12] has been incorporated as additional transport layer to abstract from networking obstacles, like inconsistent addressing and communication problems due to firewalls or heterogenous network environments. Horus The Horus component is the agent of a service provider and is responsible for all external communication. The Horus is responsible for activation and deactivation of local services. External communication is done via two pipelines for incoming and outgoing messages, respectively. Pluggable handlers are applied to the pipeline and offer specific processing support for messages. Process execution is done by exchanging process messages, which are processed by a dedicated process handler. Process Handler The process handler is plugged in the incoming message pipeline of the Horus (see Fig. 6) and executes local process activities (i.e., locally invokes a service) based on information in process messages and replicated execution units of the current process. After local 8

Global Repositories Local OSIRIS Layer

Services

Replication Manager Horus

Tightly Coupled Services

Process Handler Routing Handler

Loosely Coupled Services

Transport Layer

Output Messages

Input Messages

Figure 6: The local OSIRIS Layer activity execution, the process message is updated (e.g., with results of a service invocation) and forwarded to all subsequent activities (providers). Replication Manager The local replication manager is keeping local replicas of the global repository consistent according to configured subscription settings. Replication management is based on publish/subscribe techniques. The primary copy resides at the global repository. Each local OSIRIS layer as client that needs replicas has to subscribe for the selected information. As a result of this subscription, the repository publishes the current information. Whenever the primary copy changes, updates are published to all subscribed clients. Additionally to classical publishsubscribe, in OSIRIS a subscriber is also able to insert or update new information in the local repository and these updates are propagated back to the global repository. Update messages only contain changed parts of the subscribed information to reduce replication overhead. Additionally, freshness predicates of subscriptions may skip updates of marginal changes. As long as the replica is sufficiently close (to satisfy the freshness predicate) to the primary copy, no update is needed. Routing Handler The routing handler is a handler in the outgoing message pipeline of the Horus and responsible for selection of an appropriate provider as destination of outgoing messages. If multiple providers are able to process the corresponding message, the best suitable according to current load (the locally replicated approximation) is selected (load balancing). SOAP Component The SOAP component is an OSIRIS system service managed by the Horus, which is in charge of the invocation of SOAP calls for the interaction with loosely coupled services. This component wraps an OSIRIS service call into a standardized Web service invocation. Ideally, the loosely coupled service instance is hosted at the same provider. In this case, load balancing and service management of OSIRIS also works for loosely coupled application services. However, the invocation is also possible for remote services. Web Component The Web component is a local OSIRIS system service managed by the Horus, which is in charge of making OSIRIS functionality accessible by standard Internet protocols like HTTP/HTML and offers an HTML interface for user interaction. Process Component The process component is a local OSIRIS system service managed by the Horus, which is in charge of starting the execution of a process within the OSIRIS network. The process component first looks up the necessary activities within the process control flow and 9

creates the required process messages which are delivered to appropriate providers. The process component is also responsible for feeding the process messages with the correct input parameters and for collecting the process result parameter(s). The process result is returned to the calling application. From the calling application’s point of view, a process execution is just a service invocation. This mechanism allows also for the easy nesting of processes as services within other processes.

3.2

Application Services and their Integration

Based on the system services presented in the previous section, a digital library architect needs to integrate user-defined application-specific services within digital library processes. These application specific services are called application services. There are two possibilities to integrate application services into the OSIRIS infrastructure. Firstly, they can be integrated as tightly coupled services, which offers more functionality for process management. Secondly, they can be integrated as loosely coupled services which does not require a local OSIRIS layer at the provider’s host. 3.2.1

Tightly Coupled Services

Tightly coupled service implementations follow a proprietary OSIRIS service specification and offer additional interfaces to allow controlling the life-cycle of service instances and to retrieve information about the current state of the service instance (e.g., the load situation, reliability and correctness of service invocations) which is needed for load balancing and failure handling of processes. Currently evolving Web standards coming from the Grid community, like WSRF [27], address these additional service specification demands, e.g. life-cycle and resource management. Future versions of OSIRIS will consider these standards as replacement for proprietary formats. Tightly coupled application services behave in the same way as system services and can be directly called as activities within OSIRIS processes. 3.2.2

Loosely Coupled Services

Loosely coupled services are the fastest and easiest way to integrate application-specific services into OSIRIS-based processes. Loosely coupled services are built upon the existing Web service standards, SOAP and WSDL. Therefore, they are easy and fast to implement (or even already in place). However, due to the loose coupling, these services cannot benefit from load balancing, and advanced failure handling. In addition, a proxy component, called SOAP component, is needed which is in charge of calling the loosely coupled SOAP service.

3.3

OSIRIS: Implementation Details

In the following, we describe briefly implementation and configuration details of the two available OSIRIS versions in C++ and Java. In particular, we focus on the prerequisites needed to host an OSIRIS implementation and things to take care to get the system up and running. Furthermore, we describe pros and cons of the both versions.

10

3.3.1

The C++ Version of OSIRIS/ISIS

The C++ version of OSIRIS including ISIS contains more than 300.000 lines of code and relies on using Microsoft Visual C++ Studio [14] as programming environment. Another prerequisite is the database used, since all the components use a local database layer. This is hidden by an abstraction layer and can be therefore depending upon requirements and local installation. Currently supported databases are Microsoft SQL Server 2000 [13] and MySQL [17]. The OSIRIS project provides scripts for the creation and configuration of the database. Once the project is available (for example, after downloading it from a CVS repository), it can be compiled using Microsoft Visual C++. The compilation of an entire project implies also that the components are, at the same time, loaded into the database. The configuration can be provided via local configuration folders or by installing a configuration agent. This is a parameter that can be set in the startup file. For this purpose, the configuration agent needs its own database tables. The project also provides scripts that create and fill these tables with default values. Another important component is the software agent, for which the tables need also the be created in advance and filled in with some default values. If the local configuration files are used, after the initialization of Horus, a web interface can be used for the configuration and administration of the configuration agent. Moreover, the C++ version has several features that have not yet been ported to the Java version, such as concurrency control of processes, transactional properties of processes, detailed process monitoring, and all the ISIS components. Current activities in the context of the C++ version include the definition of a benchmark for process-based applications and, on the basis of this benchmark, the evaluation of OSIRIS C++ and IBM WebSphere, especially in terms of the scalability characteristics of both systems. 3.3.2

The Java Version of OSIRIS

The Java version of OSIRIS has been developed at UMIT and is based on Java 2 SE 1.3, which is available for a variety of different platforms like Windows, Linux, and also on mobile devices like Pocket PCs. Therefore, the only prerequisite of the Java version is to have an J2SE 1.3 compliant Java virtual machine available at the desired target system. Currently, Eclipse is used as programming environment, but every other Java IDE would be fine too. Right now, the ISIS digital library is not implemented within this version. The Java version contains about 20.000 lines of code, without counting third-party, open-source libraries, like Axis, DOM4J, or Jetty. The Java version supports a variety of databases, like Microsoft SQL Server [13], MySQL [17], or IBM Cloudscape [6], which offer an JDBC driver. In particular, it is beneficial to use IBM Cloudscape, recently available open-source as Derby [7], within the Java infrastructure. Since Cloudscape is fully programmed in Java it can run in embedded mode within the same virtual machine as OSIRIS, so there is no need to install a separate database server on a Java node. Furthermore, Cloudscape has a very small footprint (about 2 MB) and still offering transactions and sufficiently supporting SQL92. For this reason, Cloudscape is even able to run on a Pocket PC with acceptable performance and still performing about as good as MySQL on a standard Windows PC. The configuration of the Java version is currently only done in local configuration folders. The creation of necessary database schemas is done automatically during the first startup. The Java version supports process execution similar to the C++ version plus additional data stream management [5, 4]. This offers the unique support of new emerging information management applications demanding both process support and continuous processing of stream data, e.g., the

11

Operation Object.Create Object.AddLocation Object.GetLocation Predicate.Create Predicate.AddRelation Predicate.AddObject Predicate.GetObjects Predicate.Remove ...

Parameter OType OID, URI, Role OID Name Parent / child predicates OID, Predicate Path Predicate Path, Recurse? Predicate Path or ID ...

Result an new OID all locations and roles of the object Predicate ID, Predicate Path

OIDs of Objects ...

Table 1: Operations of the Meta Database Service incorporation of continuous telemonitoring into healthcare applications requires both issues. Due to the remarkable support for Java in the field of Web services and application servers, we have integrated a Web service framework within the Web component of the Java version. Technically, this is done by integration of the embedded open-source Java application server Jetty [11] and Axis [1] running as part of the component. This framework allows to host Java Web services within our OSIRIS infrastructure and control their life-cycle and load situation, which offers a higher degree of control for loosely coupled services. Additionally, Jetty supports Servlets [9] and JavaServer Pages (JSP) [10], which allow for easy standardized generation of web user interfaces. Based on Axis, Horus is extended to offer a Web service for interaction with external systems by using the standardized SOAP protocol. This enables a variety of external systems to trigger OSIRIS processes. Also in this case, we support mobile devices, like Pocket PCs, due to the small footprint required by Jetty and Axis.

4

ISIS: A Digital Library Build On Top of OSIRIS

OSIRIS has been used to build a digital library system at ETH Z¨ urich for two distinct applications: 1. The prototype system ISIS (Interactive SImilarity Search) enables content-based retrieval of images, audio, and video. 2. Within the project ETHWorld the ISIS system has been used to build a scalable search engine for digital information available in the information space of ETH Z¨ urich. The digital library system has been implemented completely as application services based on OSIRIS. In the following, we present some application services that form the ISIS system.

4.1

Collection Management

Meta Database Every object contained in the digital library is assigned a unique identifer, the OID, within the collection. The Meta Database component maintains the association of OID with objects. Each object is associated with a object type (OType), which can be image, audio, video, video sequence, or text. Every object can be associated with several locations, e.g., the original URL from which the object has been downloaded and the filename of the local copy, or the address of a thumbnail image. 12

Operation Request

Parameter Local filename

Result MIME Type and the content of the file

Table 2: Operations of the Storage Service It is also possible to store arbitrary additional properties like size in bytes of the file or whether the object is copyrighted. The Meta Database service also maintains the structure of a collection, that is, the objects can be assigned to a taxonomy of categories (Predicate). Some selected operations provided by this service are summarized in Table 1. Storage The Storage service is able to deliver the file content of the object and monitor changes in a directory of the file system. In case a change is detected, a process is started in OSIRIS. For instance, if a new file is inserted, Storage triggers the execution by sending a message to the OSIRIS process system service, which will run a process like the one depicted in Figure 3. Storage heavily uses the underlying OSIRIS middleware also for other tasks, e.g., to transfer objects from one node to another. It can be configured using the configuration agent, e.g., which process should be started when an object is inserted, which process should be started if an object is deleted, which directories should be monitored. Therefore the only operation provided directly to the user is to retrieve a requested file from disk (see Table 2). This is, again, performed using a system service of OSIRIS, the Web component. Web Crawler The Web Crawler service periodically monitors websites for modifications. For faster access, a local copy is stored and updated in case of modifications. If a new link is found and this link is qualified for inclusion, the crawler follows this link and inserts the new object. Similar to the Storage service, processes are triggered according to modification events. The only difference is that here it is also necessary to specify at what time and in which intervals the crawler should visit the websites, inclusion and exclusion patterns need to be defined, and so on. Hence, more administration options are provided to the user (see Table 3).

4.2

Search Interface and Query Processing

ISIS provides a web interface to the user to issue searches. The design is stored in HTML template files extended by a tag library using XSLT [28]. The OSIRIS Web component automatically applies those templates to visualize the results or error messages. This makes the customization of a digital library an easy task. Application services of ISIS that have been added to the OSIRIS system services are: Session Management The Session Management service provides the basic handling of the Operation Request Start Stop ShowProgress ...

Parameter URI or local filename

Result MIME Type and the content of the file

...

Webpage showing progress ...

Table 3: Operations of the Web Crawler Service

13

Operation Action

Parameter Query parameters

Result Query result

Table 4: Operations of the Session Management Service user interactions (see Table 4). Each query issued by the user is executed as a new process as displayed in Figure 2. The result of the process is handed over to the Web component, which will layout the result based on the templates. Relevance Feedback The Relevance Feedback component evaluates the feedback that a user can issue for previously executed queries. As identified in the search process, this may take place in two execution steps: (1) Before the query is executed, it can be reformulated, e.g., by adding more reference objects to the query or modifying their weights. (2) After the query has been executed, it can filter out some unwanted objects or rearrange the position within the result. Indexing The Indexing service is used to answer queries efficiently. It is therefore invoked as the middle step between the relevance feedback related steps within the query process in Figure 2. Several types of indexes are supported. The first two, attribute and set indexes, use relational databases. The difference between the two is that attribute indexes are build on one particular value, e.g., the size of an object in number of bytes. This can be handled easily by the functionality that common RDBMS provide. Set indexes, in contrast, can contain multiple values, e.g., used to provide text retrieval in vector space model, for which term frequencies and document frequencies are needed to measure relevance. This can also be handled with the support of RDMS like described in [8]. Query processing of these two indexes is optimized by the used database. For high-dimensional feature spaces, e.g., if color histograms of images are used, additional indexes are stored outside the RDBMS [25, 16]. The Indexing component optimizes and executes queries on these index types. When complex queries are used, it is also necessary to aggregate the results delivered by the individual indexes. This task is also performed by this component, since this allows for further performance optimizations [2].

4.3

Feature Extraction

Content-based information retrieval for multimedia data frequently employs features that have been extracted from the objects prior to search to determine the similarity of objects. Depending on the type of features to extract, this can be rather time-consuming. For instance, the analysis of audio files or shot detection in video material can require a lot of computation, whereas determining the height and width of an image can be performed rather quickly. There are many different features supported in ISIS, which are offered by a couple of different Operation InputFeedback

Parameter Query parameters, Feedback parameters

FilterResults

Query result, hints for filtering

Result Reformulated query, hints for filtering step Modified query result

Table 5: Operations of the Relevance Feedback Service

14

Operation Create Update Query

Parameter Index name, type Index name Query parameters

Result

Query result

Table 6: Operations of the Index Service services. To ease the handling, feature extractors that are implemented as tightly coupled services share a common interface for extraction and for registering to the ISIS system. In order to support several types of features, an administration tool exists in ISIS that automatically generates a subprocess within the activity “Extract Features” for the process in Fig. 3, where the administrator simply has to select which features should be extracted and indexed. The common interfaces for feature extractors provide the operations summarized in Table 7 Maintaining the registration information of the features is handled by the Meta Database service. Several services are allowed to provide a feature with the same name. At registration time, it is also possible to specify the storage format, dependencies on other features, and a preferred distance measure for similarity evaluation. The following feature extraction services exist and are currently in use in the ISIS system. Meta Database The Meta Database also provides information about objects, although this is not necessarily extracted by this component. It provides the image height and width, file size in bytes. It can also handle arbitrary additional properties. These values can be access efficiently through the support of an attribute index of the Indexing component. Feature Extractor The general Feature Extractor component provides the following features: i.) Color histogram in RGB color space, ii.) Color moments in HCL and Lab color space, and iii.) Gabor texture moments on luminance. All of these can be applied on the image as a whole, 3 × 3 overlapping rectangles, or “5 fuzzy regions”, which basically is one region starting in each corner and a ellipse in the image center where a membership function defines for each pixel of the image its influence on the feature of the region. The image can be extracted from a movie, e.g., a key frame. All these features are stored and processed as high-dimensional feature vectors. Face Detector The Face Detector offers several algorithms to analyze images to identify regions, that contain a face. It returns the number of face found and a bounding box for each. Audio Feature Extractor The Audio Feature Extractor uses the MARSYAS [22] library to extract the features “beat” and “pitch”, which are again, feature vectors. Hypertext Feature Extractor The Hypertext Feature Extractor analyzes HTML documents for embedded links. It also identifies images and assigns the surrounding text to it. The result is not directly used for retrieval. Term Frequency Extractor The Term Frequency Extractor is applied to plain text, in particular to the output of the Hypertext Feature Extractor. It returns a list of all found terms and the number of their occurrences. This feature is stored as a set index and used for boolean and Operation Register ExtractFeature

Parameter Supported object type, feature name. . . OID or data, feature name

Result Feature class, value

Table 7: Operations of the Feature Extraction Service

15

vector space retrieval. The sample ISIS process illustrated earlier in this paper (see Fig. 3) shows all necessary activities for inserting a new multimedia object into the ISIS digital library. Activities, like Extract Features, may itself be complex processes again using a composition of services, e.g., a color histogram service, a face detector service, and a segmentation service. Activities may be system services, e.g., storing some meta-data tuples in a database, or storing an image in a file system, but also application specific services like a face detector. In many cases, application specific software components already exist and it is beneficial to integrate these proven components into new digital library applications.

5

A Guide To Building Applications Based On OSIRIS

In what follows, we introduce how to generate a user-defined digital library application based on our OSIRIS infrastructure. We assume OSIRIS is already installed and properly configured. ISIS as an example application on top of OSIRIS is used for illustration. Generally, the building of new digital library applications based on OSIRIS consists of three major steps. 1. Integration of Application Services: Integration of existing software components someone wants to use within the new application into the OSIRIS infrastructure. 2. Definition of Processes: Design of custom processes building up the DL application. 3. Invocation of Processes: Execution of custom processes within the OSIRIS infrastructure.

5.1

Application Services: What is Needed for Inclusion in Processes

Existing software components may be integrated as tightly coupled or loosely coupled application services into the OSIRIS infrastructure. Tightly coupled services register themselves to the service repository of OSIRIS and are immediately usable by the infrastructure. In order to integrate an existing software component as tightly coupled service, the existing code has to be adapted according to the proprietary OSIRIS service specification and finally the compiled service has to be linked with the OSIRIS libraries. The effort to integrate existing software components as a tightly coupled service into OSIRIS is comparable –if someone is familiar to the OSIRIS service specification– to creating a SOAP service with an Web service framework like Axis [1]. Since many people are already familiar with Web service frameworks, the creation of a loosely coupled service is the faster way. Loosely coupled services are Web services called by the OSIRIS infrastructure. In order to let the infrastructure know about the existence of the service, the application designer has to take care that the service description is registered in the directory for loosely coupled services, the UDDI registry. Also the availability of the service is not under control of OSIRIS (i.e., the service cannot be automatically deployed if needed). After service providers have become more familiar with OSIRIS, a loosely coupled service can be easily converted to a tightly coupled service. Simplified, the service handling code has to be adapted according to the OSIRIS service specification and linked with the OSIRIS library. After activation of the service in the OSIRIS node configuration, the service will be available as tightly coupled service. Finally, all the process specifications using the affected service have to be adapted, since there is a difference in the process specification between calling loosely coupled and tightly coupled services. 16

In the following, we describe in more detail the creation of application services for the C++ and Java version of OSIRIS. 5.1.1

Application Services in the C++ Version

A newly created tightly coupled application service in the C++ version of OSIRIS inherits from a base class that provides entry points for service implementation and takes over administration task. The class serves as a container for component-wide information, manages the worker threads, and communicates with Horus. All the messages from Horus to the component go to a work queue from which they are distributed according to the message type to appropriate command handlers. A command handler handles a request and returns a result or error. The statistical information is passed from the request to the answer. There can be more than one command handler for each component, depending on the requests that need to be handled by that component. After compilation of the new C++ component, an executable file is available, which is stored in a dedicated software repository. All components in C++ are executed as separate executables and communicating with the Horus component via shared memory. In order to make a new application service available at the designated OSIRIS node, i.e., to deploy it, the configuration of the node has to be edited. The Horus of the OSIRIS node is responsible for loading the current executable of configured application services from the software repository and starting the service instances/executables. Loosely coupled application services are standard Web services. These services may be hosted at any standards compliant web server. In order to make the WSDL description of the service available to OSIRIS, the service has to be registered at the UDDI repository. OSIRIS is not responsible for instantiation of such services. 5.1.2

Application Services in the Java Version

Similarly to the C++ version, tightly coupled application services in Java are also implemented by the creation of a specialized OSIRIS component class derived from an generally defined OSIRIS base component class. Special handling code can be implemented to control startup and shutdown of component instances. As Web services may provide more than one operation, also the OSIRIS Java component can be associated with a set of specialized command handlers. These command handlers are derived from a generic parent class and allow user-defined implementation of service operations. For using the newly defined application service, the bytecode of the new Java classes has to be available to the JVM and the service has to be configured to be started in the configuration file. In contrast to the C++ version, the OSIRIS Java version runs in a single JVM, which implies a single process at the target platform. The software repository is currently holding the OSIRIS Java bytecode library. A custom class loader takes care to use the most recent bytecode. Regularly checks at the software repository allow to restart an OSIRIS node if newer bytecode is available. Also loosely coupled application services are standard Web services. These services may be hosted at any standard compliant web server. In order to give OSIRIS more control over the lifecycle of loosely coupled services it is possible in the Java version to use a Web service container controlled by the OSIRIS infrastructure, the Web component(see Sec. 3.1.2). The Web component itself is an OSIRIS system service and therefore under full control of the infrastructure. With this combination, it is possible to allow service management for loosely coupled services.

17

Figure 7: The O’GRAPE Modeling Tool.

5.2

Definition of Processes

Before OSIRIS is able to perform process execution, appropriate processes need to be defined. The modeling tool O’GRAPE [26] is a key component of OSIRIS and allows for easy composition of services into process definitions. O’GRAPE is implemented in Java and runs as a stand-alone application as well as an applet. Figure 7 shows a screen shot of O’GRAPE. O’GRAPE offers process design in a graphical boxes and arrows approach for the control flow. The ellipses in Figure 7 indicate the activities of the process (service invocations). Furthermore, the data flow of a process is designed by the concept of a process whiteboard, the in-process variable area. At the end of the process design task, O’GRAPE outputs the process description in XML format and new/modified versions of a process can be uploaded into the process description repository of OSIRIS. The whiteboard (toolbox “Whiteboard”) of the process contains the global variables of a process instance. At instantiation time of a process, the whiteboard is initialized with the process input arguments (toolbox “Process Input Arguments”). Activities consume parameters from the whiteboard and save their results back at the whiteboard. Similar toolboxes named “Activity Input Arguments” and “Activity Output Arguments” are used to map the input and output parameters of a service invocation to the whiteboard. Finally, the return parameters of a process are fed back from the whiteboard and passed to the calling instance (toolbox “Process Output Arguments”). An innovative feature of O’GRAPE are both transactional and activation properties of the activities. Each activity is described by a set of properties like execution costs, compensability, retriability, and failure probabilities. Given a transactional process description, O’GRAPE is able to check correctness of the description based on formal criteria at design time. Activities using tightly coupled and loosely coupled services differ with respect to the process specification. Tightly coupled services are directly represented as activities in the process specification. In this case, the name of the activity corresponds to the tightly coupled service name and the input parameters of the activities are the input parameters of the OSIRIS service call. On

18

Figure 8: A Standardized Web Form for Testing Process Execution. the other hand, loosely coupled services are called indirectly by the SOAP component and this fact is reflected in the process specification. The address, operation name, and parameters of the SOAP call are input parameters for the activity. In order to assist on the design of SOAP calls, the designer can choose among a list of registered Web services known to OSIRIS (registered in the UDDI repository). On selecting a registered service, the WSDL description is parsed and the available operation within the Web services are selectable by clicking (see the selection box at the left hand side at the bottom of Fig. 7). After selecting the desired operation the SOAP parameters are automatically set and the designer only needs to map input and output parameters from and to the whiteboard. Process design by using O’GRAPE does not differ for the C++ and Java version.

5.3

Invocation of Processes

After the upload of a process description into the global repository, all affected providers are able to replicate the corresponding execution units they are responsible for. An application may trigger a process execution by sending the request containing the process name and input parameters to a process component somewhere in the OSIRIS network. The process component creates the process messages and initializes the whiteboard with the input parameters. The current whiteboard is part of each process message. Process messages are delivered to the appropriate providers, which are in charge of performing the current activity. In the process handler of the provider the process message is parsed and according to the execution unit (part of process specification), the local activity request is created (e.g., mapping of input parameters from current whiteboard) and locally executed. After execution of the local activity (e.g., the invocation of the SOAP component, which calls a standard Web service), the whiteboard is modified according to the activity results and the process message is passed on to subsequent activity provider.

19

Figure 9: Process Invocation from O’GRAPE. 5.3.1

Invocation in the C++ Version

The triggering of a process execution in the C++ version of OSIRIS can be done immediately after the process has been designed and uploaded to the process repository. This can be achieved by using a standardized input form to enter the process input parameters, execute the process instance, and view the process result parameters. The same result can be achieved by using a command line script to input the process input parameters. An example of such a standardized web form is depicted in Fig. 8. 5.3.2

Invocation in the Java Version

The invocation of process execution in the Java version can be easily implemented as a custom servlet or JSP offering the user an appropriate user interface for process invocation and result inspection. A simple example servlet is included in the codebase. Additionally, O’GRAPE offers a generic user interface for process invocation out of the process design interface (see Fig. 9). Fig. 9 illustrates the “OSIRIS Execute” button and the corresponding window for input of process parameter in the lower right corner of the screenshot. For this reason, O’GRAPE is communicating by using Java’s RMI and contacting a dedicated RMI component of the OSIRIS infrastructure, which is in charge of passing the process execution requests to the process component and delivering the result back to O’GRAPE.

References [1] Apache Web Services Project - Axis. http://ws.apache.org/axis. [2] K. B¨ohm, M. Mlivoncic, H-J. Schek, and R. Weber. Fast Evaluation Techniques for Complex Similarity Queries. In VLDB ’01: Proceedings of the 27th International Conference on

20

Very Large Data Bases, pages 211–220, San Francisco, CA, USA, 2001. Morgan Kaufmann Publishers Inc. [3] K. B¨ohm, M. Mlivoncic, and R. Weber. Quality-Aware and Load-Sensitive Planning of Image Similarity Queries. In Proceedings of the 17th International Conference on Data Engineering, pages 401–410, Washington, DC, USA, 2001. IEEE Computer Society. [4] G. Brettlecker, H. Schuldt, and R. Schatz. Hyperdatabases for Peer–to–Peer Data Stream Processing. In Proc. of ICWS Conf., pages 358–366, San Diego, CA, USA, 2004. [5] G. Brettlecker, H. Schuldt, and H.-J. Schek. Towards Reliable Data Stream Processing with OSIRIS-SE. In Proc. of BTW Conf., pages 405–414, Karlsruhe, Germany, March 2005. [6] IBM Cloudscape. http://www.ibm.com/software/data/cloudscape/. [7] The Apache Derby Project. http://db.apache.org/derby/. [8] T. Grabs, K. B¨ohm, and H.-J. Schek. PowerDB-IR: Information Retrieval on Top of a Database Cluster. In CIKM ’01: Proceedings of the tenth international conference on Information and knowledge management, pages 411–418, New York, NY, USA, 2001. ACM Press. [9] J2EE Java Servlet Technology. http://java.sun.com/products/servlet/. [10] J2EE JavaServer Pages Technology. http://java.sun.com/products/jsp/. [11] Jetty 100% Java HTTP and J2EE Servlet Container. http://jetty.mortbay.org. [12] The Project JXTA. http://www.jxta.org. [13] Microsoft Windows Server System - Microsoft SQL Server. http://www.microsoft.com/sql/. [14] Microsoft Visual Studio. http://msdn.microsoft.com/vstudio/. [15] M. Mlivoncic, C. Schuler, and C. T¨ urker. Hyperdatabase Infrastructure for Management and Search of Multimedia Collections. In DELOS Workshop: Digital Library Architectures, pages 25–36, S. Margherita di Pula, Cagliari, Italy, 2004. [16] M. Mlivoncic and R. Weber. A Filter and Refinement Approach to RBIR. In DELOS Workshop on Multimedia Contents in Digital Libraries, Chania, Crete, Greece, 2003. [17] MySQL AB - MySQL. http://www.mysql.com. [18] H. Schuldt, G. Alonso, C. Beeri, and H.-J. Schek. Atomicity and Isolation for Transactional Processes. ACM Transactions on Database Systems, 27(1):63–116, 2002. [19] C. Schuler, R. Weber, H. Schuldt, and H.-J. Schek. Peer–to–Peer Process Execution with OSIRIS. In Proc. of ICSOC Conf., pages 483–498, Trento, Italy, 2003. [20] C. Schuler, R. Weber, H. Schuldt, and H.-J. Schek. Scalable Peer–to–Peer Process Management – The OSIRIS Approach. In Proc. of ICWS Conf., pages 26–34, San Diego, CA, USA, 2004. [21] M. Springmann. A Novel Approach for Compound Document Matching. Bulletin of the IEEE Technical Committee on Digital Libraries (TCDL), 3, 2006. To appear. 21

[22] G. Tzanetakis and P. Cook. MARSYAS: a Framework for Audio Analysis. Org. Sound, 4(3):169–175, 1999. [23] R. Weber, J. Bolliger, T.R. Gross, and H.-J. Schek. Architecture of a Networked Image Search and Retrieval System. In Proc. of CIKM Conf., pages 430–441, Kansas City, Missouri, USA, 1999. [24] R. Weber and H-J. Schek. A Distributed Image-Database Architecture for Efficient Insertion and Retrieval Tasks. In Fifth International Workshop on Multimedia Information Systems (MIS’99), pages 48–55, 1999. [25] R. Weber, H-J. Schek, and S. Blott. A Quantitative Analysis and Performance Study for Similarity-Search Methods in High-Dimensional Spaces. In VLDB ’98: Proceedings of the 24rd International Conference on Very Large Data Bases, pages 194–205, San Francisco, CA, USA, 1998. Morgan Kaufmann Publishers Inc. [26] R. Weber, C. Schuler, P. Neukomm, H. Schuldt, and H.-J. Schek. Web Service Composition with OGrape and OSIRIS. In Proc. of VLDB Conf., Berlin, Germany, 2003. [27] The WS-Resource Framework WSRF. http://www.globus.org/wsrf/. [28] XML Transformations (XSLT). http://www.w3.org/TR/xslt.

22