A Generic, Distributed and Scalable Multimedia Information ...

Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999 Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999

A Generic, Distributed and Scalable Multimedia Information Management Framework Using CORBA Gábor Szentiványi, Pjotr Kourzanov Database Group, Dept. Information Technology and Systems University of Technology Delft Julianalaan 132., 2628 BK DELFT, The Netherlands {G.Szentivanyi|P.Kourzanov}@cs.tudelft.nl

Abstract

ing, now it is being used in a broader sense than earlier. Generic information management systems have not yet gained wide acceptance. The focus in information management has been on systems that are tailored for one specic purpose supporting only a subset of management functionality and thus not easily extendible. The need for transition to generic systems is due to the generic nature of management tasks, the versatility of media-types in that information is embodied and the constant demand on extensions for media-types as well as management functionality. Our approach on providing a generic, multimedia information management builds on two notions: (1) a model that covers all relevant aspects of media management, that is, storage, streaming, interchange, query, manipulation and presentation of information of several media-types, and (2) a distributed architecture that provides distribution, migration and access for the object model and its realization in a seamless, congurable and scalable way. This paper aims at introducing a framework that is being built for such a purpose. The focus of this paper is to give an insight into the relevant aspects of the design and implementation, rather than tackling some of those aspects in detail. In section 2 we give a brief description of the model, in section 3 the architecture of the system is tackled, in section 4 we deal with general implementation issues, CORBA integration and implemented components. Finally, in section 5 we put our work into the context of related works and in section 6 we make some concluding remarks1.

This paper describes the design and the implementation of a framework for multimedia information management. The framework is built upon two notions. The one notion is a model that describes multimedia information management by (1) dening foundation objects that denes dierent aspects of multimedia management and by (2) providing a composition that brings those dierent aspects together in a consistent way and provides interoperability between them. The other notion is a distributed infrastructure, that is, CORBA, in which the model has been implemented. In this paper we give an overview of the COMMOTION model and its implementation. In the description of the implementation we focus on the implementation of a server and a client component. Our emphasis at the server side is the integration and implementation of standardized CORBA services as well as additional multimedia functionality. At the client side we emphasize the integration of existing frameworks/submodels into COMMOTION. Finally, we describe the main components we have built using the implementation.

Keywords: CORBA, distributed multimedia, information modeling, object model

1 Introduction

The quantity and the versatility of information to be 1 The research and development described in this paper is being managed at enterprise level is permanently increasing. The denition of management is also constantly chang- carried out in the framework of the WONDA project (ESPRIT 1

0-7695-0001-3/99 $10.00 (c) 1999 IEEE

1

Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999 Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999

2 The Model

combining the foundation objects into an aggregate that manages its parts, enables communication among them and provides a uniform view through its interface.  uniform, consistent handling of all media-types: This feature reects the nature of multimedia information to be versatile. Versatility of information can be handled eectively if we classify versatility by creating a common base and particular specializations. The main classication is based on media-types. Each media-type may have dierent requirements, complexity and behaviour. COMMOTION ensures that instances of each media-type can be managed (e.g. accessed, displayed, stored, manipulated) in a similar way, using the same semantics. This provides a straightforward way of using existing, and the incorporating new media-types.  modeling at arbitrary abstraction levels: This feature reects the nature of multimedia information to carry rich semantics. The required detail of modeling semantics of information may vary. The breakdown of semantic modeling into various aspects provides independence in dening those aspects at arbitrary abstraction levels, there is no necessity to use predened abstraction levels.  management functionality : There is a set of generic management functionality that is dened in COMMOTION, which comprises the creation, manipulation, presentation, query, storage and interchange of media information. Complex management activities can be composed of simple ones.

COMMOTION (Comprehensive Object Model for the Management Of mulTImedia informatiON) is the object model that forms the basis of the multimedia management framework. For clarity, we dene some terms we use throughout the paper: We use the term model to describe concepts in a consistent way. A schema is a particular instantiation of a model2 . An object is a generic term for denoting basic elements of models and schemas. If we want to distinguish between elements of models and elements of schemas, we break down an object into a type and an instance, respectively. The term interface is the user's notion of a type. The term class is the user's notion of an implementation of a type. An instance is an instantiation of a class. Factories are instances that instantiate other classes. Both factories and classes can be used to instantiate type implementations. For a more detailed discussion on these issues, the reader is referred to [9]. In this section we rst describe the main features we considered at the design of the COMMOTION model, after that, the model itself is described.

2.1 Main Features of the Model

The features of COMMOTION reect (1) the nature of multimedia information to be managed, that is, its (a) large quantity, (b) versatility and (c) rich semantics, and (2) the required functionality of management that applies on the information. First we elaborate on the features that originate from the nature of multimedia information, then we concentrate on management functionality.

 separation and comprehensiveness through founda-

2.2 Model Overview

tion objects : COMMOTION denes a number of foundation objects that dene the main concepts of the model. The separation of main concepts reects the need to handle large quantities of information: physical content management, semantic description and application design are decoupled, as constraints that apply to each of those aspects are dierent. Decoupling within the model ensures that developing one aspect is orthogonal to other aspects. Comprehensiveness of the model is achieved by

COMMOTION contains seven foundation objects that can be classied into three partitions, such as Core submodel, Resource submodel and Application submodel. The foundation objects are all specializations of COMMOTION Object3 , which provides generic functionality for relationships (containment, reference), naming, persistence and role management. Besides the foundation object specialization tree, there are some generic, helper objects.

3 In the remainder of this paper we refer to COMMOTION project no. 25.741). as a model or a schema, and to CMT Object as one object of 2 To avoid confusion with database terminology, in case we refer the COMMOTION model or an instance of the COMMOTION to a database schema, we explicitly mention that. schema.

2

0-7695-0001-3/99 $10.00 (c) 1999 IEEE

2

Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999 Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999

Relationship

Role

describe how the Data Object should behave in terms of media-types, whereas Subsystem Appearance denes the encapsulation of media-type appearance into a programming framework (e.g. windowing system). Actitvity Object denes activities that can be carried out on CMT objects. These may involve the creation, interchange, enquiry, manipulation and presentation of structure, appearance and/or physical content. The goal of the Application submodel is twofold: integrating COMMOTION into existing client environments (e.g. graphical, audio, video subsystems) and supporting application design. The Application submodel contains a compositional hierarchy of Media Objects, where each Media Object can be composed of a number of Data Objects, Appearance Objects, Activity Objects and other Media Objects. The compositional hierarchy is indirect, since it denes compositions by relationships, where additional semantics can be expressed. Whereas the framework developer integrates existing resources into the resource submodel and provides basic DOs, AppOs and ActOs along with the denition of their mapping onto resources, the component developer specializes those DOs, AppOs and ActOs and composes them into basic MOs to provide components for applications in a certain client environment. The application developer specializes the MOs of that environment and uses them to build even more complex MOs and/or applications. Helper objects are objects that provide functionality applicable to all CMT objects in general. The reason why those objects are separate entities rather than merged into the CMT object itself is reuse and design transparency. For eciency, however, there are some restrictions. All CMT objects may contain a set of properties, which is a sequence of name-value pairs. Properties provide a generic, exible management of states and attributes of CMT objects. A specialization of Properties is Policies. Policies also contain a set of properties, but also additional management to describe policies using properties (e.g. severity of policy). Policies may be attached to objects of the resource submodel, rather than to all CMT objects. Policies are requested values of parameters for realizing a service at a given quality level, whereas properties directly reect the behaviour or state of the object. Explicit relationship modeling with the Role-Node-Relationship group provides a exible, generic mechanism for expressing rich semantics and constraints. Besides roles in relationships, CMT

Node

Application submodel Media

CMT

Activity

Appearance

Visual

Properties

Data

Policies

Resource

Phys.Content

Storage Dev. Context

Audio

Comp.Context

Temporal

Stream

Subsystem

EndPoint

Core submodel

Computat. Dev.

StreamNet Dev.

Streaming Dev.

Resource submodel

Figure 1: UML overview diagram of COMMOTION An overview of the model using UML notation is depicted in Figure 1. The gure uses UML notation [8] in a simplied form, that is, we only denoted object types, specializations, associations and compositions. The Resource submodel describes the resources in the framework. The two main objects in this submodel are the Resource Object and the Context Object. Since these are abstract concepts, we provided parallel specializations of them for storage (Physical Content and Storage Device), for streaming (Streaming Device and EndPoints) and computation (Computational Device and Computational Context). Physical Content contains raw, uninterpreted sequence of bytes of any kind of media information. Resource Objects describe the devices on which and among which the Physical Content should be stored (Storage Device) and streamed (Streaming Device), respectively. Most common Storage Devices in COMMOTION are the lesystem, harddisk partition and main memory, Streaming Device is the network. The Core submodel contains three objects: the Data Object (DO), Appearance Object (AppO) and Activity Object (ActO). The Data Object describes the structure of the multimedia information using a compositional hierarchy. Every object of this hierarchy may contain a set of Properties and a reference to Physical Content. Visual, Audio and Temporal Appearance 3

0-7695-0001-3/99 $10.00 (c) 1999 IEEE

3

Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999 Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999

object itself has a role that expresses the notion of dynamic type. As opposed to the type of an object, which cannot change during its lifetime, roles can be assigned and deassigned any time. Roles are very useful in expressing semantic constraints on behaviour.

scope of current paper

COMMOTION streaming

CORBA CLIENT

PERSISTENT CORBA SERVER

3 The Architecture

CORBA SERVER

CORBA SERVER

CORBA SERVER

CORBA/IIOP

The architecture denes an environment for the implementation of the model. After having described the model in the previous section, we put it into a real environment. In this section, we describe how we use CORBA both as a core architectural infrastructure and as a set of services to realize our model. Finally, we point out necessary extensions to CORBA, along with Figure 2: COMMOTION in Context with basic and their implementation. extended components TRANS. CORBA SERVER

CORBA SERVER

CORBA CLIENT

3.1 Distributed Architecture: CORBA 3.2.1 Basic Components CORBA[1] is the one of the recent architectures to provide a distributed infrastructure. Other architectures include proprietary two-tier and three-tier systems. We have chosen for CORBA because of its scalability, openness and exibility. With the choice of CORBA, tierless solutions can be built using components instead of tiers, whereby each component is a rst-class element of the architecture providing a certain set of services. Even if initial costs of designing and implementing componentware systems are higher, the long-term benets substantially supersede that of traditional solutions. There are competitors of CORBA for the provision of a distributed architecture, such as DCOM and RMI, [11] provides a good comparison.

Basic components are components that enable the system to operate. A Media Repository as a CORBA server and a client as a CORBA client are essential to provide a basic architecture.

Media Repository : This is one of the major server components of the system. It provides persistence and availability for COMMOTION objects except the Resource submodel. The server has been implemented in C++, because it is a wellperforming and ecient language providing objectoriented features.

client: Client is the component that provides the user

with a user interface5 . On one hand, it acts as a CORBA client that accesses CMT objects in the Media Repository, on the other hand it realizes the Resource submodel, therefore it can send and receive streams (by the Endpoints of Streaming Devices). We have written clients in Java to facilitate security, portability and easy downloadability (e.g. into internet browser environments), but nothing prevents developers from writing clients in other languages.

3.2 Architectural Components There are two kinds of architectural component in CORBA: client and server4. In order to provide a basic architecture, we have to realize at least one CORBA server and one CORBA client. In an extended architecture, we can realize several types of server and client. We rst describe a minimal COMMOTION architecture and then possible components to extend the system. Figure 2 depicts the components in the architecture.

4 Strictly speaking, this holds only for the time of an IIOP 5 We always use the term client in this sense. When we talk request, however, there are components that mostly initiate such requests and others that mostly serve them. about CORBA clients, we explicitly indicate that.

4

0-7695-0001-3/99 $10.00 (c) 1999 IEEE

4

Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999 Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999

3.2.2 Extended Architectural Components: COMMOTION in Context

Devices and Streams. A more sophisticated use of the Media Server is to provide implementation for Computational Devices and Contexts. The Media Server can use the Media Repository to make its objects persistent if necessary. This necessity depends on the nature of the Media Server implementation and the type of objects. renderers: Renderers can provide an upscaled solution for rendering for internal CORBA components (e.g. for a client) as well as for external targets. In case the renderer renders for internal components, it comprises a CORBA server as well as a CORBA client and its input and output are COMMOTION objects (e.g. rendering 3D graphics into an image). In case it backs up external servers, it outputs data suitable for those external servers (e.g rendering documents to a mark-up language for http servers from COMMOTION schemas).

In this part we describe components that widen the applicability range of COMMOTION. Adding extended components to the basic ones impacts the whole system. The added value includes interfacing with existing nonmultimedia capable environments by providing them with middleware and client functionality, scaling up existing components, and backing up traditional servers by providing them with data (e.g. http server). An enterprise-wide information system often faces the problem of integrating the management of traditional and (multi)media data. COMMOTION has a double impact on such an integration: on one hand it serves as a multimedia document repository and presentation system, on the other hand as a persistent GUI for heterogeneous datasources as well as documents. Substantial tasks of such an environment are feeding the document repository with documents from outside the framework, maintaining relationships between traditional data and documents, and mapping data into GUI front-ends. For these extended tasks the following components have been dened: persistent mappers: Persistent mappers are CORBA clients that use the Media Repository for persistence. These components import information from outside the framework into COMMOTION by parsing an external document and storing it. Accessing the external source mainly happens using a lesystem and a well-known exchange format. transient mappers: Transient mappers are CORBA clients as well as transient CORBA servers that do not store any information within the framework. They make the COMMOTION client act as a frontend for information coming from datasources outside by acting as a middleware. This middleware instantiates and customizes GUI templates stored in the Media Repository, as well as applies bidirectional mapping between data of several sources and those GUI instances. Scaling up existing components to make them applicable for systems with a lot of simultaneous users requires the denition of extra components: Media Server : The Media Server implements the COMMOTION Resource submodel. This component is mainly used for storing and streaming physical content by realizing Storage Devices, Streaming

3.3 Extensions to the CORBA Standard

CORBA is a mature concept as a basis for distributed computing but it contains additional features that are sometimes present as immature proposals or proprietary extensions. The CORBA product we used was Orbix 3.2 and OrbixWeb 3.0 [7]) provided us CORBA 2.0 compliance as well as Naming, Event and Transaction Service. Extensions were necessary to facilitate storage and streaming of media data, and the simultaneous usage of local and distributed objects. Storage and streaming are additional concepts for persistence and time-aware interchange of Physical Content among Resources. The CORBA communication protocol, IIOP, has been developed fora request/response type of communication, rather than for communicating big amounts of bulky data between components. Storage and streaming technologies are therefore not concerns of CORBA, however, CORBA can be used to control those tasks to make them thereby transparent to the user. There already exists a CORBA Service proposal for A/V streams[3] that is quite promising. The reasons why we did not choose to reuse existing implementations ([19][20]) or did not implement it are that (1) as of now, the A/V streams specication has not yet been nalized, (2) COMMOTION handles streaming and storage within one concept, as specializations of resource management, and (3) the idea of streaming in COMMOTION is general, rather than restricted to

5

0-7695-0001-3/99 $10.00 (c) 1999 IEEE

5

Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999 Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999

4.1 Server Persistence Integration

A/V streaming. The current state of our implementation of the Resource submodel is described in section 4.4.1. Simultaneous usage of local and distributed objects requires that the component or application developer knows about and applies several APIs for developing an application. In the current case it would typically involve COMMOTION and one or several Java APIs for the client environment, and COMMOTION and several C++ libraries in the server. This increases development time and might cause a gap in semantics. With the adoption of an additional technology, smart proxies6 , we make the scope of the object (local/distributed) transparent to the client. Currently this technology is used in the Activity Object (see section 4.4.2).

A straightforward way to provide persistence functionality for a CORBA server is to reuse the services provided by a DBMS. ODBMSs have already been proven as a natural way of storing objects and object structures truly preserving and managing their inherent features (inheritance, relationships, attributes). There are many ways to realize CORBA server persistence, because both CORBA and ODBMS APIs give the implementor great exibility. Therefore, we had to base our implementation of server-side persistence on several assumptions:

 the CORBA client should not be able to control

persistence beyond instantiation (transparence): Transparence of persistence means that CORBA clients are unaware of the persistence behaviour of objects, therefore CORBA clients do not depend on the persistence capabilities of the CORBA server. This assumption enables easier integration of existing object-oriented client applications and submodels into a persistent environment, and objects are more resilient to changes in persistence on the CORBA server.  the CORBA client should be able to choose how to control transactions (exibility): Clients should be able to control the granularity of transactions from complete transaction unawareness to atomic (method based) transactions. Transaction unawareness has the same advantages described in the previous item. However, via controlling transactions, clients can not only inuence accessability (locking) of objects, but they can have some inuence on persistence if necessary (e.g updating changes by commit, canceling changes by abort).  there should be one-to-one correspondence between CORBA server components and the DBMSs providing persistence (simplicity): There are numerous possibilities what concerns the interconnection of DBMSs and CORBA servers. A simple and straightforward possibility is the one-to-one connection. The main advantages of this approach are that the CORBA server does not need any additional mechanisms to manage several datastores.  once objects are instantiated, they should not be able to change their persistence behaviour (immutability): What concerns persistence behaviour,

4 The Implementation The main driving force behind implementation considerations was reuse. Given our model and the architecture we wanted to embed it into, there have been two kinds of reuse to consider. Implementation reuse is applied for the server as well as the client component: The reuse of ODBMS functionality is a straightforward way to provide persistence to CORBA objects. In the client, the integration of the user interface is based on reusing existing submodels, rather than developing it from scratch. Design reuse has been considered when deciding to create consistent semantics by applying COMMOTION for all the components including servers and clients. Design reuse has also been the driving force behind adopting standardized CORBA services. The description of the implementation is split into four parts. In the rst part, we deal with general implementation considerations for persistence. In the second part, we focus on the implementation and integration of CORBA services. In the third part we deal with general considerations for client subsystems integration, and nally we the describe the implemented CORBA components, that is, the Media Repository, the Media Server and the client. 6 This non-CORBA-standard mechanism is provided by several CORBA implementations, including Orbix (smart proxy) and VisiBroker (smart stub).

6

0-7695-0001-3/99 $10.00 (c) 1999 IEEE

6

Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999 Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999

LifeCycleService

Factory

Persistence Service

Relationship Service

server

Database

Relationship Manager

Role

Transaction

PropertySet

represents

invokes

CMT Node Data

Generic Property

represents

invokes

Resource Context Specific Properties

PropertyService

Resource Management

Data Management

Legend Persistent Object

Specialization Composition

Association Instantiation

IDL IF / Servant

Figure 3: Implementation of CORBA services and their Figure 4: UML notation of the persistence mechanism in context relationship to COMMOTION

4.2.1 Persistence Service

objects should be immutable beyond their instantiation, that is, with the selection of the factory, the server and therefore object persistence behaviour is immutably determined. Another approach is to use persistence by reachability. Although it provides greater transparence in persistence, it requires more computation. We abandoned it in favor of immutable persistence behaviour.

Although our implementation of this service is not yet compliant to any CORBA standard or proposal, it shares the ideology with recent CORBA PersistentState Service proposals [4], that is, persistence is server controlled through the persistent object-incarnationservant chain7 . Figure 4 depicts this situation8 . In our implementation, the incarnation type is inherited from the type of the Persistent Object. The persistent object only realizes the state of the incarnation, whereas the incarnation itself implements the behaviour. This step allows for changing behaviour without changing the ODBMS schema. Moreover, DBMS functionality is not compromised, since C++ bindings of ODBM systems are not concerned with the methods of persistent objects. The servant uses the TIE concept, rather then inheritance, which is mainly due to the facts that in this way we avoid multiple inheritance and we obtain more control over which and when persistent objects are exposed to CORBA. Since the notion of transactions is essential to the DBMS, we have to incorporate transaction management into the persistence service. In order to be exible

All the assumptions listed above are in accordance with the guidelines of ODMG standards [4], and CORBA PSS proposals [5].

4.2 Integration and Implementation of CORBA Services

The COMMOTION framework makes use of several CORBA services [2]. There are standalone services that have already been implemented by Orbix, others have been implemented by us. Services that we reused are the Naming Service and the Event Service [4]. The im7 These terms originate from the POA (Portable Object plementations we provide are Relationship Service, LifeAdapter) terminology. The POA is standard component of cycle Service, Property Service and Persistence Service. CORBA 2.2. 8 The incarnation type is not shown in gure 4, because it doest Figure 3 depicts server implementations and their relationship to COMMOTION. not contain any useful information in the current context. 7

0-7695-0001-3/99 $10.00 (c) 1999 IEEE

7

Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999 Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999

(see section 4.1 on general assumptions), we have provided three transaction models: tp_none (no transaction, which can only be used for special purposes, such as standalone administration), tp_explicit (clientcontrolled transaction) and tp_automatic (servercontrolled transaction). In order to automate persistence management for server objects and facilitate the development of server components in general, we implemented the servant using the Persistent TIE (PTIE) template skeleton. We register the PTIE object with the DBMS transaction currently in progress (in case of tp_explicit or tp_automatic) in its constructor, and later, when the PTIE object receives requests, it wraps them into that transaction context. The former step is necessary in order to deregister PTIE objects (servants) from CORBA when the transaction is committed by the client explicitly, because objects can not outlive the transaction context in which they have been loaded. The appropriate transaction is begun before the call propagates to the incarnation and aborted or committed after the incarnation is ready, depending on the result of the execution. In case of tp_automatic transaction policy, transactions are started on-demand. Once the loader gets the chance to load the object from the DBMS and there is no transaction currently in progress, it starts a new transaction. All further requests are done using that transaction until either the client application (implicitly, by calling destroy() on an object) or the server (explicitly) decides to nish and destroy the transaction. If no such request or decision is made, all transactions are dealt with when the client disconnects.

Relationships must be statically subtyped in order to be used meaningfully. Our idea is having generic Relationship and Role objects that contain explicit information in the form of readonly attributes, such as degree, role_types and relationship_type, rather than having this information only be implicitly present in the interface name. The advantage of this approach is that applications can handle relationships without apriori knowledge of their semantics. In the implementation of this service, there have been three main considerations. For eciency, if objects to be related are on the same CORBA server, that is, they are collocated, we reuse that CORBA server for storing their relationships, too. For exibility, if objects to be related are on dierent servers, we store their distributed relationships on a dedicated relationship server. For design transparency, we make CORBA clients have seamless handling of relationships, regardless of whether they are collocated or distributed. We do not fully implement object identity required by CosRelationships, but base our referencing mechanism on IORs. This introduces some problems, since IORs are opaque and we have limited knowledge of object identity based on IORs9 . Therefore, we decided not to store IORs in the place we store the objects, so that in case the IOR changes (e.g. object migration) the updating of distributed relationships can be localized in the separate relationship server leaving all other servers intact (since objects in the relationship server are not referenced from anywhere else). For eciency, CosRelationships have been implemented in two ways: rst, as a library for intra-database (intra-process) referencing using ODBMS OIDs, and second, as a separate server component for inter-database (inter-process) referencing using IORs10 . Both implementations use the ODBMS for storing relationships and roles, however in dierent databases. In order to achieve referencing type (intra/inter process) transparence, we extended the rst implementation with the RelationshipFinder interface, which, given the IOR and the Role type of the object, returns a reference to the Role object that uniquely represents the object in a distributed relationship.

4.2.2 Relationship Service

Our implementation of the Relationship Service [2] contains the full CosRelationships and CosGraphs functionality, with extensions. First we describe the extensions we made to the service, then, we elaborate on implementations issues. During the examination of the service specication, we found that CosRelationships contain two misleading notions. Firstly, according to CosRelationships, one 1 to n  binary relationship must be modelled with n binary 1 to 1 Relationship objects. This is an unnecessary duplication, which can be avoided by allowing Role objects to reference several Node objects, whereby Node 9 After migration from one CORBA server to another, two objects share the same role in a certain relationship. identical objects may have dierent IORs. Secondly, the Relationship and Role objects in Cos- 10 The latter case is not shown in gure 3. 8

0-7695-0001-3/99 $10.00 (c) 1999 IEEE

8

Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999 Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999

4.2.3 Lifecycle Service (Persistent Factories)

action Objects. Once the CORBA client has the Transaction Object, it can invoke standard methods on it. If using dierent transaction policies, there is no need for the client to be concerned with the Manager and Database objects. The Relationship service is used as a backbone for COMMOTION: it connects Media Objects, Data Objects, Appearance Objects and Activity Objects in order to build compositional11 hierarchies. The Relationship service also plays a key role in the exibility of the Media Object, since it is multiply inherited from a Relationship and a Node (through the CMT Object) (see section 4.3). The exibility (separated storage of collocated and distributed relationships) of the Relationship service implementation can be taken advantage of if one knows in advance which relationships are collocated and which are distributed. A typical assumption is to make MO-to-MO relationships distributed and DO-to-DO relationships collocated. In order to constrain and facilitate the development of specialized CMT objects and applications, we provide attributes and methods as shortcuts to specialized relationship management. A typical example is depicted in gure 3 in the Data Management box, showing the IDL interface/servant expressing parent/child relationship with attributes for Data Objects, which is not present in the Persistent Object, but mapped into a generic relationship. The Property Service adds generic properties to the CMT Object. Properties also play a double role in CMT Objects in general: a mapping mechanism in the Media Repository allows clients to access static attributes dened in IDLs to be accessed and manipulated as generic properties. An alternative to this solution would be the usage of the Dynamic Invocation Interface (DII) and the Interface Repository (IR). However, this alternative would not only suppress the semantics of properties, but would also require more computation from the client.

Since it was not our goal to implement functionality for database distribution (e.g. migration) but an ODBMS independent storage/application distribution, and there is no notion of passing objects by value in CORBA 2.0, which CosLifeCycle requires, we did not implement the full LifeCycle Service, but only the concept of GenericFactories. We did not, however, adhere to exposing the Factory structure to the clients (Key s in CosLifeCycle) because this is irrelevant to the client application. Instead, we associate one RootFactory object with the Database (and thus with the Manager) and allow the client to pass object-creation requests to it, specifying all the requirements for the end-product (the object) in an NVList in a dynamic way. Then the RootFactory elects a LeafFactory , based on the criteria for that factory and the criteria specied by the client, and nally passes the creation request to it. The CORBA server that contains the Factory tree must also contain a PropertyService, since criteria are stored as PropertySet s.

4.2.4 Property Service COMMOTION uses PropertyService for dynamic attribute management. Dynamic attribute managent can be implemented using both typed and untyped attributes. Since ODBM systems in general do not allow for querying over untyped attributes, which is essential for property management, we restrict the CORBA Any type that can be passed as value of a Property to a set of xed, predened literal types (boolean, short, long, double, float, string) and to certain literal structures (e.g. 2Dpoint=). Figure 3 depicts the mapping of the generic IDL struct into specializations of the Persistent Object.

4.2.5 Integration of Services into COMMOTION 4.3 Client Subsystem Integration The Persistence Service makes COMMOTION seamlessly persistent when objects are instantiated by factories of the Media Repository (the only exception is physical content, which is handled by the storage mechanisms of the Resource submodel implementation). The CORBA client only has a notion of transactions in case of tp_explicit. In this case, it uses the Manager to access the Database object that stands for the database. The Database object solely acts as a factory for Trans-

After tackling general server implementation issues, we now have to deal with the other major components of the framework: the client. This Since the COMMOTION client provides the user interface to the user, in order to facilitate implementation reuse, the client has to be embedded into existing local subsystems. Some

11 In CosRelationship terminology compositions are called containments.

9

0-7695-0001-3/99 $10.00 (c) 1999 IEEE

9

Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999 Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999

important subsystems we have to deal with include graphical and media subsystem. Those systems have their own models and APIs through which their functionality can be exploited. Embedding user interfaces, which are embodied in local subsystems, has more advantages: COMMOTION provides a uniform semantics for local GUI as well as distributed objects, and local GUI objects can take advantage of generic services provided by COMMOTION (e.g. persistence). Our goal is therefore to make objects of the graphical system transient incarnations12 of persistent COMMOTION objects. Most graphical systems are based on the MV (model-view) or the MVC (model-view-controller) approach, which originates from Smalltalk and gained wide acceptance in graphical modeling. From the integration point of view, COMMOTION has to deal with views and models, whose integration is dierent. The view of a graphical subsystem corresponds to a Media Object. The means for the Media Object to communicate with local subsystems is locally executing Activity Objects (see section 3.3). The Appearance Object describes how the Media Object has to appear as a certain local view. The model of a graphical subsystem corresponds to any CMT object or object structure connected to the Media Object that stands for the view. The key to subsystem integration is the double semantics of the Media Object: on one hand it acts as a basic modeling element that holds information in form of a (multi)media-type by connecting its aspects by the containment relationship. On the other hand, the Media Object is the relationship itself, being a shadow view for the incarnation of a view of a particular local subsystem. Figure 5 depicts this situation13 . The functionality that has to be provided in the local Activity Object for accessing subsystem functionality consists of:

 object creation : The view object is instantiated us-

ing a factory. The factory is generic and takes the type of the view object from the subsystem appearance of the Media Object that stands for the view.  customization : The view's attributes have to be properly set. A congurator for local view objects sets the attributes using information contained in the Appearance Object.

MV based subsystem (graphical, media)

"view incarnation"

view appearance provision

model adapter factory

ActO

configurator

MO

AppO

renderer local, subsystem dependent activities

appearance in a given subsystem

model provision

"view shadow"

global, subsystem

generic, subsystem

independent activities

ActO

independent appearance

MO

AppO

Legend node relationship role

DO

information flow

"model for the view"

COMMOTION

Figure 5: Integrating MV subsystems on the client

 rendering : This task presents a certain view on

the screen. The task is either explicitly called by some activity (e.g. present) or is called back by the subsystem (e.g. when the view becomes invalid).

 model provision : In case there is a need for providing a model for a particular view, a model adapter realized by a local Activity Object maps the model required by the subsystem to any CMT object or object structure. The model adapter is called back by the subsystem on demand.

4.4 Implemented Components

We have built several components of those mentioned earlier (section 3.2). Due to space limitations, we tackle three components, that is, the Media Repository, the Media Server and the client in detail and just give a brief description of other components.

4.4.1 Media Repository

This incarnation is not the incarnation used in the context of The Media Repository is a CORBA server component the persistent service. However, their notions are similar. 13 The notation in this gure is taken from the CosRelationship that embodies the Persistence Service, Property Service, specication [2]. RelationshipService and LifeCycle Service, as well as the 12

10

0-7695-0001-3/99 $10.00 (c) 1999 IEEE

10

Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999 Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999

implementation of the COMMOTION model. We have implemented the Media Repository using the POET 5.0 ODBMS [12] through its native C++ binding. The Media Repository is also operable without a Media Server, however, in this case, media information can only be present in the properties of the Data Object, not as Physical Content. This solution is suitable for modeling and implementing GUI views and text mediatype. The limitations are due to the fact that the only communication protocol is IIOP, bulky data can only be modelled and stored as the string literal type both in CORBA and in the ODBMS.

4.4.2 Media Server

ve ways of assigning the Media Repository (MR), Resource Storage (RS) and Content Storage (CS) to storage components: ; ; ; ; . We abandoned the last two possibilities for the following reason: Combining the Media Repository and the Content Storage in one database is disadvantageous, since COMMOTION expresses a strong decoupling of structure and physical content. The only reasonable way of storing physical content within a DBMS would then be the notion of BLOBs. However, even if ODBMSs provide transaction, locking and query management for BLOBs, they do not facilitate the control of storage and streaming. Before describing the remaining three possibilities, we rst argue about the possible realizations of Storage Devices that can be a lesystem, a raw partition or even an autonomous, specialized product. The most straightforward solution is to use a lesytem. The advantage is that its integration with existing systems can rely on a lesystem-based communication and it is easy to port to other platforms. A faster way to eciently store and access physical content is to use raw partitions. When several simultaneous clients request streaming services, a high-performance upscaled solution, such as an autonomous Media Server can be used14 . A common feature of all three approaches is that they lack transaction management. This provides exibility rather than being a deciency, since the need whether to provide transactional properties to physical content management cannot be decided in general, all activities on the physical content should be able to dene their own policies (performance and eciency versus reliability). Therefore, ODBMS locking and transaction management would prove as too rigid and sometimes superuous. In our current implementation, we make use of the lesystem for physical content storage. We only provide rudimentary access control for physical content, which comprises object level read/write locking. Since there is currently no notion of transactions in the Media Server, all lock-management has to be made explicitly. The three possibilities mentioned above are depicted on gure 6. Subgure 6a shows three separate storage components. The Media Repository does not deal with resources and their contexts at all, a separate POET [{MR},{RS,CS}]

[{MR},{RS},{CS}]

[{MR,RS,CS}]

[{MR,RS},{CS}]

[{RS},{MR,CS}]

The Media Server is the component that acts as a CORBA server for COMMOTION Resource submodel objects, including a storage/streaming server for Physical Content. COMMOTION components have a notion of the Media Server through the resources it exposes to CORBA. The Media Server implements the following resources (Resource Objects): network (NetworkDevice), memory (VirtualMemoryDevice, PhysicalMemoryDevice), hard disk (FileSystemDevice). Some of the Device Objects are inherently persistent (e.g. lesystem), while others are inherently transient (e.g. memory). For persistent Resources we have to use some persistence mechanism to store their state and contexts. Transient resources and therefore their contexts cannot outlive a server session: they do not rely on any persistence mechanism. There are resource types (e.g. Computational Device), which do not correspond to concrete physical resources, but rather serve as convenient means to model streaming and state. Device objects contain Contexts. On Storage Devices, content is associated by specifying a logical region on the Storage Device. In case of the Streaming Device, Context identies EndPoints the physical content travels through. There is a special Resource called StreamNet that contains all the available streams in an environment. Its Context is the Stream that denes a transient route in the StreamNet. In case of Computational Devices, Context stands for the state of the computation that is associated with the content. The Media Server has to store its persistent devices and their contexts, and interoperate with the Media Repository. In order to analyze dierent possibilities, 14 We are currently working on the integration of Oracle Mewe split the Media Server into two logical parts: Re- dia Server [6] into our system. The detailed discussion of this source Storage and Content Storage. There are are integration is beyond the scope of this paper. 11

0-7695-0001-3/99 $10.00 (c) 1999 IEEE

11

Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999 Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999

a

Resource Storage

b

POET OID

Device

OID

local JFC dependent implementation

Content Content

Object

Content

Data Object

COMMOTION

POET Data

Device

Generic Commotion Browser Application

Media Repository / Resource Storage

smart proxies

resource impl. streaming

Object

Object

Java JFC

POET

Media Repository

OrbixWeb ORB

Path

Path

Swing

Filesystem/ Partition

Filesystem/ Partition

Physical Content Storage

Physical Content Storage

2D API

3D API

Java Media Framework

JDK 1.2

JRE c

POET

POET

Autonomous Media Server

Reference Data

COMMOTION streaming

Device

IOR

IOR

Object

Content Content

Figure 7: Overview of the generic COMMOTION browser

Object

Referred by

Media Repository

CORBA

Referee

Relationship Server

Physical Content / Resource Storage

Figure 6: Dierent server architectures for Media Server integration database is used for this purpose. In subgure 6b, persistent Devices and their Contexts are stored in the Media Repository. Subgure 6c depicts a solution, where the Media Server is autonomous, it manages its own resources. The reuse of the Media Repository for Resource Storage introduces a problem. Since the Data Object must reference its Physical Content, there has to be a means to identify a particular storage Context from within the Media Repository. As shown in subgures 6a and 6b, the problem is solved through cross-database references of the POET ODBMS. The only constraint for this is that the target database has to be of the same layout what concerns ODBMS and database-schema. Figure 6c depicts solution with a separate, autonomous media server. This solution assumes that the Media Server itself has persisitent management for Resource as well as Content Storage. For relating the Data Object and Physical Content a separate Relationship Server is used. Our implementation is currently based on the solution depicted in subgure 6a. The advantage of this approach is that the databases can reside on separate instances of DBMSs running on dierent hosts, thereby improving the distribution of resources.

4.4.3 The Client The client we have implemented is a generic COMMOTION browser (GCB). The reason for implementing a browser as our main client application has several reasons. Firstly, it serves as a general proof-of-concept application for the design of COMMOTION and its implementation using CORBA. Secondly, when building specic clients and applications, GCB helps developers browse COMMOTION objects, relationships and properties. Thirdly, it contains specic functionality to render media data in a given environment, which can easily be reused for building specialized applications. The client has been implemented in Java using the pure Java ORB from OrbixWeb. The GCB uses two subsystems that are integrated with COMMOTION: Java Foundation Classes (JFC) [10] and Java Media Framework (JMF) [24]. Both subsystems have been integrated using the mechanism described in section 4.3, which demonstates the exibility of the integration approach, since JFC is a ne grain, while JMF is a coarse grain integration. Besides accessing server functionality, smart proxies dispatch calls to a local JFC based Media Object and JMF based Resource Submodel implementation. Both subsystems are based on the MV approach, therefore we were able to integrate them in a straightforward way. Figure 7 depicts the architecture of the browser. In this paper we focus on JFC's Swing components. From COMMOTION's point of view, the integration Swing components can be split into two parts. The rst part is the provision of view shadow Media Ob-

12

0-7695-0001-3/99 $10.00 (c) 1999 IEEE

12

Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999 Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999

views and arbitrary combinations thereof can be rendered.

Legend event handling

D distributed object L

DefaultTreeModel

traversal handling

JTree

Factory/Config.

provides model L

L

TreeModel

Renderer

L local object

L L

L

CMTTreeAdapter

ActivityObject

delegation

CMT Object

4.4.4 Other Components

D

D

Figure 8: Swing MV integration into COMMOTION at the client based on a JTree example

Due to lack of space, three other components are just briey described here:

 XML Mapper and Renderer : In order to provide a

exible environment that can interchange information with its environment, data interchange standards have to be taken into account. XML [13] is an emerging standard for structured information exchange. In the context of COMMOTION we make use of XML for exchanging instances of text media-types. For this purpose, we have built a XML mapper (which is built around DXP [14]) as a persistent mapper. The XML Mapper takes an XML document instance of a given DTD and validates it. After validation, it produces Physical Content and a Data Object hierarchy from the document instance. Moreover, it creates an Appearance Object from the CSS (Cascading Style Sheets [16]) description in case it is provided along with the DTD. We have also developed a simple COMMOTION DTD along with its style denition for common usage. Our XML renderer produces a valid XML and CCS instance from the Physical Content, Data Object and Appearance Object, which can be displayed by an XML browser outside the system.

jects, which stand for JComponent s. The second is the provision of model adapters, which map the models of non-trivial JComponents into COMMOTION. Currently, Media Objects provided for GUI elements are JButton, JComboBox, JInternalFrame, JTextField, JList, JMenu, JTree and JTable. Model adapters are provided for the following model interfaces: ListModel, TreeModel and TableModel. In our current implementation, models do not dispatch event handling, it is handled by a default Swing model implementation (e.g. DefaultTreeModel ). Figure 8 shows the Swing/COMMOTION integration by means of a JTree example. The integration of JMF is a coarse grain integration, since the only the DataSource interface as model and the Player implementation as view is considered. this is due to the fact that the current version of the JMF specication does not yet allow for ner grained access to player internals. JMF integration is part of the Resource Submodel in COMMOTION. The implemented Resource Objects are the NetworkSocketDevice and the  ODBC Mapper : The ODBC mapper is a transient VirtualMemoryDevice. Due to limited access of hardmapper that establishes connection to relational ware resources through Java, further resource compodatabases. This mapper is a good example how to nents could not be implemented. incorporate sources to the media framework, which Currently, GCB allows for read-only browsing of have no direct connection with media information. COMMOTION schemas. The visualization has three The ODBC mapper can map relational tables into elements: Data Objects and their properties, but also into Data Objects and Physical Content. The Data  relationships: GCB visualizes CORBA object Objects can then be displayed using a Media Obrelationship-graphs by spawning trees in the graphs ject with a suitable Appearance (e.g. JTable from according to several criteria and visualizing them Swing). using JTree.  attributes and properties : Attributes and proper5 Related Work ties are visualized by JTable.  media content : The physical content, the structure Due to its cross-domain nature, a comprehensive and the apperance is rendered as media content. analysis would grow out of the scope of the current paCurrently text, video, the above mentioned Swing per. In this section we only focus on CORBA based 13

0-7695-0001-3/99 $10.00 (c) 1999 IEEE

13

Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999 Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999

approaches by giving some examples of several elds that resemble our work or some aspects of it. There are several CORBA based multimedia streaming approaches similar to ours. Oracle's Media Net (OMN) [17] is based on a proprietary API, the Lancaster Multimedia Group puts streaming into the context of ODP (Open Distributed Processing) standards [18]. The A/V Streaming Service of the Object Management Group also has some implementations [19][20]. The common feature of these approaches is that they deal with streaming as a separate service that can be independently applied to CORBA-aware applications. This policy follows the principles of independent, interoperable services, however, semantic coherence with other components or even with other resource aspects must be provided by layers above. Other approaches include the presentation of multimedia documents using CORBA. The W3 Consortium has dened the Document Object Model, DOM [15] as a programming API for HTML and XML documents. The PREMO (Presentation Environment for Multimedia Objects) [21] ISO standard denes another object model for streaming, rendering, presenting and interacting with media content. Despite that the DOM deals with the structure, whereas PREMO deals with media content, a common feature of the these models is that they dene the models from a programming point of view, without covering their integration into a persistent, distributed context. Moreover, specifying the true inner semantics is beyond their scope. This is due to the fact that the semantics is implicit in the languages (DOM) and in the media content (PREMO). The provision of IDL interfaces for these models is solely convenience and there is no strong commitment to use a specic object binding runtime. A wider media management context is represented by the Kangaroo [22] and Dsmily [23] projects. Kangaroo is a Media Object Server. Dsmily is a CORBA based retrieval system built upon Kangaroo. Dsmily provides querying and retrieval on distributed Media Object Servers. However, no attention is paid to the integration of standard CORBA services, especially Query Service, and it is not clear how a streaming protocol could be integrated.

CORBA implementation strategy that is suitable for realizing the model in a distributed environment. From the architectural point of view, we have chosen for CORBA, because it is a exible and powerful integration technology that can be used to build componentware systems. From the design point of view, we realized that in order to build a semantically coherent interoperable system, all components must share a common model. This common model in our case is COMMOTION. From the implementation point of view, we identied our goals what concerns the server as making COMMOTION persistent, reusing ODBMS functionality, a CORBA implementation and service specications, and what concerns the client as reusing existing local submodels. In general we learned that while CORBA gives great exibility in both design and implementation, the many alternatives make it sometimes dicult to nd the best choice. With the integration of CORBA services we provide exibility to the client. While this proves as suitable for the system developer, the application developer might want to use more constrained interfaces for more ecient application design. We realized that in order to retain both exibility and the semantic constraints, we have to provide a generic and a specic COMMOTION prole, with the possibility of the specic reusing the generic. Our main policy was to apply the smallest possible change in the server. This has been achieved by (1) utilizing smart proxies in the client, so that specialized local IDL implementations can map to the generic prole, (2) using an attribute-property mapping in the server, so that the object can incorporate new IDL attributes by storing them as generic properties without changing the object's persistent state description, and (3) separating the behaviour and the state of the persistent object, so that even if its behaviour changes, its persistent state description does not have to be modied. At the client side we found that if we do not want to design and implement user interfaces, we have to integrate existing subsystems into COMMOTION. In case of Java, we could have used serialization, however, this would have restricted our client environment to Java. Therefore, we developed a generic mechanism to incorporate MV[C] type subsystems. Besides reinforcing se6 Concluding Remarks mantic coherence, our approach let local user interface This paper illustrates an approach to model multi- objects make use of COMMOTION services, especially media information management. It also illustrates a object persistence. 14

0-7695-0001-3/99 $10.00 (c) 1999 IEEE

14

Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999 Proceedings of the 32nd Hawaii International Conference on System Sciences - 1999

7 Acknowledgement We would like to thank Prof. Dr.-Ing. habil. Waltraud Gerhardt-Haeckl for her useful comments on the design and implementation of COMMOTION as well as on the content and structure of this paper.

References [1] Object Management Group, CORBA Specication 2.0, July http://www.omg.org/corba/c2indx.htm

The 1995,

[2] Object Management Group, The CORBA Services Specication (nalized specications), http://www.omg.org/corba/sectrans.htm [3] Object Management Group, Work In Progress, AV Streams RTF, http://www.omg.org /library/schedule/AV_streams_RTF.htm [4] Object Management Group, Work In Progress, Persistent State Service RTF, http://www.omg.org/library/schedule/PSS1.htm [5] Cattell, R.G.G, Barry, D.K. (eds), The Object Database Standard: ODMG-2.0, Morgan Kaufmann, 1997 [6] Oracle Corporation, Oracle Video Server Developer's Guide, Release 3.0.2, 1996-1997 [7] IONA PLC., Orbix Programmer's Reference, Version 2.3, 1998 [8] Martin, James et al., Object -oriented methods: a foundation; UML edition, Prentice Hall, 1998 [9] Gamma, Erich et. al, Design Patterns; Elements of reusable object-oriented software, Addison Wesley, 1995 [10] Javasoft Corp., Java Classes, Swing Edition, http://java.sun.com/products/jfc/

Foundation 1997-1998,

[13] The World Wide Web Consortium, Extensible Markup Language (XML) specication, dec. 1997, http://www.w3.org/TR/PR-xml-971808.html [14] DataChannel Inc., The DXP XML parser, 1998, http://www.datachannel.com/products/xdk/DXP/ [15] World Wide Web Consortium (W3C), Document Object Model Specication V1.0, July 1998, http://www.w3.org/TR/1998/WD-DOM-19980720 [16] The World Wide Web Consortium, Cascading Style Sheets, Level 1, dec. 1996, http://www.w3.org/TR/REC-CSS1 [17] Oracle Corp., Oracle Media Net, Programmer's Guide, Version 3.2, 1997 [18] G. Blair, J.B. Stefani, Open Distributed Processing and Multimedia, Addison-Wesley, 1997 [19] IONA Technologies PLC., Orbix MX, White Paper, April 1998 [20] Mungee, S., Surendran, N., Schmidt, D.C., The Design and Performance of a CORBA Audio/Video Streaming Service, Washington University technical report, WUCS-98-15 [21] ISO/IEC, CD 14478-X, Presentation Environment for Multimedia Objects (PREMO), 1996 [22] Marder U., Robbert, G., KANGAROO: Kernel Architecture for Next Generatrion Archive of RealtimeOperable Objects, 3rd International Workshop on Multimedia Information Systems, Como, September 1997 [23] Baumgarten, Ch., Meyer-Wegener, K., Building a Distributed Scalable Multimedia Information Retrieval System with CORBA, Wissenschaftliche Beitraege der Fakultaet Informatik, TU Dresden, Heft 1/1997 [24] Javasoft Corp., Java Media Framework 1.0.5, http://java.sun.com/products/java-media/jmf/, 1997-1998

[11] Orfali, Robert et al., Client/Server Programming with Java and CORBA, Wiley, 1997 [12] POET Inc., POET 5.0 Database Suite, 1997-1998, http://www.poet.com/products.htm 15

0-7695-0001-3/99 $10.00 (c) 1999 IEEE

15