Multimedia Ontology-Driven Architecture for Multimedia Systems Ernesto Exposito1,2 , Jorge Gómez-Montalvo1,2,4 ,Myriam Lamolle3, 1 CNRS ; LAAS ; 7 av. du Colonel Roche, F-31077 Toulouse, FRANCE 2 Université de Toulouse; UPS, INSA, INP, ISAE; LAAS; F-31077 Toulouse, France 3 LIASD/LINC, IUT de Montreuil, Université Paris VIII 4 Facultad de Matemáticas, Universidad Autónoma de Yucatán, México {jorge.gomez, ernesto.exposito}@laas.fr
[email protected] Abstract - Diversity of standards, languages, protocols, and hardware components leads to important incompatibility issues when designing and developing multiplatform multimedia systems. Furthermore, user and community requirements and preferences should be taken into account when instantiating and configuring these kind of systems. This work introduces an ontology framework for distributed multimedia systems design and deployment driven by user requirements. Case studies illustrating how distributed multimedia systems can be dynamically deployed are presented in order to demonstrate the advantages of using this framework. Keywords: Model Driven Architecture, OntologyDriven Architecture, Multimedia systems.
1. Introduction The accelerated development of Internet and the high diversity of networked devices available in the market have led to the fast development of networked multimedia applications (e.g., IPTV, VoIP, VoD, videoconference, etc.). These applications are usually designed and developed following specific platforms approaches. With the recent explosion of social networks popularity, new requirements for fast integration of heterogeneous software and hardware architectures in order to design new multimedia systems have appeared. Furthermore, in contrast to the well-known multimedia applications, spontaneous social networks claims for innovative human interfaces guided by users' requirements and allowing complex configurations of multimedia information exchange between fixed, mobile and ad-hoc communities of users. These new requirements lead to two major challenges: interoperability and dynamicity. Firstly, current multimedia systems are highly heterogeneous. In fact, multimedia applications are implemented using various languages and frameworks (e.g., Java, C/C++, .net), software environments (e.g., J2ME, Android, iPhone, Java Media Framework (JMF) and windows mobile) and a diversity of protocols are used to control, represent and exchange multimedia data (e.g., RTP, HTTP,
RTSP, SIP, SAP, TCP, UDP, etc.). Likewise, specific hardware platforms as well as network characteristics such as security, firewall, NAT, mobility and quality of service have an important impact in the time-constrained multimedia sessions and need to be taken into account in order to guarantee final users satisfaction. Secondly, configuration and deployment of distributed multimedia systems should be driven by the user requirements in order to facilitate the spontaneous establishment of multimedia sessions, thus allowing the exchange of information between the members of the social networks. In this way, a large number of new multimedia systems resulting of the composition of multimedia functions deployed over the adequate protocols, services and platforms and guided by user and community requirements could be dynamically developed as illustrated in Figure 1.
Figure 1. Multimedia systems design Several research studies have been carried out in order to propose standard frameworks and platforms for designing and developing multimedia systems. Likewise, a large diversity of development platforms for fixed and mobile devices is being constantly released. However, even when relevant efforts of standardization have been carried out, a generic multimedia framework aimed at integrating the basic distributed multimedia concepts has not been defined yet. Such generic multimedia framework should be able to represent the various configurations of
distributed multimedia systems and should help to automatically configure and deploy new systems driven by the user and community requirements. This work proposes the Multimedia OntologyDriven Architecture framework (MODA). This framework is based on well-known standards integrating multimedia, distribution and platform specific ontologies. MODA also includes an automatic process based on the use of these ontologies allowing dynamic translation between user/community requirements and multimedia components deployment. Deployment specifications produced using MODA can be used to self configure multimedia communication systems. Furthermore, these specifications could also be used to deploy the adequate communication components at middleware, transport or network layer, i.e. in order to provide the quality of service required to guarantee the multimedia session requirements. This paper is organized as follows. Second section introduces model driven technologies and multimedia frameworks providing the basis of the MODA framework. Section 3 presents the MODA framework, the basic ontologies and the model driven process for multimedia components deployment. Section 4 presents several case studies aimed at illustrating how distributed multimedia system can be dynamically deployed following this approach. Finally, Section 5 presents the conclusions and perspectives.
2. State of the art In this section the state of the art providing the basis of the MODA framework is introduced. The objective of a Model Driven Engineering (MDE) approach is to shift the complexity from the implementation of an application to its specification. Thus it is a question of making an abstraction of the programming language using an abstract modeling process focused on the use of several standards such as Meta Object Facility (MOF) [1], Object Constraint Language (OCL) [2], and Unified Modeling Language (UML) [3]. Model Driven Architecture (MDA) [4,5] is a specific field of the MDE, which specifies three levels architecture: • Representation of Computation Independent Model (CIM) from a business model point of view. CIM describes the context in which the systems will be used. • Representation of Platform Independent Model (PIM) from CIM. It describes the system itself without any details of its use or its platform. A PIM will be suited for one or several real architectural platforms. • Representation of Platform Specific Model (PSM) from PIM. At this level, the environment of implementation platforms or languages is known.
MDA allows designing a process based on different mappings from CIM to PIM and from PIM to PSM. The mappings can be automated, particularly when a model type mapping specifies the mapping rules in a meta-model. In this manner, an MDA approach increases the interoperability in heterogeneous environments and provides a method for systems integration using generic mapping engines. In recent years, the W3C Semantic and Web Best Practices and Deployment working group (SWBPD) [5] has proposed to extend the MDA methodology by using semantic models or ontologies (e.g., RDF and OWL languages). This extension has been defined as the Ontology-Driven Architecture (ODA) methodology. ODA is aimed at extending MDA by providing representation of unambiguous domain vocabularies (e.g., requirements, constraints, services, properties, etc.), model consistency checking and validation as well as to enable new automatic software engineering capabilities. Several works have been carried to illustrate how the ODA approach can be used in the design, development and management of distributed systems [6,7,8,9,10]. These works propose the use of ontologies to represent knowledge about platform diversity and how this information is used to perform safe configuration of refinement transformation between the platform models. Systems produced following MDA and ODA approaches can get major benefits in terms of flexibility and extensibility when they are designed following a service oriented approach. Indeed, Service Oriented Architecture (SOA) and Service Component Architecture (SCA) [11,12] approaches can be used for designing applications focused on services composition and orchestration which is the specification level required by PIM models. SCA provides a set of specifications, which support the development of applications based on SOA. SCA proposes organizing in components the development and deployment of applications. Each component offers services and may need references (services of other components). The assembly of the components by interconnecting (wiring) services and references indeed allows building applications whether they are distributed or not, running in one process or multiple processes. In addition, SCA allows the integration of heterogeneous components written in many languages like Java, C++, PHP, JavaScript, and BPEL.
3. MODA framework In the previous section, a state of the art of model driven oriented methodologies as well as service component oriented architectures has been introduced. Likewise, standard multimedia frameworks as well as legacy multimedia protocols
have been presented. Moreover, the diversity of languages and developing environments for the implementation of multimedia applications has also been presented. This section introduces the MODA framework, which aims at automatic deployment of networked multimedia systems driven by user requirements. As illustrated in Figure 2, users can request for a distributed multimedia system (DMS) at the CIM level using well-known multimedia standards (i.e. ITU-T F.700). This DMS instance will be then mapped to the PIM level by using multimedia protocols, distribution models and service oriented approaches in order to allow dynamic translation between the user/community requests and a platform independent deployment specification. This specification will be finally mapped at the PSM model by using specific platform factories able to translate the DMS specification into the specific multimedia components.
• Time Continuity. This class allows expressing if the communication task is buffer capable or not. • Media Interrelation. It allows specifying if there is some synchronization between media (e.g., lips or subtitles synchronization), or symmetry in order to indicate bidirectional of the same media type, or conversion between media in order to indicate when a media is converted into another type of media, i.e. when graphics are converted into still pictures. In Figure 3, we can observe the relations between the main classes of our ITU-T F.700 ontology.
Figure 3. ITU multimedia services ontology
Figure 2. MODA framework
3.1. Ontologies 3.1.1. The ITU multimedia services ontology. In this section we describe the ITU-T F.700 ontology. As we mentioned before, an important part of F.700 is the description and construction of multimedia services based on their communication tasks. The ITU-T F.700 ontology is focused on the description of communication tasks through their attributes. In this manner, a communication task has: • Communication Configuration. This class is used to express if the communication is point-to-point, point-to-multipoint, multipoint-to-point, or multipoint-to-multipoint. • Symmetry of information flow. It allows specifying the direction in which the information is sent. • Transmission control entity. It allows saying who is controlling the transmission of the information (e.g., source and/or sink). • Communication delay. The type of delay supported by the communication task (e.g., real time) • Media. The media or medium, mandatory or optional ones, transmitted by the communication task; as well as the quality level of the media.
In the ITU-T F.700 ontology the communication tasks sending, conversing, conferencing, distributing, collecting, and receiving are subclasses of CommunicationTask class. We define all the relations (object properties) of the CommunicationTask class and its subclasses so we can describe individuals which correspond properly to any of the communication tasks. 3.1.2. IETF protocols ontology. Based on the various protocol standards specified by the IETF, a multimedia protocols ontology integrating session control, session description and transmission protocols has been elaborated (Figure 4). Individuals of this ontology are represented by applications implementing RTSP, SIP or SAP session control protocols in order to negotiate and establish the required multimedia session described by the implementation of the SDP protocol. For most of these applications, the RTP/RTCP protocol will be used for streaming the multimedia data between the distributed system components (e.g., server to client for a VoD system).
Figure 4. IETF protocols ontology 3.1.3. Service Component Architecture ontology. The SCA Assembly Model defines the configuration of SCA domains using composites, components and the artifacts which allow describing how they are
connected or linked. As a matter of fact there are no many concepts used in the SCA Assembly model in order to provide the SCA's programming model. So we have developed a SCA ontology integrating all of these concepts, relationships, and attributes. Figure 5 depicts the main classes our SCA ontology.
Figure 5. SCA ontology 3.1.4. Software implementations ontology. In order to facilitate the automatic translation to platform specific models using an ontology-driven approach, a multimedia specific platform ontology describing the components of generic multimedia systems has been elaborated (Figure 6). This ontology mainly specifies the two basic components to be deployed in every host participating in the multimedia session: the session and the media controller. The former is suited to describe, negotiate, establish and terminate the multimedia session between the participants. The second is in charge of the multimedia processing functions (i.e. capturing, coding/decoding, streaming, receiving, recording and presenting). In order to incorporate a specific platform to the MODA framework, individuals of that specific platform need to be instantiated. For instance, in order to integrate the JMF implementation, one JMF individual related to one or several session control components (e.g., RTSP server, RTSP client, SIP agent, etc.) and to one or several media controller components (e.g., Player, Processor, etc) has to be added.
case, our proposition of CIM presents networked multimedia systems taking into account the different standards’ ontologies. The PIM level uses a Service Component Architecture (SCA) model to obtain a generic description of networked multimedia systems from CIM. For the PSM level, implementation ontologies can be used to translate our PIM of networked multimedia systems. Figure 7 shows the MODA process to build automatically a multimedia networked system.
Figure 7. MODA Process The first step of our workflow is the transformation of OWL multimedia source instance through CIMtoPIM engine to produce the SCA instance. This transformation is performed based on the CIM instance and following rules to identified the adequate components and composite to be deployed in every end system, as well the adequate protocols to manage the multimedia sessions. Figure 8.a, 8.b and 8.c illustrates some rules guiding this transformation process. Sending, collecting and receiving functions are mapped to RTSP, SDP, RTP/ HTTP protocols for session management, description and data transport respectively (see Figure 8.a). Likewise, conversing and conferencing functions are mapped to SIP, SDP and RTP protocols for session management, description and data transport (see Figure 8.b). Finally, SAP, SDP and RTP protocols are selected for distributing applications (see Figure 8.c).
Figure 6. Software implementations ontology
3.2. MODA Process The MODA process includes the three levels recommended by OMG (see Section 2.1). In our
Figure 8.a. CIM to PIM mapping for sending, receiving and collecting systems
Figure 9. VoD study case
Figure 8.b. CIM to PIM mapping for conversing and conferencing systems
The MODA process (CIMtoPIM) allows generating from the Sending communication task a SCA domain including both server and client composites. For each composite, the SessionController and MediaController components are inferred. Following the MODA process (PIMtoPSM), the adequate session controller (i.e. RTSP server and client) as well as the media controller implementations (i.e. JMF media controllers) have been selected. Figure 10 depicts the PSM SCA individual for the VoD system.
Figure 10. SCA description of the VoD system
Figure 8.c. CIM to PIM mapping for distributing systems In the second step, PIMtoPSM engine takes as input the SCA instance and produces the deployment of specific platform components taking into account the mapping rules between OWL multimedia specific platform (Figure 6) and software implementation (e.g., JMF or Java ME). This transformation is carried out within the MODA framework by using platform specific factories able to translate a generic SCA specification of the distributed multimedia system into the specific components of the targeted platform.
4. Case study. Video on Demand (VoD) We have evaluated the MODA framework for the automatic deployment of a VoD system. In order to characterize such type of application, the MODA ontologies have been used to describe the communication tasks involved. Figure 9 shows the CIM individual describing our VoD application.
XSL templates used to implement the mapping rules guiding the MODA engines are detailed at [10]. In order to illustrate the possibilities offered by MODA, another study case including composition of communication tasks has been elaborated. As illustrated in Figure 11, a hybrid DMS offering distributing and messaging (i.e. conferencing) functions has been required by participants of a remote learning session. From the user requirements, a DMS at the CIM level has been specified including the distributing and conferencing functions. This CIM individual is then mapped to PIM and PSM levels and is finally deployed in the participants hosts. In the example, 4 participants are considered: the lecturer representing the source of the distribution and the remote participants able to receive the multimedia data distributed by the lecturer as well as to communicate between the other session participants.
Figure 11. MODA framework usage for complex user requirements (hybrid DMS) In order to better illustrate how this transformation between the CIM/PIM and PSM instances have been done, the Figure 12 illustrates several of the
components/composites being deployed over the various hosts.
Figure 12. CIM to PSM (hybrid DMS) At the CIM level, user requirements have been traduced as a hybrid DMS offering distributing and conferencing tasks. In order to provide these functionalities, several components at the session and media levels need to be deployed. At the session level the adequate protocols able to control the complex multimedia session for distributing and conferencing need to be selected. At the media level, adequate multimedia components allowing exchanging audio, video and text data need to be selected. Audio and video streams will participate within the distributed session and text data will be exchanged during the conferencing session. At the PIM level, the adequate protocols and multimedia components have been selected for specific JMF and J2ME platforms. At the session level, implementations of SAP and SIP protocols have been selected in order to manage the distributing and conferencing sessions. At the media level various composites connecting input, controllers and output multimedia components have been assembled. For instance, for the distributed session two kinds of composites can be produced. The first one aimed at being deployed at the lecturer host and including inputs represented by audio/video capture components, controllers represented by codec components and output represented by network sending components (i.e. RTP stream producers). The second one is intended to be deployed at each participant host and includes inputs representing the network receiving components (i.e. RTP stream consumers), controllers represented by codec components and outputs represented by render components (i.e. screen for video data or sound outputs for audio data). Likewise, for the conferencing sessions, composites representing the production and consumption of text data at each participant host need to be produced.
5. Conclusions and perspectives In this paper, the Multimedia Ontology-Driven Architecture framework (MODA) has been introduced. This framework is intended for automatic networked multimedia systems deployment and configuration guided by the user
requirements and preferences. This framework is based on well-known standards and proposes a set of ontologies integrating multimedia, distribution and platform specific concepts. Furthermore, this framework also integrates an automatic process aimed at guiding the dynamic generation of multimedia system deployment configurations. The use of this framework in order to allow dynamic translation between the user/community requests and the components deployment specification has been illustrated by the instantiation several distributed multimedia systems. This study cases illustrate the benefits of following such approach to automatically deploy hybrid multimedia services for community of users. MODA framework can also be used for other activities such as self-configuration of the communication system (e.g., firewall, NAT, etc.). Likewise, deployment of the adequate communication components at middleware, transport or network layer in order to provide the quality of service required to guarantee the multimedia session requirements can also be envisaged using the MODA framework. More information about this framework can be found at the MODA website [13].
6. References [1] http://www.omg.org/spec/MOF/2.0/ [2] http://www.omg.org/spec/OCL/2.0 [3] http://www.omg.org/spec/UML/2.2 [4] J. D. Poole, “Model-Driven Architecture: Vision, Standards And Emerging Tech”, in Workshop on Metamodeling and Adaptive Object Models, ECOOP 2001 [5] http://www.w3.org/2001/sw/BestPractices/SE/ODA [6] T. Sloan, "Business Model Collaborations: Pushing Software Revolution." Software Magazine, September 2003. [7] A. Kleppe, J. Warmer, and W. Bast, MDA Explained: The Model Driven Architecture Practice and Promise. Addison Wesley, 2003 [8] Oberle Daniel et al., Developing and Managing Software Components in an Ontology-based Application Server. Middleware 2004, ACM/IFIP/USENIX 5th International Middleware Conference, Toronto, Ontario, Canada, volume 3231 of LNCS, pp. 459-478. Springer, 2004 [9] Daniel Oberle, Steffen Staab, Andreas Eberhart, Towards Semantic Middleware for Web Application Development. IEEE Distributed Systems Online. February 2005 [10] D. Wagelaar, "Platform Ontologies for the Model-Driven Architecture", Thesis dissertation, Faculty of Science of the Vrije Universiteit Brussel, 2008. [11] SCA Project, at http://www.osoa.org/, accessed 11/03/2009 [12] D. Chappel. Introducing SCA. Technical report, Chappell & Associates, 2007. [13] Multimedia Ontology-Driven Architecture framework, http://homepages.laas.fr/eexposit/pmwiki/pmwiki.php/MODA/Ma in, accessed 11/30/2009.
