2012 IEEE International Conference on Multimedia and Expo Workshops
A Model-driven Approach for Integration of Interactive Applications and Web Services: A Case Study in Interactive Digital TV Platform Raoni Kulesza, Silvio R. L. Meira Informatics Center (CIn), UFPE Recife, Brazil [rk,srlm]@cin.ufpe.br
Thales P. Ferreira, Eduardo S. M. Alexandre, Guido L. S. Filho
Manoel C Marques Neto Celso A. S. Santos
Informatics Center (CI), UFPB João Pessoa, Brazil [thales,edu,guido]@lavid.ufpb.br
DMCC/UFBA Salvador, Brazil
[email protected] [email protected]
developing this type of application should consider the involvement of specialists in (i) software design, (ii) interface design and (iii) graphics and media production. Currently, it is possible to identify a gap in the definition of processes, methods and tools to help the systematic development of multimedia applications integrating these three aspects [12]. The Model Driven Development (MDD) is one of the approaches that seek to reduce the complexity of building interactive multimedia applications [10]. The MDD allows the transformation of a high-level model design into a description of an abstract set of elements that compose it and then, the transformation of these elements into code for a specific platform. The main benefits of this approach are: (i) inclusion of an intermediate phase (project) between the requirements specification and implementation, (ii) improvement of development team communication due to the existence of representations of the system at various levels and (iii) automation of code generation through the use of transformation engines between the models and tools for editing the models. However, using this approach in the field of multimedia applications is still limited [14]. Applying concepts of MDD to design interactive multimedia applications, StoryToCode model [9] has proposed an approach with focus on component reuse, however, it leaves open a classic problem of systems engineering: the requirements structuring for reducing the gap between media design professionals and software on the making of high-level models. The main consequence is the increase of rework (by the software team) in adjustments to the model before its final approval (by the media production team). Another StoryToCode related problem is the amount of rework needed to finalize the code generated from the created elements diagram. This occurs because the elements diagram which represents an abstraction of the modeled application requirements does not include instance information. This diagram describes only the information structure that must be present in an application and not the instance values of this structure. The StoryToCode also does not model features on the latest applications, which use, for example, Web services, since their model focuses on the information’s structure and not on treating dependencies or external behaviours that may affect such a structure.
Abstract—This work proposed a model-driven development approach related to interactive multimedia applications and Web services integration. It is based on extension of an existing modeling language, which integrates modeling concepts for interactive applications and adds support for Web Services. Three Interactive Digital TV applications were modeled and developed. As we show, the evaluation of the approach brought benefits not supported by related works, like requirements structuring and reducing amount of work needed to finalize the code generated. Keywords: Model-Driven Development, Template-based Authoring, Digital TV, Broadband TV, Web Services.
I.
INTRODUCTION
Digital convergence is a process that can be seen from two different points of view:(i) as a marriage of technologies or industries (Internet, telecoms and digital television networks, for example) and (ii) the union of different types of media through an unique technology (multimedia services and applications access using a Web browser). On the other hand, parallel to the evolution of interactive multimedia applications, Web applications come with "mashups" that aggregate content or functionalities from Web services. Such scenario can also be observed recently in the field of interactive multimedia applications, through TVs with direct access to the Internet, the so called Broadband TV [1]. An interactive multimedia application can be defined as software that has a GUI with at least one media object (audio, video, image or text) and that are closely linked to the application logic. This implies that the relationship between the media objects may include: (i) creation, deletion, alteration of media objects from the application’s logic at runtime, and (ii) generating events of media objects propagated for the application logic. Moreover, the design of these applications must also involve a process of media production [5]. The development of interactive multimedia applications is still a challenge. In addition to business logic, multimedia applications typically offer a graphical interface that is both sophisticated and integrated with media objects (images, 3D graphics, audio and video). Consequently, the process of 978-0-7695-4729-9/12 $26.00 © 2012 IEEE DOI 10.1109/ICMEW.2012.52
266
To overcome some of the StoryToCode drawbacks, this paper proposes an evolution of this model through three additional elements in its architecture: (i) classifying a set of applications to specialize the generated models, (ii) using of templates oriented concepts of authorship to increase speed, simplicity and efficiency of development and (iii) using of modeling tools and code generation to facilitate the construction of applications. The goal is to improve the development process through better structuring of requirements that allows: (i) standardizing the structure of the requirements through the use of templates, (ii) including instance information important to improve code generation process; (iii) reducing rework in the design and final coding of interactive applications and (iv) reducing the gap between the TV professionals and the software developers. iTV A case study of the implementation of Interactive TV applications is used for evaluating the proposed approach. In this case study, the following steps were performed: (i) definition of a taxonomy of applications into three categories (push, pull, distributed), including integrated applications with Web services; (ii) modeling of the templates from the types defined in (i); (iii) code generation and instantiation of templates. This paper is organized as follows: section 2 discusses related work. The Section 3 details the proposed approach. The fourth describes a proposal evaluation through a case study with three iTV applications. The fifth and final discusses the conclusions and future work. II.
StoryToCode [9], it aims to apply some of the best practices of software development in a specific area, including three additional features: (i) taxonomy of applications (process scope), (ii) use of concepts of templates oriented authorship (speed, simplicity and development efficiency) and (iii) use of modelling tools and code generation (partially automates the construction of applications). The approach dynamics starts from requirements extraction from a high-level model (such a storyboard and/or scenario description) and use templates oriented authorship concepts and system modeling to create a set of elements that make up an multimedia application. These elements represent both the media and software components. This creation process is done to highlight systemic views (structural, scenes, media objects, user interaction, user interface and so on.) In order to make development faster, easier and efficient, besides allowing the reuse of artifacts generated in other contexts. Thus, as the StoryToCode, the architecture proposed here is inspired by the MDD and is divided into 4 stages related to g each other, as shown in Figure 1.
RELATED WORKS
The paper [12] has focused on developing interactive multimedia applications by specifying the MML (Multimedia Modelling Language). The main goal is to allow the process to involve different experts belonging to the three types of existing projects: software design, graphical user interface and media production. The work proposed here uses UML, adopting it’s models in a MDD approach. In addition to using MML language in a development methodology, which creates better conditions through the use of templates and semi-automatic generation of code through a tool, we could also validate the language in previously unexplored scenarios: Digital TV applications integrated with Web services. The NCL Eclipse plug-in [2] is an IDE for coding iTV NCL applications. It is an open source text editor for NCL integrated into the Eclipse environment. However, the plugin does not provide functionality to support the code reuse or even the more abstract design specifications. The focus is NCL code generation of an interactive application by programmers. The use of the tool assumes that all functional and non-functional requirements of the program are already specified and does not provide modules to support this specification. III.
Figure 1 - Approach Overview
The dynamics of this architecture defines how the transformation of Storyboards/Scenarios in a set of abstract elements should be done, and then the transformation of these elements into code. It is important to note that the steps are in sequence, but you can go back to the previous step in any part of the process to refine a set model. The first stage aims to define requirements and applications usage scenarios, through the narrative descriptions in storyboard form, which are usually defined by the media production team (e.g. TV crew, games and graphics designer and so on). From there, it is possible to define the scope by classifying a set of elements of an application. The generated artifacts are known as CIMs (Computation Independent Model), since they are independent of computational representation. In the second stage, Abstract Templates are generated. They represent PIMs (Platform Independent Model) developed from the categorization of the applications. Among the advantages of the use of templates and the taxonomy it is possible to highlight: (i) consistency between applications (branding), where you can define and follow the same pattern for structure, visual identity etc.; (ii) the definition of a set of applications that allows to easy indexing and grouping other applications; (iii) as a consequence of (i) and (ii), there is also the possibility of increasing the reuse and; (iv) despite demanding an initial effort to establish a
APPROACH OVERVIEW
This section describes the proposed model-driven development approach, which supports the construction of interactive multimedia applications. Inspired by
267
IV.
common set of applications, once this work is performed, the development of applications based on the templates becomes faster. It’s noteworthy that this set of elements is not only a document to aid in understanding, maintenance or evolution of software, usually found in the conceptual models for software specification. This set is also an artifact that can be compiled (converted) directly to PSMs (Platform Specific Model). To make this possible, the construction of the set of Abstract Templates should use a standardized and unambiguous notation in order to allow its transformation into code for different platforms. For such a modelling language with a level of abstraction that is independent of technology and even programming language should be used. In addition, due to the need to model not only the applications software design, it is interesting to use a language or a set of them that support more than a systemic view of applications in the field of interactive multimedia applications, we suggest the adoption of the MML language [11], discussed in section 4.2. The third step is to turn Abstract Templates into Concrete Templates. These represent the PSM models, which are associated with a paradigm and programming language and are specializations of the PIM templates. Although it is impossible to map the PIM models for all technologies represented by the PSM, the structuring Abstract Templates on a specific and limited set (the taxonomy) facilitates this transformation. On this stage instances information are filled as well, since the same Concrete Template can generate many different applications. The last step transforms Concrete Templates with instance information into an application’s source code for a particular target platform through a transformation. Each application instance contains a set of transformation rules, based on the based knowledge (Concrete Templates), and the structure of the destination elements (code for a specific platform). Thus, a single set of elements can be transformed into code for different platforms. For this, there must be a transformation module for each specific platform. As illustrated in Figure 1, all these steps must be supported by the adoption of a tool. This tool should offer simple and intuitive techniques for building an application in order to hide the implementation details of modelling and programming languages. The main advantages of using the proposed approach are: • Decreasing the responsibility of the content generator through the decentralization of production steps that are outside the realm of its original work: the specification and implementation of a software artifact. • Reducing the effort spent during the production process of the interactive components. • Allowing the reuse of software components in several domains. • Reducing the amount of code programmers need to make when creating the application. • Automating (partially) the generation of application code from a consistent set of templates. • Allowing the participation of other actors (systems analysts, programmers, designers etc.) in the production chain of an interactive multimedia applications.
CASE STUDY: IMPLEMENTATION OF ITV APPLICATIONS
Besides significant improve in video and sound quality, a digital TV system also allows the evolution in the way people access content through the possibility of integrating software and audiovisual data flows transmitted by the broadcasters. This innovation creates opportunities for production and development of applications of all kinds, for example: electronic program guides, entertainment (games) and educational applications, video on demand (VoD), government and e-commerce applications among others. Such services impose the same development challenges mentioned above, since they demand the definition of new productive processes that add the features of TV production (media objects) and software [7]. This section aims to assess the approach in the context of developing applications for iTV, more specifically, dealing with a specific case of applications: those sent to receivers by a TV station along with their programs, using the channel that is under your control and platform with or without a communication channel with the a TV station or the Web. A. Applications Taxonomy An important factor to improve the production process of iTV applications is reducing the scope of the existing possibilities diversity, until recently unknown, since there are new formats and modes of interaction every day. Since it is impractical to generate all types of templates, the proposed approach aims at organizing the characteristics of the applications according to a structure that covers the main usage scenarios of iTV services. Based on [3], applications can be categorized into three categories: pull, push and distributed. This taxonomy consider anything in iTV is a story for immersing people into the narrative flow, creating virtual communities, communicating with each other and obtaining multimedia assets in a natural and guided form. The goal is to address the major usage scenarios for interactive iTV services and provide technical platform for narratives. Push Mode: groups the relevant programs to the perspective of traditional suchlike TV. Only the linear narratives and those with limited options are considered. Altering the flow of the narrative development (known as branching) is obtained by adding parallel stories within a video or using more than one stream in the transmission. At first, the broadcaster does the change of the narration and in the second, the viewer makes the decision to change as he switches the flow to follow another point of view of the current story. This model allows local navigation and interactivity through interaction devices such as remote control. The response of the interaction with the network is always indirect, using telephone, SMS, fax, e-mail, among others. The content and application data are limited to what is pre-televised and broadcast via a unidirectional channel (diffusion) controlled by the broadcaster (hence the name push). Examples of these types of applications that have been defined are: (1) "Cul-del-sac": applications with linear narratives with no temporal relation to the main content, (2)
268
"TV Voting" applications with interactivity through indirect channel with the station (telephone, SMS, email, website etc.), (3) "Add-On" applications that complement and are synchronized (time dependence) and sent in parallel with the main content, also known as "Enhanced TV" [4], (4) "Y": applications that change the sequence or flow of the main display of the content. Pull mode: the difference of this mode compared to the previous one is the presence of direct and bidirectional communication with the station. As a result, there is the possibility of individual content delivery, unlike the push mode, where the interactive content is collective and all users receive the same opportunities for interaction. This aspect is of fundamental importance in the applications’ narrative building possibilities because it is now possible to create more elaborate branches. In this context, three options are possible: (i) Branches with mandatory paths, (ii) Branches with bottlenecks and (iii) Dynamic Branches. Although allowing more elaborate ways to interact with the content and storytelling, this set of applications reuses existing structures in several instances of the push category. However, all these new paths could be sent individually by the station through the return channel (e.g., new stages of a game), saving the receiver resources and not limiting the amount of allowed options. Thus, the types of the pull category applications are: (1) "Cul-del-sac ReturnChannel" (2) TV Voting ReturnChannel, (3) Add-On ReturnChannel and (4) Y App ReturnChannel. Distributed Mode: In this category fall the more recent applications that exploit two traits also found in Web 2.0 applications: (i) location and distributed processing over the Internet or other elements’ devices that can comprise the interactive application (for example, a web service) (ii) collaboration among users in the construction of the narrative. The goal of the first characteristic is to develop applications that take advantage of the network effects, both to become better, as well as more popular. In this case, the most important benefit is adding content, features or entire platforms already available on the network. Examples of this type of applications could be found in theses works [4][7][8]. According to [14], this application category is characterized as an evolutionary narrative application, since the final state of implementation of this type of application is non deterministic, whereas at no point in time you can determine it. In this work, to make feasible the modelling of this type of application in the study were treated only iTV applications that contains a Web service integrated with its content and that originate from pull applications. Thus, in this category the supported applications are the same as pull, but they may have the following modules: (1) WS TVConsumer: uses information from a Web service, (2) WS TV Producer: produces information for a Web service and (3) WS TVProsumer: produces and consumes information for a Web service.
interactivity, there may be many ways of watching the same content. In other words, the program is not determined by a single timeline, but a core set of events and other secondary that depend on user interaction. This feature requires a production process that must be fast, efficient and simple [14]. The speed is associated with the demand to meet the competitive and dynamic characteristic of the construction of interactive programs. Efficiency is related to the need to implement new applications correctly. Now the simplicity becomes important to enter and to facilitate communication of the various job profiles existing in the multidisciplinary team involved in the production of the application. These characteristics can be targeted with the use of predefined templates applications. On the other hand, the existing modelling languages are not sufficient for development of multimedia applications, because they do not even cover the integration of media or the general aspects of user interface. Additionally, despite the traditional approaches of the multimedia area providing extensive support for media creation, they neglect the application logic and software engineering principles [13]. The MML language seeks to address the interdisciplinary nature existing in the development of multimedia applications. Through a language based on UML 2.0 specification, the MML seeks to integrate the different teams and produced artifacts that permeate the three different project types described above. To this end, the MML defines five types of models: (i) structural model, representing application entities (domain model, and media objects); (ii) scenes model, which helps user interface design, describing how media objects can influence the choice of different scenes (application screens); (iii) abstract interface model, which defines for each scene, the abstract elements associated with the user interface; (iv) media interface model, derived from abstract interface model and the structural model media objects, which is responsible for the integration of the media components with the abstract user interface and (v) interaction model, which specifies how the user interface events and media events can trigger calls in structural model elements In the proposed approach, we recommend using MML to generate Abstract Templates, that describe platform independent concepts and can be defined based on the information from CIM models, such as the Storyboard. To demonstrate the advantages of the MML language within the proposed approach, three templates modelling are used: TV Voting, TV Voting with RetunChannel and WS TVProsumer (section 4.1). To simplify, described below has only one template, since the first and second case is a simplified model of the third. Because of space limitation we only illustrate the use of some diagrams. The template of applications of the WS TVProsumer type consists in an application that sends and receives information from an external entity represented by a Web Service. According to MML, the first modelling to be performed is the structural, where the domain elements are evidenced and the related media.
B. Templates modelling On the one hand, the use of templates becomes even more important in the development of iTV applications, since the content is no longer necessarily linear. Through
269
element (WS TVProsumer) for dispatch confirmation. The Result encapsulates the image (see ResultBackGround Figure 3) that displays the output response to the user. C. Evaluation The tool implemented to evaluation modelling the templates based on the graphical interface visual prototyping technique. This technique has the major advantages of the approximation to the needs of the system users, the reduction of misunderstandings between users and developers and also the reduction in development effort [6]. Templates do not include information concerning the user interface (instance information). For example, a diagram (element set) that represents a template for a poll describes the structure of elements like Question and Answer. This description can inform that a question element contains an attribute that represents the text or image of a question and another attribute that represents a list of response elements. However, it does not exist, in this diagram, information defining what are the questions and answers (instances) of the poll nor does it inform how they are formatted (text font, screen position, color etc.). In other works this information is reported directly in the code after the transformation. In this proposal, this information is inserted into the Concrete Templates, facilitating an evaluation before the code generation by the media and user interface production team. This code is more complete and generates a smaller amount of (re)work Once the set of elements was created and validated, we can continue the transformation of this set of models (Concrete Templates with instance information) into code for a specific platform. This is easily possible because MML is defined as a standard-compliant metamodel implemented with the Eclipse Modeling Framework (EMF). The advantage of EMF is its integration with many other existing tools from model-driven engineering, like tools for model transformation, validation, or model weaving In short, each graphical model of the tool has a XMI/XML representation that is transformed into code for a particular programming language, an already used technique in several other studies [12] [14]. In this case study applications were generated for Ginga-NCL [15]. The evaluation of the modeling of the "Concrete Artifacts" and tool code generation was accomplished through the deployment of three applications in a broadcaster test environment capable of running Ginga-NCL applications. Such a scenario provides a condition almost identical to an actual production environment. The first application corresponds to a push Poll with no direct communication channel (the answers are sent by phone, the result is sent by the broadcast channel for everyone with the final outcome of the vote) and that represents an instantiation of the TV Voting template (Figure 4a). The second application corresponds to a push poll and TV Voting ReturnChannel template (Figure 4b), where the communication channel is direct (the response of each user is sent directly to the station through an Internet connection and the partial result is sent individually to each receiver). The last application allows a guessing competition for soccer
Figure 2 - WS TV Prosumer - Structural Model
In Figure 2, besides input elements for voting (VoteInput) and the voting result (Result) overlapping the main video (PrincipalContent), the elements related to Web services access are shown: ProduceWebService for the production of information and ConsumeWebService for consumption. Abstraction and modularization of these elements are intended to facilitate its customization on application, and allow the predefinition of a default set of services (e.g.. ProduceWebService for sending messages to Twitter, ConsumeWebService reception of messages from News Corp's, among many others). Another important aspect is the representation of the return channel (ReturnChannel) allowing to configure the access technology (type) and associate it with the application in a modular fashion.
Figure 3 - WS TVProsumer - Interaction Model
Figure 3 shows the relationship of the graphical interface elements with the execution flow of the Vote scene (this and other elements are represented by a MML scene individual diagram, not shown here for simplification). Two relationships should be stressed: (i) the elements of graphical user interface (keyPressed and TextInputArea), which relate to the inputKeyPressed operation to respectively receive and display the voting text and (ii) the need to relate the operation sendMessage with a graphical user interface
270
games, representing the distributed mode and WS TVProsumer template (Figure 4c). In this case, the application contains a Web service that sends a hunch to Twitter and receives the current ranking through another Web service connected to a application server.
predefined RESTFul services and (iii) use the methodology in other areas of multimedia applications in order to evaluate the development of templates and code transformation on other execution platforms and technologies (Mobile and Web applications). REFERENCES [1]
[2]
[3]
[4]
[5]
[6]
[7]
Figure 4 - Applications Developed
V.
CONCLUSIONS
[8]
This work proposes an MDD approach to support building interactive multimedia applications from taxonomy of the main usage scenarios, definition of pre-designed structures (templates) and a tool that uses visual techniques of prototyping and semi-automatic code generation. The adoption of the MML language allows addressing the multidisciplinary nature of interactive multimedia applications. As a first evaluation of the approach it was developed using examples of three types of templates (TV Voting, TV Voting ReturnChannel and WS TVProsumer) in building three iTV applications. One of the main benefits of using the approach was a better structure of the requirements. For example, the inclusion and configuration of Web services integrated with other elements of the application (application logic and media objects). The main difficulty encountered was the effort to model the templates. However, once completed this step, it was possible to inherit all the benefits of “template-based authoring”. In pursuing this work, the following points should be addressed: (i) carrying out of other studies (controlled experiments) to evaluate quantitatively (e.g.: speed, efficiency and simplicity) the approach usage, (ii) extension of the modeling language to address the semantics of
[9]
[10]
[11]
[12]
[13]
[14]
[15]
271
B. Alfonsi “I Want My IPTV: Internet Protocol Television Predicted a Winner”. IEEE Distributed Systems Online, 2005. R.G.A Azevedo, et. al. “Interactive Digital TV Integrated Development Environment for NCL Applications”. In 27th Brazilian Symposium on Computer Networks and Distributed Systems. p53–57, 2009. S. Bachmayer, A. Lugmayr, G. Kotsis “Convergence of Collaborative Web Approaches And Interactive TV Program Formats”. Int. Journal of Web Information Systems, 2010. T. Coppens, K. Handekyn, F. Vanparijs. “AmigoTV: A Social TV Experience Through Triple-Play Convergence”. Alcatel Technology white paper. 2005 G. Engels, S. Sauer “Object-oriented Modeling of Multimedia Applications” Handbook of Software Engineering and Knowledge Engineering Ed. Singapore: World Scientific, v. 2, p. 21–53, 2002. T. H Kaskalis, T. D. Tzidamis, K. Margaratis. “Multimedia Authoring Tools: The Quest for an Educational Package”. Educational Technology & Society, p.135-16, 2007. C.J. Le, et. al.. “Emotionally reactive television”. In Proceedings of the 27th International Conference on Intelligent User Interfaces, pp. 329-32, 2007. C. Mac Williams, and R. Wages. “Video conducting the Olympic Games 2008: The iTV field trial of the EU-IST project live”. In: Proc. 3th Int. Conf. on Digital Interactive Media in Entertainment and Arts, pp. 436-40, 2008. M. C. M. Neto, and C. A. S. Santos. “StoryToCode: A new model for specification of convergent interactive digital TV applications”. Journal of the Brazilian Computer Society. p3-21, 2010. Perovich, D., et al. “Model-Driven approach to Software Architecture design”. In: ICSE Workshop on Sharing and Reusing Architectural Knowledge. p.1-8, 2009. A Pleuss and H. Hussmann. “Model-Driven Development of Interactive Multimedia Applications with MML”. Studies in Computation Intelligence. Springer: v. 340, 199-218, 2011. C. S. Soares Neto, et al. “The Nested Context Language reuse features”. Journal of the Brazilian Computer Society, v. 16, p. 229245. 2009. M.F. Ursu, et al. “Interactive TV narratives: opportunities, progress, and challenges”. In: ACM Trans. on Multimedia Computing, Communications, and Applications, v.4, n.4, 2008. X. Yang, P. Gu, H. Dai. “Mapping Approach for Model Transformation of MDA Based on XMI/XML Platform”. In: 1st Int. Workshop on Education Technology and Computer Science. p.10161019, 2009. ITU-T Recommendation H.761, 2009. Nested Context Language (NCL) and Ginga-NCL for IPTV Services. Geneva, April 2009.
