collaboration meets digital television - fact or fiction?

1. S. Bachmayer, G. Kotsis, and A. Lugmayr. Collaboration meets Digital Television - Fact or Fiction? in Internet and Multimedia Systems and Applications (EuroIMSA 2009). 2009. Cambridge, UK: ACTA Press

COLLABORATION MEETS DIGITAL TELEVISION - FACT OR FICTION? Sabine Bachmayer Department of Telecooperation Johannes Kepler University Altenberger Strasse 69 Linz, Austria [email protected]

Gabriele Kotsis Department of Telecooperation Johannes Kepler University Altenberger Strasse 69 Linz, Austria [email protected]

ABSTRACT Web 2.0 is the keyword for web-based services during recent years. Consumers socialize with advanced web services such as Flickr, Facebook, and YouTube. They share, contribute, collaborate, network, and communicate through social media services instead of browsing the web as individuals. Traditional media, such as TV face the problem of how to cope with these rapid developments. Broadcasters are exploring possibilities to converge the traditional TV services with novel services coming from the Internet service community. One example is CNNs iReporter concept, where consumers can contribute with their audio-visual materials to live news broadcast. Other examples are reality shows, interactive gaming shows, or interactive polls. We perform a first analysis to which extend the shift from individual TV experience watching towards a more networking and collaborative form of TV can be envisioned. We present an initial idea for a more social and collaborative TV.

Artur Lugmayr Department of Signal Processing Tampere University of Technology Korkeakoulunkatu 1 Tampere, Finland [email protected]

A simple example are Video Jockeys (VJings), where the consumer contributes to the live production of video performances through his interaction with music. There is a direct communication and interaction with the music and the disc jockey [4]. In a digital TV environment on the contrary, people collaborate by using the television environment for performing a common task and reaching a common goal (see section 2 for examples). Nowadays television is still one of the major Europeans favored spare time activity [5], with an increasing trend towards Internet services. To cope with this increasing trend, broadcasters consider the convergence of their service spectrum with collaborative service types existing on the Internet as described the following. On the

KEY WORDS collaborative systems; collaborative TV; collaborative application; digital TV; interactive TV;

1 Introduction The movement away from the lone warrior to socialization and collaboration happened not only in the web. This trend is also visible in other entertainment areas, such as television is. The term “television” can be defined as a system for converting visual images (with sound) into electrical signals, transmitting them by radio or any digital medium, and displaying them electronically on a screen [1]. Within the scope of this paper, we see the TV as ecosystem of services for human collaboration, ranging overall the total value-chain from content creation to content consumption. To provide more collaborative services it is also essential to consider service architectures providing a more collaborative form of television, beyond set-top-boxes and TV sets. We envision the need for new hardware and software architectures to enable advanced types of collaboration and a more social medium. However collaboration, in this context, can be defined as working together with others for mutual benefit [2, 3].

Figure 1. Producer-Consumer-Chain for Internet and television in the course of time (P = Producer, C = Consumer)

media market, the value-chain evolved from a sequential producer-distributor-consumer chain towards a “valuechain-web”. The value-web is a web of links, where clearly distinguished roles of value-chain partners moved from their traditional ones to new forms. A crossover from the typical producer - consumer

- chain into social networks (1. and 2. in Figure 1) happened, where the consumer becomes content producer and contributes via social networks. In contrary, in the field of television, this trend propagates more sneaky. Although there exists growing research activity in digital and interactive television, in practice the typical producer - consumer - chain is still prevalent - with the producers and broadcasters on the one side and the consumers on the other side (3. in Figure 1) - strictly separated. Nevertheless, a turnaround was recognizable also in this area with those kinds of game shows, reality shows, home shopping, etc where passive viewers became active in the way of participating directly in the show or by voting procedures via the phone, Internet or mail. This is the first generation of so called interactive television (4. in Figure 1). The great success of these shows (which is pointed with their growing number of broadcasts) demonstrates the willingness of the consumers getting more involved - becoming more active and getting influenced on the progression of the TV programme. The next step to 5. in Figure 1, to networked television where the role of producer - consumer becomes more and more blurred, is still open and in research. With this publication, we elaborate in a first step in which fields of television the trend of more collaborative service already took place. We attempt to answer the question “is collaborative TV fact or fiction?”. In the second part of the paper we focus more on the technical solutions for collaborative TV systems. We especially identify a niche for a TV system. The work reported with this paper constitutes the initial phase of a PhD project. Paper Outline: We will show observable trends (section 2) followed by an outlook to future digital TV based on collaboration (section 3). Finally we suggest a conceptional model for a collaborative TV system. Conclusions close the paper.

2 Observable Trends Even if the combination of collaboration and television couldn’t yet find its way into most households, trends are observable in different areas as the following:

• Interoperability of Hardware: Collaboration of TV environmental hardware as it was done with the Grid TV environment. This project connects set-top-boxes to a grid environment to provide an autonomous, efficient and independent management of the amount of resources available in the TV environment. Goal is to simplify sharing information stored inside of each TV channel as well as the heterogeneous computational resources. [6]

• Interoperability of User & Hardware: Collaboration between the user and their equipment to enable interaction with content, as the LIVE project does. It enables viewers to shape their own screening material during live events [7]. The show is adapted in response to the consumer feedback, allowing viewers to switch between stories (sub-channels) according to their mood and interest. For “inviting” the viewer to another sub-channel, so called switching-points are created1 . • Convergence of TV and Web 2.0: Collaboration of TV environments with concepts like Web 2.0 services. This connection is done in A3 TV [8] which allows the composition of a TV channel with any Web 2.0 service. Thereby concepts like the combination of channel and service with respect to the content broadcasted in a channel. For example a talk show with a chat service is meaningful for allowing discussions between viewers and the guests of the show. • Collaborative Content Contributing Consumer: Collaboration for content creation as it was realized with the “Wreck A Movie” (WAM) project. WAM allows the collaborative production of a movie using an online platform2 . Thereby, the producers are Internet users making components for a certain movie. A production leader picks out the best components to be used in the final movie. Individuals may specialize to small things but the mass of users with different talents enables the films to be produced with much lower costs than in the blockbuster model [9]. • Specific Collaborative Services (eLearning): Collaboration for learning called T-learning (learning over the TV platform) which is already an own and wide community and will not be explained further in this paper. • TV as Community Platform for Communication and Exchange of Ideas: To enable communication between viewers across the TV environment, by providing typically Web 2.0 functionalities, like messengers, avatars for representing the viewers, grouping of like-minded viewers, and so on are. Example projects to this are collaboraTV [10] and A3 TV [8]. • Entertainment Services: Collaboration for entertainment - by solving common tasks and goals within special TV program formats as it was done in 2005 by the German television sender “Kabel Eins”. In their show “K1 Magazine”, they built a lighthouse in cooperation with their viewers. They could act as co-architects to be involved actively on the design of the building which was finally raffled amongst the participants. In 1

http://www.ist-live.org

2 http://www.wreckamovie.com/

this case, we can define the TV content as collaboration medium although the collaboration was very simple. The participants decided, for example for the 1st floor, out of x example floor plans per telephone call, SMS or e-mail. Finally the simple majority decided. The presented areas and example projects (just a compendium of existing ones) show clearly the multi functional usage of “collaboration” in combination with the television environment and its impact on the whole TV value chain: 1. Pre-production: Collaborative script-writing. 2. Production: Collaborative production as shown with “Wreck A Movie” or E = M C 2 + 1 [11]. 3. Content Offering: Amongst others with social PeerTo-Peer alike systems as e.g. Tribler [12] which connects people with similar taste. 4. Distribution: The distribution can happen via certain technologies like IPTV, Peer-To-Peer systems as done with [12], Internet (Web TV) or conventional broadcasting infrastructures (cable satellite, terrestrial) using an applicable back channel. 5. Consumption: Viewer can act collaborative with the content during the consumption as done with the “Kabel Eins” show. Or with special Web 2.0 services as offered with A3 TV [8] where viewer can e.g. provide real-time feedback for the producers. Note: we deal with TV as described by the TV value chain, not with movie production. If we take the so far presented trends as initial point, it is noticeable that hardly any applications / program formats, using collaboration for entertainment and active participation of the consumer, exist. This gap was only bridged by “collaborative-like” reality TV with voting procedures as BigBrother, Idols, the “K1 Magazine” show and so on are. Another point is the rare usage of the TV content as media for collaboration (instead of content as result of the collaboration). For this purpose, this results in the production of special TV program formats, less in the production of systems and applications around the existing TV content.

3 Towards a Future Digital TV based on a Collaborative Infrastructure In the further part of the paper, we specialize on two points by designing a collaborative TV system for entertainment using the content as media for collaboration. In the following, we created an example scenario to give an idea how collaborative interactive television can be performed: The following is given

1. Teams: 1 to n Teams with at least two participants in a team. A team can be open or closed as described below - this of course depends on the collaborative activity. The team players are separated at their TV stations or computers and may or may not know each other but work together as a team, sharing the payoffs and outcomes. If the team wins or loses, everyone win or lose [13]. The team players have the same interests and beliefs and they need each other (their knowledge, special skills, brave, power, etc) for a positive outcome. 2. TV program format: A special collaborative TV program format. In the simplest case, this is an interactive game where team members have to collaborate to win (against another group). A more complex case would be the production of a special TV program format like a collaborative game show, quiz show, etc. In this context we limit to the simpler case for now. The linchpin is that the TV show controls collaboration in the way of offering / controlling possibilities for cooperation with its contents content as medium for collaboration. 3. Interface: An interface, where the navigation through the game and participants should happen with interactive elements. This interactive elements can be added or removed during the show to control the activity. For the hardware interface we take the conventional remote control for now, since interface design is an own research area. 4. IT architecture: Underlying IT architecture for handling the IPTV, collaboration and data management (see next section). The scenario could be 1 (or more) Team(s) participate to this collaborative show (the exact contents of the show is not important for the moment). The team players do not know each other, the teams are arranged by the operators of the show where people can make an application by e-mail. We suggest, the show is broadcasted via IPTV with the Internet as back channel. The participants use a (specific) interface which is designed for navigation with the remote control. The operators of the show activate interactive elements for the participants (only the participants can interact with these elements, they are inactive for others) and can control the activities amongst the participants within a team and amongst competing teams (if existing). With that, the teams have to solve certain tasks or games collaborative. Of course this is still an abstract example but it should give an idea how collaborative TV could work in the entertainment sector. A collaborative group can be (1) closed with selected participants or (2) open with an (in theory) unbounded number of and consistent changing users. Taking this into account, the following applications for TV content as medium for collaboration can be found:

• Representation of performance and outcome (for a closed and open group). • Representation of content (for a closed and open group). • Interaction (for a closed group). • Coordination of the participants (for a closed and open group). • Collecting statistics to a special topic (content) (for an open group). Fact is, watching television is a typical lean-backward activity (a passive and sometimes automatic “activity”), which is done for getting informed, for relaxing or secondary for background noise. Hardly for active and joint participation, except voting shows as BigBrother, Idols and so on. We think it is an interesting approach turning TV into a lean-forward activity for certain shows. The upcoming popularity of these reality shows, indicates the attendance of the viewer firstly to participate in another life, career or fate. Secondly, the high participation on the voting procedure (voting per phone call, SMS or e-mail) also indicates the willingness of the viewers to play a part in this show and bring in their opinion. Why not creating a show, where the viewers participate actively and cooperative from their home TV station? The first question that rises with this idea is if people accept the turn from viewing to participating a TV show at home. Since this is a multi disciplinary question, it can be answered from a sociological, economic or computer science perspective. We can examine only the computer science perspective where the acceptance surely depends on the technical feasibility, usability, performance, possibilities of interaction and the variety of applications - just to name some. Based on this, a reference architecture will be presented in the next section.

4 A Collaborative TV Reference Architecture for IPTV Distribution This section describes a possible, but still abstract reference architecture for a collaborative TV system and its requirements with its focus as described above. In addition, its aims and objectives as well as open research questions are addressed. But before going into detail, the following meta-structure can be identified: 1. Programme Format: designed with respect to the collaborative application. The programme format should provide the content and goal for the collaboration in that way, that the content is used as the medium for collaboration and the goal is public in each steps. The open question here is to check if existing TV programme formats and contents are capable for this as

Figure 2. Meta architecture of a collaborative TV system

well as possible interfaces between the collaborative system and the programme format and its contents. 2. Collaborative Technology: is one of the open research questions. We have to check whether existing collaborative technologies are applicable for this context or not. In addition the demands of a collaborative architecture for this context must be collected as well distinguish this collaborative system from existing ones if necessary. 3. Digital TV Environment: including only the consumption with the receiver, display unit and IPTV for the broadcast. With the following subsections, we provide a first insight into the requirements as well as to the hardware and software architecture with respect to this meta-architecture. 4.1 Requirements for the Technical Architecture We define some requirements for a collaborative TV system architecture, unfortunately the requirements for the collaborative part itself is still in progress and will be published separately later on. However, in this subsection we describe WHAT is required of the architecture, not HOW it is realized.

4.1.1 System Point of View The system shall support central control of collaborative elements / contents but separated from the collaborative layer which manages the access control to files, display, etc. This central control can be done by a special TV program format with the contents as already mentioned above. The real time client / server system should be usable for different TV contents and adaptable for several collaborative TV program formats. Another very important point is the scalability which affects different areas of the system like number of participants and shows, contents, run-time (one episode, several episodes) and so on. The standard requirements for a software architecture like

performance, security, portability, etc are mentioned for the sake of completeness.

4.1.2 User Point of View Very important is the usability in some areas concerning the interface (hardware and software), collaborative content (it must be clear what participants have to do) and the application for participation. In addition we have to differ between interaction within a group and between competing groups with maybe a view separation / merging for the group/s. As this is a system for the layperson (in technical stuff and social interaction amongst a group) in front of the TV it is necessary to provide support for interaction amongst strangers and interaction amongst separated people via the interface. Again here we mention the standard requirements like compatibility (no extra software), security, etc for the sake of completeness. 4.1.3 Broadcaster Point of View 4.2 Hardware Architecture

Figure 3. Possible hardware architecture for a collaborative television system (IPTV part is taken from [14])

2. Replicated, which uses no central server but decentralized server processes on each client which shares the connection, user and session management. 3. Hybrid which uses on the one hand a central server taking all heavy processes, but also decentralized server processes on each client for disburden the server. Figure 3 shows a gray hexagon for the collaborative architecture which should suggest that the decision was not made until now. As mentioned above, we still have to check whether existing collaborative architectures are applicable for this, especially concerning to the types of users (in extreme case, unknown and continually changing amount of users to collaborate) and the usage of the TV content for the collaboration. By using the Internet as basic media for both, the broadcast and the collaboration architecture, the design depends although on the number of users and their intention. For in 2 listed examples, a centralized or hybrid architecture is necessary including one central server (like an Apache web server running a MySQL database and a Subversion version control system) at the TV studio where (amongst others) the outcome is collected and the session may be controlled. Concerning the amount of and maybe constantly changing users, the clients may be overwhelmed with a distributed collaboration architecture. In addition, the hardware connection between the collaboration architecture to the TV program format and its channel back to the IPTV architecture has to be designed. For this, it is necessary to check existing interactive TV environments based on IPTV for their capability. The design of the connection (hardware and software) between the collaboration system and the TV program format is (besides the collaboration architecture) the second research challenge of this project. Figure 3 implements the points (2) and (3) of the meta-structure described in 4 respectively on its right and left side. 4.3 Software Architecture

In general, one has to differ between the broadcast architecture, which is an IPTV [14] environment, and the collaboration and communication architecture, as described above with the meta-structure (Figure 2) and shown in Figure 3. For the latter one, the Internet seems to be the optimal media. It is technically matured and concerning to the IPTV architecture, the logical choice for the moment. Since interactive TV and back channels in television are two growing research areas, the independence of the underlying broadcast and communication architecture is important. Concerning the collaboration architecture, let’s have a look at the three different types of architectures for collaborative systems as described by Borghoff and Gerlicher in [15, 2]: 1. Centralized, where a central server carries about the connection, user and session management.

Figure 4. Software layer architecture

Because of the big amount of possible application scenarios, the software architecture will be a layered framework, with bottom-up working layers, which offers

the opportunity to include several cooperative applications, independently of the remains.

“Allowed” content and collaboration services are activated within the TV program and can be obtained by the user with interaction.

Starting from the bottom, it is built up as follows: 1. Connection Layer The connection layer uses standard protocols to interact with the IPTV devices, as well as for establishing all necessary connections via the Internet. 2. Communication Layer Uses existing (web)services to offer verbal or textual communication between the participants and runs locally. 3. Content Management Layer Which cares about the storage and recovery of the content as well as its versioning by using any Subversion control system. Additionally the management and check of digital rights comes within this layer whereas at the beginning only a manual control of the contents (at least before it is broadcasted) by a human being will happen. 4. Collaboration Middleware This layer cares about all collaboration responsibilities like for example the collaboration environments developed by the W3C collaboration group3 or the ones listed at suns homepage 4 . The synchronization of the content, management of the concurrency control, voting and coordination of the users (just to name some) belongs to its task area. The middleware runs primarily centralized and / or locally. To decide this, a distinction between critical and non-critical as well as heavily and non-heavily operations will happen. The big difference to the above listed projects is the TV content which influences and restricts the collaboration. For example interactive input from the TV program controls if collaboration is allowed, between which users it is allowed, which content may be taken and so on. In case of designing and “building” a house collaborative, the viewers vote about the collaborative partners for this episode which are then activated by interactive elements. In addition, the content for the collaboration (for example the floor plan to design) is provided by this way. 5. Application This part considers to the main collaborative application which offers the main, application based operations for doing the job. For example in case of the scenario of designing and “building” a house together within a show, the application would offer the main operations for managing and editing the floor plan, some graphical tools for creating new ideas and so on.

6. Program Format This will be the linchpin of the whole collaboration and application process. The darker arrow in Figure 4, represents the interface to the collaboration middleware and to the application which is essential for the coordination of the participants as regards content as well as the representation of the solution and in general to get people interested. The part “TV Program Format” is positioned outside of the architecture because it is not fully part of it - only the interfaces. Possible interfaces are for instance interactive elements built in the program which are used by the participants for the collaboration. 7. User Interaction This deals with the interface to the users and includes the graphical user interface as well as the interface to the hardware input device. One can recognize that the software architecture implements fully point (2) Collaborative Technology as well as connections to (1) Program Format and (3) Digital TV Environment of the meta-structure. For the software architecture, we concentrate especially on the collaboration middleware, on the application and as already mentioned on the programme format as well as on the interfaces to it. 4.4 Aims and objectives of the architecture In general, the architecture should enable taking an active part in TV shows by using the home TV stations - away from the passive viewer role if desired. Taking an active part on such a TV show means the collaboration of volunteers within this special TV show by using the aired television content of this show. Therefore the primary goals / advantages of this architecture are the following 1. Flexibility in case of the underlying technologies. Since interactive television is a growing sector, it is impossible to keep the underlying technology open. 2. Flexibility in case of its application area. As shown in 2, collaborative TV has in general a big potential in its fields of applications, especially in the entertainment sector. 3. It should offer interfaces to the TV program to enable the usage of its content for the collaboration.

3

http://www.w3.org/Collaboration/ http://research.sun.com/projects/dashboard. php?id=85do 4

4. Simplicity of usage for both sides, the viewer and producers.

5 Conclusion and Future Work With this paper, we extended the term “television” to an environment for human collaboration including the whole TV value chain. Within this context we have shown several collaborative TV projects as well as the multiplicity of application areas on the basis of this. Noticeable is the gap in the area of entertainment for such systems and content as collaboration media to which this paper has its focus on. By determining television itself as the core for the collaboration (in contrary to “Wreck a Movie” where it is a social networking website), the main challenges results in identifying and connecting the actors of the system under this point of view: ( ... connect, ... by) (1) User User Content (2) User Content Program Format (3) User Viewer Program Format (4) User Program Format Interactive Elements which leads to interactivity as well as TV content as main connectors. In addition it changes the TV contents “task” from an exclusively informative factor to TV content as media for collaboration. Depending on the amount of participating users as well as the definition of the group as open (constant change and variable number of collaboration partners) or closed (predefined and constant collaboration partners) the new tasks (a) representation of performance and outcome, (b) representation of content, (c) interaction and (d) coordination of the participants (content) for content can be defined. The research challenges in this field can be seen clearly in designing a system which can on the one hand implement the above described concept and on the other hand handle an unknown and continuous changing amount of users. Future work focuses on the evaluation of existing case studies as well as technological platforms to figure out the needs and interests of the people. In addition we will check how content and special TV program formats are suitable as medium for collaboration. This is followed by the design of an application scenario just as a feasibility study concerning to the amount of users and digital rights management. The scenario should show the interaction between hardware, software and the TV program format. Designing and implementing a prototype system result from that. Summarized, with the growing field of interactive TV and the willingness of the viewers getting a more active role in the television environment (as defined in the Introduction), we see the next step of collaborative television absolutely as a seminal research area with suitability for daily use in several years. It is already fact in parts of the TV value chain like the production, but still fiction in the consumer area as the entertainment sector is to which our project leads to.

Nevertheless we think it is an interesting research topic not simply using existing social and collaborative web platforms on top of the TV architecture. But researching in the field how special collaborative TV program formats and their content can change the attention of the viewer away from the television as we practice now.

References [1] Oxford dictionary. [2] A. Gerlicher. Computer-Supported Cooperative Work (CSCW) - kollaborative Systeme und Anwendungen, chapter 3, pages 143–195. X.media.press. Springer Berlin Heidelberg, July 2007. [3] Chris Huxham, editor. Creating Collaborative Advantage. SAGE Publications Ltd., 1996. [4] A. Engstroem, M. Esbjoernsson, and O. Juhlin. Mobile collaborative live video mixing. In MobileHCI ’08: Proc. of the 10th international Conf. on Human Computer Interaction with Mobile Devices and Services, pages 157–166, New York, NY, USA, 2008. ACM. [5] Europaeische zeitbudgeterhebung (european time budget investigation), 2004. [6] P. Herrero, J.L. Bosque, M. Salvadores, and M.S. Perez. On board: Sharing resources in a collaborative grid-tv environment. In WI ’06: Proceedings of the 2006 IEEE/WIC/ACM International Conference on Web Intelligence, pages 601–607, Washington, DC, USA, 2006. IEEE Computer Society. [7] S. Gruenvogel, R. Wages, T. Buerger, and Janez Zaletelj. A novel system for interactive live tv. In ICEC ’07: Entertainment Computing, pages 193– 204. Springer Berlin / Heidelberg, 2007. [8] F.S. de Oliveira, C. Eduardo, C. F. Batista, and G.L. de Souza Filho. A3tv: anytime, anywhere and by anyone tv. In MindTrek ’08: Proc. of the 12th international Conf. on Entertainment and Media in the Ubiquitous Era, pages 109–113, New York, NY, USA, 2008. ACM. [9] A. Joutsen, V. Nieminen, T. Vuorensola, and L. Lekman. Wreck a movie: empowering the masses for film productions. In MindTrek ’08: Proceedings of the 12th international conference on Entertainment and media in the ubiquitous era, pages 141–144, New York, NY, USA, 2008. ACM. [10] N. Mukesh, C. Harrison, S. Yarosh, L. Terveen, L. Stead, and B. Amento. Collaboratv: making television viewing social again. In UXTV ’08: Proceeding of the 1st international conference on Designing

interactive user experiences for TV and video, pages 85–94, New York, NY, USA, 2008. ACM. [11] A. Lugmayr, A. Hornsby, P. Golebiowski, S. JumiskoPyykko, F. Ubis, S. Reymann, V. Bruns, A. Kybartaite, J. Kauranen, and D. Matthes. E = mc2 + 1: a fully digital, collaborative, high-definition (hd) production from scene to screen. Computers in Entertainment, 6(2):1–33, 2008. [12] J.A. Pouwelse, P. Garbacki, J. Wang, A. Bakker, J. Yang, A. Iosup, D.H.J. Epema, M. Reinders, M.R. van Steen, and H.J. Sips. Tribler: a social-based peerto-peer system: Research articles. Concurrency and Computation: Practice & Experience, 20(2):127– 138, 2008. [13] J.P. Zagal and J. Rickand I. Hsi. Collaborative games: lessons learned from board games. Simulation and Gaming, 37(1):24–40, 2006. [14] A. Lugmayr. Television in the age of iptv, high definition and fully digital asset management. Tampere University of Technology, 2007. [15] U.M. Borghoff and J.H. Schlichter. ComputerSupported Cooperative Work - Introduction to Distributed Applications. Springer, Berlin, 2000. All links were checked in January 2009.