Sun-Bum Youn, Hogun Park, Geun Young Lee, Dongmahn Seo, Suhyun Kim and Heedong Ko ... cheap and accessible tools to anyone to publish information. It.
T15-S01c / 3
2011 IEEE International Conference on Consumer Electronics (ICCE)
1569350015
Social Media-Based Three-Screen TV Service Sun-Bum Youn, Hogun Park, Geun Young Lee, Dongmahn Seo, Suhyun Kim and Heedong Ko
Abstract — This paper presents a practical application of interactive TV in baseball watching. It has two features. First of all, it employs social media to generate semantic metadata of the media stream that reflect the interest of the public. The metadata are extracted by our proposed bursty feature extraction algorithm, and it provides immediate but rich summaries of live TV contents. Second, utilizing them, threescreen TV service was proposed to provide a new interactive TV watching environment. In the environment, metadata make the system available on interest-based information providing and suitable intelligent interface. We implemented a prototype system in a baseball watching environment at home or public viewing zone.1 Index Terms — Interactive TV, Social Media, Three-screen Service.
I. INTRODUCTION This paper presents a practical application of interactive TV in a baseball watching environment at home or public viewing zone. In the previous research and commercial applications, many services such as enhanced TV [1], Personalized TV [2], and Internet@TV [3] have provided interactive TV contents, and new forms of media consumption are being created accordingly. However, in the previous work, it is limited in extracting metadata to understand contents themselves and providing suitable intelligent applications. Even though they tried to utilize simple metadata by hand or extract low-level visual features from a video, it is still challenging to provide immediate and intelligent supports such as query-free information providing or context-aware interface. In order to overcome this problem, we focus on social media streams in the Web. Social media like micro blogs and chats provide cheap and accessible tools to anyone to publish information. It attracts people to create numerous real-time data streams referring to the same TV program. These real-time data streams are likely to provide good sources for finding the most representative terms at the moment. The characteristics benefit us to get immediate but rich metadata. Our proposed system has 2 novel features. First of all, it employs Twitter and chat data to extract semantic metadata. The collective intelligence of social media already has been adopted by many video retrieval systems [4] and media sharing applications [5]. In this paper, we propose a novel bursty feature algorithm to extract a hot topic. It represents live metadata that reflect the interest of the public and is utilized for later intelligent applications in a three-screen service.
Fig. 1. Procedure of Metadata Extraction.
Second, we designed a three-screen service to provide a new baseball watching system. Previously, there were researches that describe a three-screen concept that use portable devices as a second display [6]. On the other hand, our proposed system focuses on role-based three-screen service. In the system, we employ concepts of a public display like TV, a private display (e.g. mobile phone), and the middle of them (e.g. PC.) Depending on available activities of each display, we provide different services to them. For example, people enjoy personal interest-based services (e.g. highlight, group chat, and information service) using their mobile phones, and TV, a public display, only presents a main video stream and brief summaries of available information. In a PC, they experience both services of the mobile phone and TV, and select their interest for getting details. This paper shows an implemented prototype system in a baseball domain. II. METADATA EXTRACTION AND THEIR UTILIZATION To generate semantic metadata, there are several steps to go through. Firstly, a collection of social media referring the same TV program is segmented into a fixed group of time spans for extracting the most representative keywords. To get a keyword with a parameter-free approach, it employs binomial distribution. It treats keywords with a stream of words and computes probability that each time span contains a particular feature. Only keywords that have significantly higher CDF values are selected as metadata at a time span. Fig. 1 illustrates the procedures.
1 Sun-Bum Youn, Hogun Park, Geun Young Lee, Dongmahn Seo, Suhyun Kim and Heedong Ko are with the Imaging Media Center, Korea Institute of Science and Technology, Seoul 136-791 Korea.(e-mail: {dmonkey, hogun, gylee, sarum, suhyun.kim, ko}@imrc.kist.re.kr).
978-1-4244-8709-7/11/$26.00©2011 IEEE
920
Fig. 2. Highlight Service in Mobile.
Time spans with a set of hot features present temporally topical summaries. Utilizing them, our system provides highlight videos and interest-based contents push services in later three-screen service. To provide a highlight, it measures an impact factor of each time span using a message occurrence. Time spans that have high occurrences describe trending moments, so the system can find out hot periods using the time span and utilize their metadata as a keyword description. Timestamps of each time span direct the beginning/ending moments of the event, and recorded video with the timestamps will provide highlight video. In addition, the metadata are useful for providing an interest-based contents push service. Assume that a user selects some keywords as interest in advance. If the metadata which include the keywords are extracted, the system can alarm to the user. For example, in a baseball game, a user could select “home run” as an interest keyword beforehand. When the metadata that have ‘home run’ are occurred, the system can notice other users who are absent or watching other TV programs.
To guarantee QoS to various devices, the system utilizes SVC (Scalable Video Coding) with P2P multicast overlay network. It efficiently supports different devices with the same video bit stream. Meanwhile, to extract metadata of them, the system collects chat data and Twitter data referring to a TV program. The twitter data are acquired by hash keyword at the same timeline. In addition, the system records TV programs with their timestamp for highlights. Matching a timestamp between metadata and video, it can generate a suitable highlight. Although we have introduced the concept of three-screen service-based interactive TV and shown its advantages, future work remains in enhancing underlying techniques such as extracting semantically more meaningful metadata and implementing a scalable multimedia framework still demand more efforts to support the three-screen TV service. In addition, when more media supporting multi-angle becomes possible near future, our service will provide people better experience. V. ACKNOWLEDGEMENT
III. THREE-SCREEN TV SERVICE The three-screen TV service is based on role-based watching environment. We assign inherent roles for watching TV to public and private displays. In the previous work, F. Kazasis et al. [6] provided a personalized service through mobile device. However, the work is limited in watching in cooperation. In this paper, we introduce a new social interface employing the three-screen concept: a public display, a private display, and an intermediate display. A public display means opened and shared display with other people. Therefore, it should not disturb others and prohibit the visualization of excessive information. On the other hand, a private display is suitable for providing a personal service because it is solely operated and oriented by the user. From the intuition, our watching system provides corresponding services to each device. In our implementation, the TV presents a main video stream and brief summaries of available highlights and chat groups as a public display. A mobile, the most representative private display, presents alternative video streams from different camera angles, highlights and group chats. This interface is more meaningful for media supporting multi-angle videos. It allows people who have other interest to select other camera angles. For instance, if we select some people or teams, they can enjoy totally personalized video with related highlights and issues. This kind of participation can create a new type of media consumption such as interest-based and group-based collaborative watching. In the PC which is the third display, it allows people to have more detail information and experience both services of mobile phone and TV.
This research is supported by Ministry of culture, Sports and Tourism(MCST) and Korea Creative Content Agency(KOCCA) in the Culture Technology(CT) Research & Development Program 2010 and is also supported by Korea Institute of Science and Technology under "Development of Tangible Web Technology" project.
Fig. 3. A Screen Dump of Implementation.
REFERENCES [1]
[2]
[3] [4]
IV. IMPLEMENTATION
[5]
The prototype of proposed system is presented in fig. 2 and fig. 3. In fig. 3, the TV visualizes a main video and available highlight. In a mobile phone, group chat and personal cheering interfaces are presented. Fig. 2 presents highlight service in mobile.
[6]
921
J. F. Jensen, "Interactive television: new genres, new format, new content," In Proceedings of the Second Australasian Conference on interactive Entertainment (Sydney, Australia, November 23 - 25, 2005). ACM International Conference Proceeding Series, vol. 123. Creativity & Cognition Studios Press, Sydney, Australia, 89-96. J. Xu, L. Zhang, H. Lu, Y. Li, "The Development and Prospect of Personalized TV Program Recommendation Systems," mse, pp.82, IEEE Fourth International Symposium on Multimedia Software Engineering (MSE'02), 2002. E. Noam, J. Groebel, D. Gerbarg, Internet Television, LAWRENCE ERLBAUM ASSOCIATES, PUBLISHERS: Mahwah, New Jersey London, 2004, pp. 34-40. D. Yamamoto, T. Masuda, S. Ohira, K. Nagao, "Video Scene Annotation Based on Web Social Activities," IEEE Multimedia, vol. 15, no. 3, pp. 22-32, July-Sept. 2008, doi:10.1109/MMUL.2008.67. J. B. Carlucci, "Social media television in today's cable systems," In Proceedings of the 7th IEEE Conference on Consumer Communications and Networking Conference (Las Vegas, Nevada, USA, January 09 - 12, 2010). IEEE Press, Piscataway, NJ, 963-967. F. Kazasis, N. Moumoutzis, N. Pappas, A. Karanastasi, and S. Christodoulakis, "Designing ubiquitous personalized TV-anytime services," In Proceeding of CAiSE workshops (UMICS), Velden, Austria, 2003, pp. 136–149.
