AAU Video Browser With Augmented Navigation ... - Semantic Scholar

AAU Video Browser With Augmented Navigation Bars Manfred Del Fabro, Bernd M¨ unzer, Laszlo B¨osz¨ormenyi ITEC – Information Technology, Alpen-Adria-Universit¨ at Klagenfurt, Austria {manfred,bernd,laszlo}@itec.aau.at

Abstract. We present an improved version of last year’s winner of the Video Browser Showdown. In a preprocessing step video segments are detected and clustered in several latent classes of similar content based on color and motion information. The navigation bars of our video browser are then augmented with different colors indicating where elements of the detected clusters are located. As humans are able to classify the content of clusters fast, they can benefit from this information when browsing through a video.

1

Introduction

The Video Browser Showdown (VBS) 2012 showed that certain Known-ItemSearch (KIS) tasks can be performed effectively and efficiently with our AAU Video Browser[1]. It is solely based on intelligent interaction means and refrains from content analysis, but it takes use of the human abilities to recognize and classify items very fast. Therefore, scenes that are significantly different from the other scenes in a video or scenes that are expected at certain locations of a video (e.g. in most cases weather reports are at the end of news videos) can be found fast. On the other hand, specific tasks exist where it is difficult to find the searched scene with the same approach. In particular, in videos that show similar content from the beginning to the end (e.g. TV shows) or videos that consist of repeating similar situations (e.g. videos where anchorpersons or some sports acts are shown again and again) scenes are hard to find only by human observation. This year we present an improved version of our video browsing tool, which combines automatic content analysis with human cognition to overcome the problems mentioned above. We present a video browsing tool that augments the navigation bars with additional information that indicates where certain content classes are located. The users still have to look for the searched items manually, but the search effort can be reduced by taking advantage of the additional information provided by the augmented navigation bars.

2

Augmented Navigation Bars

Augmented navigation bars are annotated with colored blocks, where each color represents a certain class of video content. An example is shown in Figure 1.

The idea is (1) to automatically detect repeating segments within videos, (2) to cluster them in groups of similar content and (3) to annotate the navigation bars with colors indicating the location of the clusters in the video. Each cluster is visualized by a different color. A large number of analysis methods can be used for step 1. For this approach we only rely on the color information and the motion information, which are extracted from the videos in order to cluster video segments into groups of similar color or motion. A latent indexing of the content is performed, thus no predefined classes are used.

Fig. 1. Example of an augmented navigation bar

The classification of the emerging clusters is the part where the user comes into the loop. A preview panel is provided to the users, which shows the cluster centers of each cluster. Each preview is surrounded by a border colored with the same color that is used for marking all segments of the corresponding cluster on the navigation bar. The preview panel is shown next to the video windows in Figure 2. It helps the users to quickly make a basic discrimination of the content of a video. If the representative frames are not discriminative enough, users can load all segments that belong to a cluster in an own playlist by clicking on the cluster center in the preview panel. The elements of a playlist are ordered chronologically, thus scanning the items of a playlist from the beginning to the end is also an option to search for a certain video segment. An example of a playlist can be seen in the right part of the window in Figure 2. The most important point regarding the presented video browsing application is that users are still interacting with videos and not only with static key-frames. Therefore, they experience videos in the same way as usual, but in addition the amount of time needed for certain KIS tasks can be reduced.

3

Conclusion

We improved our AAU Video Browser for the Video Browser Showdown 2013 with augmented navigation bars. The idea is to reduce the search space for users by coloring the navigation bars with different colors that indicate different clusters of video content. At the competition we want to investigate whether the problems that occurred in videos with recurring similar scenes can be overcome with this new approach. The combination of well-known video interaction concepts with the results of content analysis has several advantages. The presented solution preprocesses the content and suggests classes of similar content to the users, thus they get a first overview of the content of a video. Moreover, users can interact with videos

Fig. 2. The four windows at the left side can be used to browse four parts of a video in parallel. Next to them the preview panel is displayed. At the right side the playlist view is showing the segments of one cluster.

as they were always used to. Compared to other key-frame-based tools they can still watch and interact with a video stream. The augmented information is only an additional help for them. Furthermore, the temporal order of the content gets preserved, thus users always have an overview of the temporal correlation of different segments. We are going to investigate the integration of further analysis methods, such as local feature descriptors or face detection methods, and also the combination of different methods in future.

Acknowledgment This work was supported by Lakeside Labs GmbH, Klagenfurt, Austria and funding from the European Regional Development Fund and the Carinthian Economic Promotion Fund (KWF) under grant KWF-20214 22573 33955.

References 1. M. Del Fabro and L. B¨ osz¨ ormenyi. AAU video browser: Non-sequential hierarchical video browsing without content analysis. In K. Schoeffmann, B. Merialdo, A. Hauptmann, C.-W. Ngo, Y. Andreopoulos, and C. Breiteneder, editors, Advances in Multimedia Modeling, volume 7131 of Lecture Notes in Computer Science, pages 639–641. Springer Berlin / Heidelberg, 2012.