Data-driven QoE optimization techniques for multi ... - Semantic Scholar

The final version of this manuscript appeared in the Proceedings of the International Conference on Computing, Networking and Communications (ICNC 2015) Anaheim, California, USA, February 16-19, 2015 DOI: not yet available

Data-driven QoE optimization techniques for multi-user wireless networks Daniele Munaretto, Daniel Zucchetto, Andrea Zanella, Michele Zorzi † Department of Information Engineering, University of Padova, Italy E-mail: {firstname.lastname}@unipd.it

Abstract—The proliferation of heterogeneous data services and applications, with different communication requirements, has led to the design of Quality of Service (QoS) mechanisms to provide service differentiation and, possibly, performance guarantee to a range of classes of applications. In this paper, we propose a data-driven media delivery framework for the optimization of multi-user wireless networks that differs from the classic approaches in the following aspects. First, it goes beyond the QoS paradigm to embrace the Quality of Experience (QoE) approach, which discriminates data streams based on their actual content rather than just their class. Second, it applies cuttingedge cognitive science techniques to automatically learn data models and discover optimization strategies. To substantiate our argumentation, we discuss a couple of use cases regarding the transmission of multimedia content over a wireless link shared by users belonging to different QoE classes of service.

I. I NTRODUCTION As the consumers’ addiction to smart mobile devices and their need for mobile services accompanying their daily life dramatically grow, network operators face the need for more efficient management strategies of the network resources without overburdening their own network. This turns into planning a radical change of the network architecture that shall acquire the capability of adapting to the service demands in a mostly autonomous manner, in order to optimize the use of the transmission resources while guaranteeing the best possible Quality of Experience (QoE) to the end users. Hence, contextawareness is a key feature to be provided by next generation cellular networks. Using context information, an operator can adapt the behavior of its network according to how content delivery requests are generated. This turns into a better use of the available resources while delivering the service to the user at the expected QoE. The context may include information regarding the type of data (video, email, chat, voice), the geographical distance of the end-points (close proximity, within the same local/metropolitan area, at far distance), their mobility pattern, the service requirements expected by the end users, and so on. This information should be exploited to determine the best network technology to be used at the access side, optimal scheduling strategy, sustainable resource allocation procedures, and so on. While this principle applies to basically all the services offered to the users, it gains interest when dealing with mobile This work was supported by the project A Novel Approach to Wireless Networking based on Cognitive Science and Distributed Intelligence, funded by Fondazione CaRiPaRo under the framework Progetto di Eccellenza 2012.

video traffic that, according to the latest global mobile data traffic forecast in [1], accounts for more than 50% of the total mobile data traffic and is expected to undergo a further 75% increment by 2018, also fueled by the spreading of heterogeneous high-speed wireless technologies by means of LTE, femto-cells and WiFi hot-spots. Hence, supporting the always growing demand for mobile video traffic is a non-trivial challenge for network operators and service providers, which are called to seek effective solutions to increase the network performance and revenues. A possible approach to afford this challenge is to develop mechanisms to recognize the type of video content and determine the encoding format that provides the best QoE to the final user, which depends, for instance, on the current channel conditions [2]. As observed in [3], indeed, reducing the encoding rate of a video is way less critical in terms of QoE degradation than increasing the packet loss probability or the delivery delay. However, the perceived QoE at a certain encoding rate depends on the video content itself, e.g., dynamics of the scene, mobility of the source and frameby-frame motion that cannot be easily predicted nor abstracted. In this paper, we discuss a practical approach to provide QoE guarantees to mobile video clients by employing contentaware video admission control (VAC) and resource management (RM) policies at the bottleneck node of the system, e.g., a WiFi access point or a congested LTE base station, or even an edge router with low-speed Internet connection. We collect a set of reference H.264-AVC [4] video clips, which have been coded into multiple copies by varying the quantization level, thus obtaining multiple versions of each video with different rate-to-distortion gradient. The visual quality of each video is assessed in terms of the average Structural SIMilarity (SSIM) index [5], which is a widely accepted objective measure of the QoE for video. As shown in [2], [6], the quality-rate characteristic of each video can be suitably approximated by means of a polynomial expression, whose coefficients can be associated to each video and communicated to the cognitive VAC and RM algorithms in order to provide QoE-aware video traffic management. Although the computation of the quality-rate characteristic of each video and its polynomial fitting are actually time and resource consuming tasks, we proved in [2], [7] that it is indeed possible to estimate the polynomial coefficients by means of a machine learning approach that only requires the size of the frames in a video Group of Pictures (GoP). Indeed, the sequence of frame sizes in a GoP after the encoding is related to the video compressibility that, in turn, depends on

its content. The neural network is then capable of unraveling this relation and inferring the type of content of the video from its packet size trace. Following the general framework described in [2], in this paper we extend the study presented in [6], [7] by investigating novel QoE-aware VAC and RM algorithms for customers with different QoE guarantees, i.e., belonging to different classes of service, but sharing the same transmission channel. We assume that videos are provided from servers, which act as source nodes in the core network, in the form of short chunks of a few seconds, which are then delivered via HTTP sessions, similarly to what Dynamic Adaptive Streaming over HTTP (DASH) does [8], [9]. Each video streaming session starts with an HTTP request sent by the user to the server for the list of titles and formats of the available videos [10]. DASH enables the use of standard HTTP proxies to intercept and redirect requests for chunks. A DASH video file consists of the Media Presentation Description (MPD) file and the set of segments building the media content. The MPD provides information characterizing the video file and the available locations of the segments, and may contain multiple representations for the same media, that is, multiple versions with different resolutions and bitrates. A DASH client dynamically selects a representation which best fits his network conditions. We assume that all HTTP requests are intercepted by a cognitive HTTP proxy that implements our video admission control and resource management policies in a centralized manner. Furthermore, we assume that the MPD descriptor carries the coefficients of the polynomial that approximates the qualityrate function of the video. This information can then be used by the VAC and RM algorithms in the proxy to provide the negotiated QoE to each user. By means of simulation, we show that our QoE-based VAC and RM algorithms make a better use of the available transmission resources than content-agnostic schemes in the presence of mobile users with different QoE guarantees, and provide a valuable tool for quasi-realtime adaptive video streaming applications. The remainder of the paper is organized as follows. In Section II we review the related work. Our video analysis is presented in Section III. In Section IV we describe the QoEbased and QoE-agnostic resource management algorithms, whose performances are compared by simulation in Section V. Section VI concludes the paper. II. R ELATED W ORK Prior work on video classification over communication networks mainly focuses on extracting objective networking and quality metrics. In [11] the authors classify videos based on selected common spatial-temporal audio and visual features described by the MPEG-7 compliant content descriptors. Due to the complexity of the method, the authors make use of the principal component analysis to reduce the set of features under study. Nevertheless, this work is strictly dependent on the MPEG-7 multimedia format. Scene detection mechanisms were developed in recent years based on predictive analytical models. In [12], the authors propose a scene-change detector for video-conference traces that works based on the average

number of bits generated during the scenes. The proposed low complexity method comes at the cost of requiring full knowledge of video content to properly set the thresholds for the scene recognition. Further related work focuses on quality prediction models to capture the behavior of video scenes. In [13], an objective model to predict the quality of the lost frames for 3D videos is designed based on the header information of the video packets at different ISO/OSI layers. This model is able to roughly capture the SSIM of some video clips based on the size of the lost frames and via deep packet inspection, which is usually avoided in cellular deployments due to the complexity and national privacy rules. Nevertheless, in [14], the authors claim that the frame loss probability provides only limited insight into the video quality perceived by the user. Moreover, the authors state that the rate distortion curves drawn using the Peak Signal-to-Noise Ratio (PSNR) provide a limited representation of the perceived video quality. In our work, we analyze and group video test sequences based on the relation between video compression rate and SSIM. It is widely recognized that the SSIM index improves traditional metrics like PSNR and mean square error (MSE), which have proven to be inconsistent with the human eye perception. Although the SSIM characterization of a video sequence is computationally expensive, in [6] we showed that it can be compactly represented by means of polynomial curves that can be associated to each video. Tagged videos can be handled by simple traffic shaping mechanisms in case of network congestion or under-provisioned network resources. Despite its appeal, a major drawback of this approach is that it requires to tag all the videos with the corresponding polynomial coefficients [6]. Computing the SSIM-rate for each video being transmitted is infeasible even in mediumscale scenarios. An alternative approach is to use automatic methods to support the tagging process [2]. Machine learning algorithms represent the state-of-the-art in many classification tasks, especially when the structure of the domain is difficult to characterize. However, extracting information from visual sequences has proven to be challenging for machine learning algorithms. In the so-called “content-based” video retrieval [15], a range of different techniques can be applied depending on the task of interest, e.g., video indexing, scene recognition and/or classification, object tracking, and motion detection.

III. V IDEO ANALYSIS For the reader’s convenience, we report here the video analysis framework described in [6], [7]. We evaluate the objective QoE of the videos in terms of SSIM, which is a full reference metric that measures the image degradation in terms of perceived structural information change, thus leveraging the tight inter-dependence between spatially close pixels which contain the information about the objects in the visual scene [5]. SSIM is calculated via statistical metrics (mean, variance) computed within a square window of size N × N (typically 8×8), which moves pixel-by-pixel over the entire image. The measure between the corresponding windows X and Y of two

TABLE I M APPING SSIM TO M EAN O PINION S CORE SCALE ( SEE [16]) SSIM ≥ 0.99 [0.95, 0.99) [0.88, 0.95) [0.5, 0.88) < 0.5

MOS 5 4 3 2 1

Quality Excellent Good Fair Poor Bad

between different video clips, it is convenient to normalize the video rates to the full quality rates. Moreover, following the Weber-Fechner’s law that postulates a logarithmic relation between the intensity and the subjective perception of a stimulus, we introduce a logarithmic measure of the normalized rate, here named Rate Scaling Factor (RSF) and defined as

Impairment Imperceptible Perceptible but not annoying Slightly annoying Annoying Very annoying

TABLE II V IDEO TEST SET Name 3inrow 5row1 Akiyo Boblec Bowing Bridge close Bridge far Vtc1nw Bus CaesarsPalace Cheerleaders City Coastguard Container Crew FlamingoHilton Flower Football Football ext Foreman Hall Monitor Harbour Highway Husky Ice Sign Irene Washdc Mobile Mother Daughter News Pamphlet Paris Redflower Silent Soccer Stefan Tempete Waterfall

Full quality rate [kbit/s] 11856 11135 5387 11504 10325 18246 18304 11210 16954 17001 21757 14139 16570 12229 16179 25622 16335 15806 18092 14642 16291 17929 17529 24065 9517 14091 12948 19172 11348 7824 10917 12450 14168 11586 14063 17589 17850 14950

ρ = log(rv (c)/rv (1)) . (2) The dynamics of the video content impact the perceived QoE for a certain RSF value. Note that the SSIM characteristic of a video v can be approximated with an n-degree polynomial expression:

Duration [s] 12 12 10 12 10 66 70 12 5 12 12 10 10 10 10 12 8 3 12 10 10 10 66 8 8 18 12 10 10 10 10 35 12 10 10 3 8 8

Fv(n) (ρ) ' 1 + av,1 ρ + av,2 ρ2 + av,3 ρ3 + . . . + av,n ρn . (3) The vector of coefficients av = {av,i } provides a compact description of the relation between the QoE and the RSF of a video v. It is conceivable to tag each video with such a compact representation of its QoE characteristic that can then be used by RM and VAC algorithms. As shown in [6], in general, a 4-degree polynomial provides an accurate approximation of the SSIM values in the range of ρ of practical (4) interest. Hence, in the following, we consider Fv (ρ) as the reference (exact) QoE characteristic of video v ∈ {1, . . . , V }. IV. SSIM- BASED RM AND VAC A LGORITHMS

images is computed as follows: SSIM (X, Y ) =

(2µX µY + c1 )(2σXY + c2 ) 2 + σ2 + c ) (µ2X + µ2Y + c1 )(σX 2 Y

(1)

with µ and σ 2 denoting the mean and variance of the luminance value in the corresponding window, and c1 and c2 being variables to stabilize the division with weak denominator (we refer the interested reader to [5] for details). The range of the SSIM index goes from 0 to 1, which represent the extreme cases of totally different or perfectly identical frames, respectively. Tab. I shows the mapping between SSIM and Mean Opinion Score (MOS) scale, which assesses the subjective perceived video quality on a scale of 5 values, from 1 (bad) to 5 (excellent), as reported in [16]. We consider a pool of V = 38 CIF video clips, taken from standard reference sets.1 Each video is encoded with the Joint Scalable Video Model (JSVM) reference software [18] into H.264-AVC format at C = 18 increasing compression levels (i.e., quantization levels), which correspond to as many quality levels. The list of video names, full quality rate and duration is provided in Tab. II. Note that there are no scene transitions within each video sequence. The SSIM of a frame encoded at compression level c is obtained by comparing the decoded frame with the full quality version of the same frame. For practical reasons, we evaluate the quality of the whole video for a certain compression level c as the average of the SSIM index of all the frames of that video. We denote by rv (c) the transmit rate of video v ∈ {1, . . . , V } encoded at rate c ∈ {1, . . . , C}, with rv (1) being the maximum (i.e., full quality) rate. To ease the comparison 1 Video

traces can be found in [17].

In this section, we recall the approach presented in [6] and revisit the RM algorithms in order to account for different QoE guarantees of the users. Given a mechanism to infer the QoE characteristic of a video, we develop VAC and RM mechanisms that can make use of such information. We consider a framework where video clips are multiplexed into a shared link of capacity R by a control unit that performs VAC and RM. More specifically, the RM module detects changes of the link capacity (e.g., due to concurrent data flows or fading phenomena) and triggers an optimization procedure that adapts the video rates to maximize a certain utility function. Similarly, the VAC module determines whether or not a new video request can be accepted without decreasing the QoE of any video below the threshold Fq∗ that shall be guaranteed to users of quality class q. To this end, the VAC invokes the RM module to get the best resource allocation for all the videos potentially admitted into the system and, then, computes the expected SSIM of each video by using (3). If the estimated SSIM is below Fq∗ for some active video, the last video admission request is refused, otherwise the video is accepted and the rates of the all active videos are adapted according to the resource allocation provided by the RM module. Formally, let R denote the transmission capacity that needs to be allotted to the videos, and let Γ = {γv } be an allocation vector that assigns to the vth video a fraction γv of R, with γv = 0 indicating that the video is not accepted into the system. Although the H.264 encoding can only offer a discrete set of transmit rates, in the formulation of the optimization problem we assume that video rates can change in a continuous manner. Under this assumption, the RSF of the vth video can be expressed as   γv R . (4) ρ˜v = log rv (1)

The optimization problem addressed by the RM module can then be defined as X Γopt = arg max U (Γ, R, {Fv }) s.t. γv ≤ 1 (5) Γ

v

where {Fv } denotes the set of SSIM functions of the videos, while U (·) denotes the utility function considered by the optimization algorithm. We consider two baseline utility functions that reflect different optimization purposes. Rate Fairness (RF): Resources are distributed to all active videos proportionally to their full quality rate, without considering the impact on the perceived QoE. In this case, the optimal rate allocation for the v-th video is simply given by rv (1) (6) γopt,v = P j rj (1) P so that the RSF of each video equals ρ˜ = log(R/ j rj (1)). This allocation strategy is used as a baseline to assess the effectiveness of the QoE-aware strategy described next. SSIM Fairness (SF): Resources are allocated according to a max-min fairness criterion based on the relative SSIM of the different videos with respect to the SSIM threshold of their respective quality classes. Formally, we define the utility function ∗ Fv (˜ ρv ) − Fq(v) U (Γ, R, {Fv }) = min , (7) v 1 − Fv (˜ ρv ) where q(v) is the quality class of the user that requires video v. The numerator of (7) accounts for the gap between the actual and the minimum acceptable level of SSIM, which shall be as large as possible. The denominator is intended to favor videos with low quality, provided that all QoE guarantees are satisfied. V. V IDEO ADMISSION CONTROL PERFORMANCE We compare the performance of the VAC and RM algorithms by means of simulation. We consider a scenario where a transmission link is shared among N users, e.g., the wireless downlink channel of a cellular system, with an average total capacity of R.2 We assume that users subscribe a contract with the mobile operator that entails three different classes of service, namely bronze, silver and gold, which correspond to minimum SSIM values of F1∗ = 0.9, F2∗ = 0.95 and F3∗ = 0.98, respectively, corresponding to an average MOS of 3, 4 and 5 (Tab. I). Each user generates video requests based on a Poisson process with λ = 0.66 requests/s, where each video request refers to a video randomly picked from the dataset with a specific quality-rate function. The duration of a video sequence follows an exponential distribution with mean T = 5.5 s. The VAC mechanism is invoked at each new video request, which is accepted only if, according to the RM algorithm, the quality levels of all active videos remain above the SSIM threshold of their respective class. Whenever the minimum quality threshold is violated for at least one video in the system, the incoming video request is blocked, otherwise the resources are redistributed among the videos as dictated by the RM. 2 We assume that the time scale of the wireless channel capacity fluctuations is much smaller than that of the video service, hence VAC and RM can work with the time-averaged value of the channel capacity.

Results We compare the RF and SF algorithms in terms of: (a) average number of admitted videos, (b) average SSIM of admitted videos, (c) blocking probability of a video request, and (d) percentage of non-utilized channel capacity. Fig. 1 shows the performance indices when varying the channel rate R with respect to the nominal average rate request for full-quality videos, i.e., G = λE[rv (1)]. In Fig. 1(b) all users achieve a QoE level that is always above the negotiated threshold, as per the VAC objective, but with some differences linked to the algorithm in use. For instance, gold users benefit from a QoE-aware admission mechanism whereas silver and bronze users achieve better performance in terms of quality delivered when the RF scheme is being used. The reason of this apparent anomaly is revealed by the curves shown in Fig. 1(a) and Fig. 1(c), which report the number of active videos and the blocking probability for each quality class, and for the two RM algorithms. We can observe that the number of active videos for each quality class is larger for SF than for RF and, consequently, the blocking probability is smaller for SF than for RF. This is due to the capability of SF to allocate radio resources in a SSIM-aware fashion, thus reducing the quality of videos with larger margin to the minimum SSIM threshold in favor of those which are closer to the lowest quality level for their class. As a consequence, the VAC algorithm will accept more video requests, though the quality of the accepted videos will be, on average, lower than that achieved with RF. In Fig. 1(d) we report the fraction of the channel bandwidth that is left unused by the SF and RF algorithms. Both algorithms show a U-shaped behavior as a function of the ratio R/G. When the channel rate is much lower than the aggregate offered traffic for full quality videos, e.g., R/G < 0.1, we note that the RF uses less resources than SF (i.e., leaves more unused capacity). The reason is that, considering only the nominal video rate for resource allocation, the RF algorithm will not decrease the rate of videos with the least share of resources, even though they are well above the minimum quality threshold. Therefore, it cannot make room for additional videos, which will hence be blocked by the VAC even though some resources remain unused. Note that while the blocking probability steadily decreases with the channel capacity for both RM algorithms, with SF we observe a slight yet noticeable decrease of the average quality of bronze and silver active videos, followed by a pronounced increase when R/G > 0.1. The reason is that, when R/G is small, the algorithm strives to accommodate as many video flows as possible with the available resources, i.e., a new video is admitted as soon as resources are sufficient to guarantee the minimum required quality to all active videos. When R/G is larger than 0.1, however, the channel capacity is sufficient to accept the majority of video requests (with SF), and the larger the capacity the higher the quality that can be provided to active users. Quite interestingly, from Fig. 1(d) note that, for R/G > 0.1, the percentage of unused capacity by SF grows quite rapidly, and from Fig. 1(b) and Fig. 1(c) we see that the average quality of active videos also grows, while the blocking probability drops to zero. In this regime, the channel resources are sufficient to accommodate all video requests,

1

3.5

0.99 3

0.97 Average SSIM

Number of active videos

0.98 2.5 2 1.5 RF, gold RF, silver RF, bronze SF, gold SF, silver SF, bronze

1 0.5 0 −2 10

−1

0.95 0.94 RF, gold RF, silver RF, bronze SF, gold SF, silver SF, bronze

0.93 0.92 0.91 0.9 −2 10

0

10 R/G

0.96

10

−1

(b) Average SSIM of the admitted videos. 0.14 RF, gold RF, silver RF, bronze SF, gold SF, silver SF, bronze

0.8 0.7

RF SF 0.12 Unallocated channel rate

0.9

Blocking probability

10 R/G

(a) Number of admitted videos. 1

0

10

0.6 0.5 0.4 0.3

0.1 0.08 0.06 0.04

0.2 0.02 0.1 0 −2 10

−1

10 R/G

0

10

(c) Probability that a video request is blocked.

0 −2 10

−1

10 R/G

0

10

(d) Unused channel capacity, SF vs. RF algorithms.

Fig. 1. Performance comparison of our proposed algorithms when varying the channel capacity.

though not always at full quality rate. Furthermore, when the offered traffic is lower than the average value because of the fluctuations of the video request process, some resources remain unused. This is also true for the RF algorithm that uses slightly less efficiently the available resources, even when they are abundant, thus leading to a higher blocking probability and a smaller share of unused resources. As a general observation, our QoE-aware video admission control scheme better handles different classes of users in terms of admission probability and channel usage, thus it efficiently scales with the number of media requests and can make room for other concurrent applications that temporarily suffer from a lack of resources. VI. C ONCLUSIONS AND FUTURE DIRECTIONS We designed a framework for video admission control in a multi-user wireless system that exploits QoE-based optimization resources management. By means of simulation, we showed that QoE-aware strategies outperform QoE-agnostic video admission techniques in terms of the trade-off between QoE delivered and computational costs and bandwidth usage. As future work, we will extend the analysis to multi-hop networks, where a reduction of the channel capacity usage for multimedia applications is particularly challenging. R EFERENCES [1] CISCO, Cisco Visual Networking Index: Global Mobile Data Traffic Forecast Update, 2013-2018. White Paper, Feb. 2014. [2] L. Badia, D. Munaretto, A. Testolin, A. Zanella, M. Zorzi, and M. Zorzi, “Cognition-based networks: applying cognitive science to wireless networking,” in Video Everywhere (VidEv) Workshop of IEEE WoWMoM, Sidney, Australia, June 2014.

[3] N. Amram, B. Fu, G. Kunzmann, T. Melia, D. Munaretto, S. Randriamasy, B. Sayadi, J. Widmer, and M. Zorzi, “QoE-based transport optimization for video delivery over next generation cellular networks,” in IEEE ISCC, 2011, pp. 19–24. [4] “Advanced Video Coding for Generic Audiovisual Services,” ITU-T Rec. H.264 & ISO/IEC 14496-10 AVC. [5] Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Process., vol. 13, pp. 600 – 612, Apr. 2004. [6] M. Zanforlin, D. Munaretto, A. Zanella, and M. Zorzi, “SSIM-based video admission control and resource allocation algorithms,” in WiVid Workshop of IEEE WiOpt, Hammamet, Tunisia, May 2014. [7] A. Testolin, M. Zanforlin, M. De Filippo De Grazia, D. Munaretto, A. Zanella, M. Zorzi and M. Zorzi, “A Machine Learning Approach to QoE-based Video Admission Control and Resource Allocation in Wireless Systems,” in IEEE MedHocNet, Piran, Slovenia, June 2014. [8] “DASH.” [Online]. Available: http://www-itec.uni-klu.ac.at/dash/ [9] T. Stockhammer, “Dynamic Adaptive Streaming over HTTP-Design Priciples and Standards,” in ACM MMSys, New York, NY, Feb 2011. [10] D. Munaretto, F. Giust, G. Kunzmann, and M. Zorzi, “Performance analysis of dynamic adaptive video streaming over mobile content delivery networks,” in IEEE ICC 2014 - Communication QoS, Reliability and Modeling Symposium (ICC’14 CQRM), Sydney, Australia. [11] L.-Q. Xu and Y. Li, “Video classification using spatial-temporal features and PCA,” in IEEE ICME, Baltimore, MD, July 2003. [12] I. Spanou, A. Lazaris, and P. Koutsakis, “Scene change detection-based discrete autoregressive modeling for MPEG-4 video traffic,” in IEEE ICC 2013, Budapest, Hungary, June 2013. [13] B. Feitor, P. Assuncao, J. Soares, L. Cruz, and R. Marinheiro, “Objective quality prediction model for lost frames in 3D video over TS,” in IEEE ICC 2013, Budapest, Hungary, June 2013. [14] P. Seeling, M. Reisslein, and B. Kulapala, “Network performance evaluation using frame size and quality traces of single-layer and twolayer video: a tutorial,” IEEE Communications Surveys and Tutorials, vol. 6, pp. 58 – 78, Oct-Dec 2004. [15] S. Antani, R. Kasturi, and R. Jain, “A survey on the use of pattern recognition methods for abstraction, indexing and retrieval of images and video,” Pattern recognition, vol. 35, no. 4, pp. 945–965, 2002. [16] T. Zinner, O. Hohlfeld, O. Abboud, and T. Hossfeld, “Impact of frame

rate and resolution on objective QoE metrics,” in Workshop on Quality of Multimedia Experience (QoMEX), Trondheim, Norway, June 2010. [17] “Test media repository.” [Online]. Available: http://media.xiph.org/video/derf/ [18] “Joint scalable video model - reference software.” [Online]. Available: http://ip.hhi.de/imagecom G1/savce/downloads/SVC-ReferenceSoftware.htm