A QoE management system to improve the IPTV ... - Semantic Scholar

INTERNATIONAL JOURNAL OF COMMUNICATION SYSTEMS Int. J. Commun. Syst. 2011; 24:118–138 Published online 30 April 2010 in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/dac.1145

A QoE management system to improve the IPTV network Jaime Lloret∗, † , Miguel Garcia, Marcelo Atenas and Alejandro Canovas Polytechnic University of Valencia, Camino Vera s/n, 46022 Valencia, Spain

SUMMARY One of the biggest problems of the IPTV providers is to offer enough quality of experience (QoE) to their customers. The minimum bandwidth in the access network required to provide IPTV services, jointly with the necessity to guarantee the QoE to the customers, creates the need for new type of algorithms to satisfy the network requirements. Apart from the parameters that depend on the application aspect and design, the user’s QoE mainly depends on the video quality (VQ), which can be affected by the network parameters (jitter, delay, lost packets, etc.), and on other parameters, such as zapping time and synchronization time. In this paper, we implement several benches to know how each measurable parameter affects the user’s QoE. Taking into account these parameters, we propose an analytical expression for the QoE calculation. Then, the paper presents a network management algorithm that takes into account the information received by the user to take the appropriate actions and vary some features of the IP network to provide enough IPTV QoE to the customer. Finally, we measure how one of the actions taken by our system affects the network performance and the VQ. Copyright 䉷 2010 John Wiley & Sons, Ltd. Received 29 November 2009; Revised 14 March 2010; Accepted 16 March 2010 KEY WORDS:

QoE; IPTV; network measurement; test-bed study

1. INTRODUCTION QoE can be defined as the overall performance of a system from the point of view of the users [1]. Those users have sensations, perceptions and opinions when they interact with the IPTV service. Moreover, QoE is an indicator of how well the system meets the user needs and is quantified in terms of Mean Opinion Scores (MOS). The MOS scale is defined in International Telecommunication Union (ITU) recommendation BT 500 [2]. Moreover, the ITU presents a QoE measurement framework that can be delineated into different specific measurement points (see ITU Recommendation J.144 [3]). The first point is placed prior to transmission, the second is inside the network, and, finally, the third is at the receiving device. To provide a proper IPTV service to end users, the IPTV providers must have an appropriate IP network to guarantee QoE at the services level. An IPTV network topology can be split into five parts (it can be seen in Figure 1): • The first part is the network header. In this part the servers send the content to the subscribers. It could be devices that receive, transform and distribute the content. • The core network distributes the video flows from the header to the distribution network of the service provider. • The distribution network goes from the end of the core network to the aggregation router, where the access network starts. There could also be video distribution servers. Its main ∗ Correspondence

to: Jaime Lloret, Department of Communications, Polytechnic University of Valencia, Camino Vera s/n, 46022 Valencia, Spain. † E-mail: [email protected] Copyright 䉷 2010 John Wiley & Sons, Ltd.

A QoE MANAGEMENT SYSTEM TO IMPROVE IPTV NETWORK

119

Figure 1. IPTV infrastructure.

function is to multiplex content from different service providers and it adapts the transport system to the specific characteristics of the subscriber loop. • The access network lets the user connect to the service provider and allows access to the multimedia content. The first requirement of an access network is to have enough bandwidth to support multiple IPTV channels for each subscriber, while it allows other services (telephony and data). • Finally, the customer network enables communication and information exchange between the computers and devices connected at home to the services offered through the residential gateway. The service providers must verify that the IPTV services can meet the user expectations. Users will not tolerate service interruptions, image degradations or long zapping times (time between channel changes). The four major responsible systems for distributing IPTV services are the video network header, the IP network, the middleware and the set-top box. Each one of them can affect the quality of experience (QoE) in a different manner. In an IPTV service environment, the monitoring of the quality of the video streams has to be performed in all parts of the network. It is a difficult task because there are rigorous network requirements and it is difficult to isolate IPTV issues from other services. A video degradation could affect just a group of customers in a particular segment of the network. In this paper, we will show the influence of several network parameters on the quality of the video. These measurements will be used to parameterize the QoE analytically. Then, it will be used by an algorithm running in a server that will take the appropriate actions in the network to change some of its characteristics and improve the QoE of the users. The proposed system provides a dynamic network which changes based on the QoE parameter. The paper is structured as follows. Section 2 presents some works related to video assessment methods and QoE enhancement solutions. To know which parameters affect the QoE, Section 3 presents several tests performed in an IPTV network. Section 4 gives the study made to quantify the video quality (VQ) at the user’s side. The IPTV QoE parameterization is explained in Section 5. Section 6 explains a network management algorithm based on the QoE parameter. The algorithm implementation using the transcoding technique is shown in Section 7. Finally, Section 8 shows the conclusion and future work.

2. RELATED WORK There are two ways to measure and verify the VQ: the objective and the subjective quality assessment methods. The aim of both methods is to obtain the Video Quality Metrics (VQM). It Copyright 䉷 2010 John Wiley & Sons, Ltd.

Int. J. Commun. Syst. 2011; 24:118–138 DOI: 10.1002/dac

120

J. LLORET ET AL.

is defined in the ITU Recommendation J.149. This recommendation provides several techniques for specifying the accuracy and cross-calibration of VQM in both methods. 2.1. Subjective quality assessment methods The subjective assessment methods are used to establish the performance of TV systems using measurements that more directly anticipate the user perceptions. To evaluate those perceptions, a group of people watch the video and give it a quality score. The results of the tests are treated statically and the output is often an average of MOS [4]. Video and voice MOS are rated on a scale from 1 to 5, where 5 is the best score. In [5], the authors state that audio and visual media QoE should be evaluated in subjective quality terms. Moreover, objective quality assessment methods are extremely useful for in-service quality monitoring and management, as well as in codec optimization, codec selection and quality design of networks or terminals. However, subjective testing is time consuming, expensive and requires special assessment facilities to produce reliable and reproducible test results. In subjective quality assessment methods, the time selection channel (zapping) is another problem that affects the QoE. In [6], the authors propose a system that uses a list of channels (hotlist), based on the real-time popularity of content and on the user’s past streaming history that reduces this selection time. 2.2. Objective quality assessment methods Reference [3] defines the objective assessment methods as a term that refers to the measurement of the VQ using objective methods. They give an approximation of the rating, by means of performing numerous tests and using the results, which would be obtained from a subjective assessment test. It is possible to create a model of human perception of quality. The objective measurements of quality (VQM) can be performed using one of the following approaches depending on the specific problem and application: • Full reference (FR): The video at the input of the system is compared with the processed signal at the output of the system to determine the quality objectively. But, unfortunately, it cannot be applied to packet networks. This methodology is used by the ITU for objective perceptual VQ measurements in digital cable television (ITU Recommendation J.143 [7]). In [8], the comparison is done on a frame-by-frame basis and, thus requires precise alignment of the two video sequences, which can be an issue if there is a variable delay in the system. • Reduced reference (RR): In this approach, only selected parameters are extracted from the input and the output to be compared. It reduces the transmission bandwidth consumption considerably. E.g., in [9], the authors only use parameters, such as packet loss and related jitter. In [10], the authors propose some quality measurement techniques based on an audiovisual quality assessing method that uses an equivalent signal-to-noise (S/N) ratio conversion method. • No reference (NR): Only the received video signal is used to determine the VQ objectively. It is also known as a ‘single ended’ technique. 2.3. Other proposed systems In [11], the authors focus on the solution of the QoE problem by distributing knowledge, monitor and action plane in all access network components. When the knowledge plane detects that a parameter decreases in the access network, it determines the appropriate actions autonomously to restore it. In [12], the authors improve the QoE using QoS parameters. They focus their work on the interaction of the second parameter with both network and application layer arbitration. The results demonstrate that a better QoE can be obtained using QoS metrics, network feedback and dynamic user requirements. Finally, in [13], Zapater and Bressan propose a multilayer system that uses QoE and QoS metrics. But, a large number of parameters must be taken into account to provide adequate service. Copyright 䉷 2010 John Wiley & Sons, Ltd.



121

3. IPTV QOE PARAMETERS MEASUREMENTS QoE parameters can be measured in the transport layer and the application layer of a TCP/IP network. Although there are many subjective parameters that can be included in the QoE, such as content availability, easiness and available content indexation, user interface, palette colours, ergonomics, navigation design and program guide, there are two main areas where the IPTV quality of experience can be measured objectively: • Zapping measurements: They show how fast the customers change to a channel and verify that they are receiving the correct one. A delay of 1 s is considered as acceptable channel zapping, and between 100 and 200 ms it is considered instantaneous [14]. • Video and audio quality parameters: There are many factors that compromise the audio and VQ. On the one hand, there are the amount of IPTV subscribers, their behaviour and the triple-play convergence, and, on the other hand, there are the network parameters that are affected by the precious ones: bandwidth, packet losses, jitter and latency. All of them should be tested from the network header to all customers of the network independently to guarantee their QoE. Media Delivery Index (MDI) is accepted by the industry to test the quality of video and audio through the elements of the network in a video distribution infrastructure. MDI is defined in the RFC 4445 and is supported by the IP Video Quality Alliance (IPVQA). The main components of the MDI are the Delay Factor (DF) and the Audio and Video Loose Ratio (MLR). They are based on the Jitter and Packet Losses parameters. MDI parameters are even more relevant for knowing the performance of the network equipment than the VQM based on the codification and compression properties of the codec. To isolate the effect of the devices of the network in the QoE, the metric should be based on the measurements in the packet level. Because MDI is not based on the decoding, the measurements can be scaled to thousands of customers. DSL WT-126 forum [15] recommends a maximum loss of five consecutive IP packets for every 30 min of Standard Definition Television (SDTV) or VoD and the same maximum for 4 h of High Definition Television (HDTV). Other studies [16] recommend a maximum DF of 50 ms and a maximum MLR of 0.004 for SDTV and VoD, and 0.0005 for HDTV for all the codecs. 3.1. Test bench We have set up the test bench shown in Figure 2 to take IPTV QoE parameter measurements. There is a video server that simulates the network header, a Cisco Catalyst 3550 switch that simulates the network core, a computer and two Cisco 2600 series routers that simulate the distribution network and allow us to vary the network parameters, and a Cisco Catalyst 2950 switch, that

Figure 2. Test bench layout. Copyright 䉷 2010 John Wiley & Sons, Ltd.


122

J. LLORET ET AL.

Figure 3. Jitter test.

jointly with the other routers allow us to modify several parameters, such as multicast groups, routing, VLANs, management of type of queues. Finally, three final users are connected to the switch. One of them uses a computer with the VLC Media Player [17] and the other two have set-top boxes (one is wired and the other is wireless IEEE 802.11b/g, for the wireless user we used a Linksys WAP54G Access-Point). VLC Media Player is used in the IPTV server as video streaming software to stream an SDTV channel (the SDTV channel is a video with a resolution of 720×576, 25 fps and duration of 396 frames). On the other hand the network is simulated using Netdisturb Software, of ZTI Company, in the PC inside the emulated ISP network. At the end of the network, the video is captured for further study using the VLC Media Player as client. Finally, we used ‘Elecard Video Quality Estimator’ [18] software, of Elecard Company, to get the VQM values. It makes a comparison between the original video and the impairment video sequences for an FR objective quality assessment method. 3.2. Delay test First, we introduced several delay values to know how they affect the VQ. These values were 5, 10, 20, 200 and 400 ms. All these values presented similar behaviour. There was no change of the video observed by the final user. The measurements obtained indicate that the overall quality of the videos is very good. But, delays higher than 400 ms will affect the Zapping time of the customers. 3.3. Jitter test The jitter measured in our tests is represented in Figure 3. When IEEE 802.11 b/g is used as the access network to the IPTV service, there is a greater jitter. The average jitter is 0.58 ms. When we are using a wired technology (FastEthernet, IEEE 802.3u) the jitter has a value of 0.04 ms. The jitter in FastEthernet is stabler and lower. The wired network gives more stability to the measurements. 3.4. Bandwidth test To perform the bandwidth test, we have supposed several environments: A European Wide Band RDSI channel (2 Mbps), a typical MPEG-2 SDTV channel (3 Mbps), the typical bandwidth offered by an IPTV service provider (6 Mbps), a wireless IEEE 802.11b network (11 Mbps) and a wireless IEEE 802.11g network (54 Mbps). We have limited the bandwidth for all these values using a PC with Netdisturb Software, which reject incoming packets when the bandwidth threshold is exceeded, or when the queue is full. As a reference our original video sequence requires 6.3 Mbps of bandwidth to be transmitted without problem across our test bench network. The software for testing the image quality (Elecard Video Quality Estimator), implements the VQM_cur_YUV metric. As VQM_cur_YUV is an instantaneous value of comparison between the original video and the impairment video, hence it is useful to identify an individual problem in Copyright 䉷 2010 John Wiley & Sons, Ltd.



123

Figure 4. VQM_curYUV for different bandwidths.

a sequence video. The VQM_cur_YUV metric gives its value starting from zero, if the value is zero this means that the original video and the impairment video are equal, or that there are no significant differences between them. On the other hand for a higher value of the VQM_cur_YUV metric, there are more differences between the original video and the impairment video. In Figure 4, we can see that for 2 and 3 Mbps there is not enough bandwidth to transmit an SDTV channel, hence many packets are lost. On the other hand in the case of 11 and 54 Mbps the value of the metric (VQM_cur_YUV) is close to zero, which means that the impairment video has no significant degradation when it is transmitted across our test bench network. A special case is the bandwidth of 6 Mbps (alternatively dot-dashed line in Figure 4), where there is a sudden rise at the end of the line. This is produced by a significant difference between the original video and the impairment video, which means that the impairment video has a bad image quality generated by a couple of packets lost. But, why is there packet loss in a link of 6 Mbps? The reason is the average size of the frames (I,B,P frames) in our original video sequence which presents a similar size in the first 200 frames, which is well-managed by the virtual queue at the router (PC using Netdisturb Software). However, when we analyze the video transmitted, we see that the size of the frames are increased at the end of the video, and this exceeds the maximum capacity of the memory of the virtual queue at the router and therefore all new incoming packets are lost, giving a bad image quality to the final user. A solution to this issue is to add more memory to the virtual queue at the router, to be able to handle the increase of size of video frames, or a more appropriate solution is re-encode the original video with a lower bit rate to reduce the size of frames and avoid packet loss in the router, in any case, the problem that occurs in the blue line is for a small portion of time, whereas the rest of the time the video stream has no visual degradation. 3.5. Lost packets test To perform this test, we use the measurements taken from our previous work for the lost packets in wireless and wired networks [19]. There was a loss rate of 0.04% in FastEthernet and 0.19% in IEEE 802.11 b/g. We fixed the packet loss rate to 0.01, 0.5, 2, 8 and 15%. Figure 5 shows the result of the VQM_cur_YUV metric. The higher packet loss percentage is fixed, the higher metric instant value is obtained, hence the quality of the video worsens. The extent of the peaks represents the amount of frames with problems within the video streaming. When there are packets lost, distorted blocks appear in the image of the final user. 3.6. Zapping time test The quick transition from a stable state to another state can stress the residential gateway significantly and cause the delay and loss of the packets having a big impact on the QoE; this delay Copyright 䉷 2010 John Wiley & Sons, Ltd.


124

J. LLORET ET AL.

Figure 5. VQM_curYUV for different lost packets in the network.

Figure 6. Zapping time for different access networks.

can produce a variation of zapping time that will be different from one type of access network to another. In Figure 6, we compare the zapping time test performed in our test bench (IEEE 802.11 b/g and IEEE 802.3u networks) plus a zapping time test for two commercial access networks (xDSL, HFC networks). The highest zapping time has been obtained in IEEE 802.11u with an average value of 3.35 s, a maximum value of 6.34 s and a minimum value of 1.76 s. The lowest zapping time has been obtained in the xDSL network with an average zapping time of 2.77 s, a maximum value of 3.25 s and a minimum value of 2.25 s. We can observe that the zapping time for IEEE 802.11u is 60 ms higher than the IEEE 802.11 b/g zapping time. We think that it is given because the autosense mode configured by default in the Cisco Catalyst 2950 switches (used in our test bench) produce an extra delay on the ports when we change the channels. Other models or other manufactures may provide lower delay. 3.7. VQM vs MOS In Figure 7, two different video images are shown (one in the first row and the other in the second row). They are used to map MOS values to VQM measurements. From the left to the right side (for each one of the rows), it shows: the transmitted image, the image captured at the user’s screen and the comparison image (which is the difference between them). The first row shows an image transmitted during the bandwidth test performed for Section 3.4 (concretely 3 Mbps in Figure 4). It shows a big error in the video sequence due to the large number of packets lost. The frame has different colours and a number of squares, which indicates that the video has a poor quality. The Copyright 䉷 2010 John Wiley & Sons, Ltd.



125

Figure 7. Sequences used to map VQM to MOS.

second row corresponds to the jitter variation performed in Section 3.3 for the IEEE 802.11b/g test (Figure 3). It shows a good video sequence during the jitter variation test. The comparison image does not show too many colours and MPEG-2 macroblocks, hence the video has a good quality. An excellent quality is obtained during the delay test because it showed a black frame image (it does not produce any difference). MOS are rated on a scale from 1 to 5, where 5 is the best possible score, and indicates the degree of the user’s satisfaction [2]. But MOS value is a subjective metric, and our goal is to map this subjective metric to an objective metric, like VQM; to build a reference model to be used in our QoE management system. To create our mapping from MOS to VQM, we took 5 as the maximum MOS value when there was a VQM value lower than 0.5, because from this value the impairment video in the user‘s screen is imperceptible. Conversely, a minimum MOS value of 1 was elected when there was a VQM value higher than 3, because from this value the impairment video in the user‘s screen became very annoying. According to the tests performed and the VQM values obtained in our measurement tests, we built a mapping between VQM values and the MOS values: MOS = 5 when VQM0.5, MOS=4 when 0.5