ADAPTIVE QUALITY OF SERVICE FOR STREAMED MPEG-4 OVER THE INTERNET Nicola Cranley1 and Liam Murphy2
Abstract: In streaming technologies, multimedia data is continuously fed to the user while they are processing (viewing, listening to, reading) the data. However, streaming with a specified Quality of Service (QoS) is not yet a solved problem. We are building a prototype system to adapt multimedia streaming to a fluctuating network load and/or client requests, thereby providing adaptive QoS. We have shown the feasibility of adjusting QoS levels during transmission by dynamically modifying which transmission layers are actually sent, as well as the transmission profiles and packet priority ratios. I. INTRODUCTION MPEG-4 is a new standard designed for video conferencing and other video applications such as interactive multimedia on mobile, digital multimedia broadcasting, and interactive local multimedia. MPEG-4 decomposes the scene into objects, each with its own audio and video track that will vary over time. MPEG-4 is very flexible. Its content can be tailored (e.g. by layered encoding) so that if the network is congested or overloaded, not all objects need to be sent to the client, only those that make the video sequence coherent [1] and which match the client’s needs. In computer and telecommunications networks, QoS refers to the network operators’ commitment to providing and maintaining acceptable values of parameters or characteristics of user applications in order to satisfy the users’ application requirements and expectations [2]. Providing QoS guarantees is difficult in networks that offer "best effort" service, such as the Internet. The IP protocol makes no guarantees about when data will arrive or the order of packet arrivals or how much data it can deliver [3]. RTP [4] and its control protocol, RTCP, are designed to support real-time applications, independent of the underlying transport and network. By itself, RTP does not ensure QoS guarantees are met, but simply facilitates QoS support by adding extra functionality to UDP/IP such as payload identification, sequence numbers, timestamps, and delivery monitoring. RTCP is used to periodically convey statistics about the transmission to the client and server. RTCP is particularly useful as it enables the integration of some QoS mechanisms by means of client feedback, allowing the server to modify its transmission in response to the clients’ requirements and network conditions. II. SYSTEM DESIGN & IMPLEMENTATION FOR QoS Our system decomposes an MPEG-4 transmission into a hierarchy of layers. Within each layer there are a number of profiles and transmission schemes which can provide an adaptive QoS during the session. A typical user has little interest in the number of dropped or lost packets or the round trip delay once the overall perceived quality of the transmission is acceptable [5]. One possibility for adaptive QoS employs the use of layered encoded streams, whereby the video stream is composed of a hierarchy of sub-streams: each sub-stream enhancing the quality of the lower layers’ sub-stream. The sub-streams are classified using the specifications of the encoding layer. For example, the base file would have an average acceptable value for the transmission parameters, and the enhancement layers would augment these values, thus increasing the overall application quality [6]. At the time of connection establishment there is a deterministic guarantee at a default or user-specified level. Once the session has begun it is possible to adapt the QoS levels based on the clients’ feedback, thereby enhancing/degrading the transmission. In the system developed the MPEG-4 codec/player is based on the open source code from CSELT. Layered encoding is pseudo-emulated by decomposing the MPEG-4 1
School of Electronic Engineering, Dublin City University, Glasnevin, Dublin 9, IRELAND
[email protected]
2
Department of Computer Science, University College Dublin, Belfield, Dublin 4, IRELAND
[email protected]
stream into 3 sub-flows or layers: the base layer (BL), enhancement layer1 (EL1), and enhancement layer2 (EL2). It is assumed that the base layer contains essential information pertaining to the MPEG-4 file, such as Initial Object Descriptors, important Audio Visual Objects and scene descriptors, etc. The enhancement layers contain information which is not essential to the MPEG-4 file reconstruction on the client side, but which enhances the overall perceived quality of the stream. These layers can be assigned differing QoS levels with a different QoS guarantee class and adaptation policy (Fig. 1.A.). Within each layer (BL, EL1, EL2) it is possible to have a number of transmission profiles which determine the packets actually transmitted by the server. Three different transmission profiles have been defined: • LOW: this starts with normal transmission but scales the servers’ transmission rate to match the reception rate of packets on the client side by discarding a percentage of the low priority packets. Thus, only high priority packets and a percentage of the lower priority packets are sent. The worstcase scenario for this profile is that only high priority packets are sent; • MEDIUM: high & low priority packets are sent (normal transmission); • HIGH: normal transmission, plus a copy of each highest priority packet is also sent. Within each transmission profile, the system tries to adapt its transmission by fine-tuning the packet prioritisation ratios within a stream, relying on a number of fields within both the marker bit in the RTP header and the Type of Service (TOS) field in the IP header [7]. The TOS field indicates the type of treatment an IP packet requires during its transmission through the Internet. Within a stream it is desirable to have a number of different packet priority types so that, should packet dropping be necessary, packets of a lower priority will be dropped first. The application’s measured QoS is discovered through analysis of the RTCP packets quasiperiodically received from the client. The Sender Report (SR) and Receiver Report (RR) are used for the transmission and reception of statistics from each active sender and receiver. The session statistics conveyed via RTCP reports are the fraction of packets lost, the cumulative number of packets lost, the highest sequence number received, the inter-arrival jitter the time since the last SR, and the delay since the last SR. An application's QoS can be re-negotiated under the following circumstances: • To cope with a change in an application's end-to-end performance as observed by the client. In this case, the client directly requests an upgrade/downgrade in quality. • To cope with change in a connection's performance (delay, loss, jitter) in the network. Transmission adaptations should be seamless to the user [8]. For control information (such as user QoS requests) it is important that the message arrives at the server as soon as possible. It was decided that there should be some the Out Of Band (OOB) messaging system to send an urgent message (Fig.1.B). During transmission the client may request via the OOB channel to quit, rewind or upgrade/downgrade the transmission of the various sub-stream flows. It is possible for the user to have total control over the transmission, but then the server and user could be inundated with control messages. So, in a practical system, the user should only be able to indicate fundamental changes (such as upgrade or downgrade quality) and not low-level changes (such as setting RTP marker bits or IP TOS values).
Fig.1.A. System Design
Fig.1.B. System Architecture
III. RESULTS AND ANALYSIS All packet header information sent and received on both the client and server side is monitored to assess the QoS. The system has been tested on a local loop with a network simulation implemented within the server program. Using a network simulation which can specify the mean one-way network delay and loss rate is advantageous to observe and control the network behaviour enabling differing test cases for varying network conditions. While the server is generating packets, it randomly drops a percentage of the packets according to the network loss rate. If a packet is to be dropped, the server program will update sequence numbers and timestamps, then delete the packet without actually transmitting it. If a packet is not dropped, it enters a network delay component. To simplify the system, only 3 packet priority types are possible within a stream: the highest HP=2, HP=1, and the lowest HP=0. HP=2 is defined as having a TOS value of 5 and the RTP marker bit being set to 1; HP=1 is defined as having a TOS value of 5 but the marker bit is not set; HP=0 is defined as having a TOS value of 4 and the marker bit is not set. The results are based on the RTCP sender reports from the server and receiver reports from the client. The transmission rate is calculated by dividing the number of packets transmitted during the RTCP interval by the duration of that interval. In the results presented below, the transmission rate will appear to have a high standard deviation, this is due to the variance in the RTCP intervals and the network delay. The weighted throughput efficiency is evaluated by the client who keeps a count of the number of packets of each type received by reading the IP TOS value and RTP marker bit. Corrupted packets, redundant packets and packets delayed past their play-out time are subtracted from the received packet count. The count of these is weighted in the ratio of 3:2:1, where HP=2 packets have a weighting of 3, HP=1 packets have a weighting of 2, and HP=0 packets have a weighting of 1. III.A. Feasibility of fine-tuning by packet ratio adaptation The network simulation conditions chosen for demonstrating how adapting the packet ratios within a stream can affect the weighted throughput efficiency values on the client side are a mean one-way network delay of 150ms and a network loss of 5%. This network delay means that the number of packets received past their play-out time is negligible and has little effect on the clients’ assessment of the QoS by the weighted throughput efficiency. The graph below shows the advantages of using preferential packet dropping within a network. Without implementing a complex packet dropping policy, these results show how a simple scheme of dropping packets affects the weighted throughput efficiency on the client side. Should dropping the ratio of HP=0 packets be insufficient to contain the 5% network loss, then the remainder of required packet losses are taken from the HP=1 packets. These results are the average from a number of test cases where the packet ratios are modified (Fig.2.). Ratios are assigned at the server side in the format, for example, 40:20:40 where 40% of the transmitted stream is HP=2 packets; 20% of the transmitted stream is HP=1 packets; and 40% of the transmitted stream are HP=0 packets. The bold line at 95% represents the expected unweighted throughput efficiency i.e. where packets are randomly dropped and each packet has equal weighting.
Fig.2. Average Throughput Efficiency by Packet Prioritisation
If HP=2 represents a large proportion of the stream (50%-90%), then packet losses have a greater impact and thus reduce the weighted throughput efficiency values and report that the QoS is severely degraded. The main aim of employing subtle modifications to the packet ratios is to intelligently control the dropped packets to be those of lower priority weighting. The overall effect of this improves the weighted throughput efficiency to be greater than the expected unweighted throughput efficiency, and yields a more accurate measurement of the QoS of the received stream on the client side. III.B.1. Feasibility of profile adaptation The purpose of this test is to show how switching between the various profiles can improve the weighted throughput efficiency significantly more dramatically than by modifying the packet ratios. In the test cases chosen, the network simulation parameters are a mean one-way delay of 200ms with a network loss of 5%. These packet priority types are in the ratio of HP=2 40%, HP=1 30% and HP=0 30%, which remains unchanged for the duration of the simulation. The reason for fixing the packet ratios is that the subtle improvements made by adapting the ratios would be unnoticeable. Switching to the low profile is to the worst case situation where just the high priority packets are sent representing 40% of the stream without gradual scaling of the transmission rate. There is no preferential packet dropping in the network simulation. It is expected that the effective throughput should be less than the expected throughput as duplicated packets and packets received past their play-out time are not counted as playable and usable. These packets, therefore, do not increment the actual number of packets received. This can be seen when switching to the high profile, as redundant packets received do not increment the effective throughput; however they increase weighted throughput efficiency by providing greater reliability for high priority packets (Fig.3.A.).
Fig.3.A. Weighted Throughput Efficiency with Profile Adaptation
Fig.3.B. Packet Transmission and Reception Rates with Profile Adaptation
III.B.2. LOW Profile 2: Transmission rate adaptation The purpose of this test case is to show how the transmission rate can be scaled down to suit the reception rate on the client side. Stepping the transmission rate up and down is in increments of 2% of the maximum transmission rate (Fig.4.A.). The network conditions chosen for this test case are a one-way network delay of 150ms and a network loss of approximately 5%. The packet priority ratios are fixed as HP=2 50%, HP=1 30% and HP=0 20%. Transmission rate adaptation is useful, for example, when the client buffer is insufficient to process all the packets received, or in cases of network congestion, when the server may wish to reduce the load it sends over the network allowing the network to recover gracefully from the bottleneck by scaling up and down the load. The worst case situation of scaling down the transmission rate is to send just the high priority packets, which has been shown with the transmission profile adaptation (Fig.3.B.). The measured effective throughput efficiency is consistent at approximately 96% despite the scaling. The reason for this is that the effective throughput is calculated from the packets transmitted and received, so if the transmission rate is changed then the corresponding reception rate is proportionately reduced, keeping consistent the effective throughput. The graph below shows how preferential dropping of lowest priority packets can ensure that there is no loss of higher priority packets.
Fig. 4.B. Client Jitter as a result of Transmission Rate Adaptation An interesting consequence of transmission rate adaptation is the variance in the inter-arrival jitter measured on the client side. It can be clearly seen how rate adaptation is inversely related to the transmission rate (Fig. 4.B.). This must be taken into consideration for multimedia transmissions that are particularly jitter-sensitive, for example VoIP applications. Fig. 4.A. Server Transmission Rate Adaptation
III.D. Feasibility of Adding and Dropping Layers The purpose of these tests is to show the feasibility of adding and dropping layers during a session. The network conditions chosen are a mean one way network delay of 150ms and a network loss of approximately 5% with no preferential packet dropping or profile adaptation, as only very minor differences were noticed by adapting profiles and adapting packet ratios whilst adding and dropping layers. The results for the profile being fixed at normal profile for the duration of the test are presented. The base layer must never be dropped as it contains essential information required to reconstruct and play out the MPEG-4 stream. The two enhancement layers just add to the overall perceived quality of the stream. From the graph this sequence can be seen more clearly, the base layer (BL) increases linearly for the duration of the test run. Then enhancement layer 1 (EL1) is added, and so its sequence numbers begin to increase, this layer is then dropped again seen as the straight horizontal line, next both enhancement layer one and enhancement layer 2 (EL2) are added so both increase again, so on and so forth (Fig. 5.A.). It can be seen that there is feasibility to add and drop layers seamlessly in response to user requests (Fig.5.B.). The effective throughput is calculated for each layer independently of the others to calculate the quality of each layer and make the appropriate profile and packet ratio adaptations to improve the stream. In addition, these layers are weighted to calculate the overall quality of the session. From this a “recommender” policy could inform the user that a stream must be added or dropped to improve perceived quality.
Fig.5.A. Adding and Dropping Layers Sequence
Fig.5.B. Resulting Transmission Rate with Adding and Dropping Layers
IV. CONCLUSIONS This paper outlines a system which streams MPEG-4 from a server to a client and dynamically adapts the client’s QoS. This adaptation is accomplished by switching between defined QoS levels, which are implemented in our system by different transmission layers (base and enhancement), transmission profiles (LOW, MEDIUM, and HIGH), and packet priorities (using the RTP marker bit and the IP TOS field). The results obtained so far show the feasibility of this scheme for a simple client-server implementation over a simulated network. As well as direct client requests, the server should monitor (via RTCP reports) the client’s perceived performance, and adjust its transmission automatically as needed. This would remove the burden for the end-user of having to explicitly signal QoS changes, while allowing this capability for those who want it. The next step is to formulate and implement a “recommender” feedback policy which controls how the client and server applications communicate in order to adapt the client’s QoS. If the user accepts this recommendation, then the client application would send the appropriate OOB message to the server. This should also address the issue of whose requests take precedence: those from the feedback policy or those coming directly from the end-users. The problem is that sometimes it may be necessary to downgrade a client’s QoS temporarily to improve the quality over the longer term, or to avoid dropping connections or blocking new connections. Users who are experiencing poor quality may not agree to a QoS downgrade despite the fact that it is in their (or the overall system’s) best interests.
References [1] http://woody.imag.fr/MPEG-4/ [2] http://www.qos.net/ [3] http://www.securitydogs.com/qos_library.html [4] H. Schulzrinne, S. Casner, R. Frederick, V. Jacobson, RTP: A Transport Protocol for Real-Time Applications, RFC1889. [5] G. Ghinea, J. P. Thomas, “QoS Impact on User Perception and Understanding of Multimedia Video Clips”, ACM Multimedia ’98, Bristol, UK. [6] Ali Marefat, “Multilevel QoS and Traffic Models for Dynamic Management of Multimedia Connections”, 5th International Conference on Telecommunication Systems, 1997. [7] N. Cranley, L. Fiard, L. Murphy, “Quality of Service for Streamed Multimedia over the Internet”, Proc. of ISSC 2000, Dublin, Ireland, June 29-30, 2000, p. 181-188. [8] N. Cranley, L. Murphy, “Adaptive Quality of Service for Streamed MPEG-4 over the Internet”, ICC 2001, Helsinki, Finland, June 11-14, 2001. [9] H. Schulzrinne, RTP Profile for Audio and Video Conferences with Minimal Control, RFC1890.
