Adaptive MPEG video coding in the presence of failures as an example for quality of service management Jan-Peter Richter and Hermann de Meer
Abstract
University of Hamburg Department of Computer Science 1
Multimedia systems provide prototypical properties of realtime, performance, and dependability characteristics, that interplay in a complex manner. In this paper we rst state the need for an appropriate quality of service management component. Then, we focus on coding and compression techniques as an integral part of a video conferencing system. This setting allows us to discuss the problem of tradeo analysis between performance, real-time, and reliability issues by way of example.
Keywords: quality of service, multimedia systems and
services, dependability modeling, real-time systems, MPEG coding
1 Introduction Multimedia systems have to be designed to provide realtime behavior in the presence of component failures. Whereas fault events are random in nature one would like to guarantee to meet deadlines deterministically. It is this property which makes the development of these systems a most challenging task. Measures that support fault tolerance might impose negative e ects on the desired real-time behavior and vice versa. The concept of error recovery in distributed realtime systems provides an example of the notion of supportive measures with opposite e ects. Error correcting codes usually work well to compensate for bit transmission failures up to a certain rate. But this is costly with respect to usage of network bandwidth, and as a result a given bandwidth limit might be exceeded, or even a prohibitive increase of end-to-end transmission delay of data units could occur due to additional contention e ects. A more resource economical method would be to trigger retransmission of error prone data units. But this could turn out to be too time consuming for the particular data unit in order to meet its deadline. Real-time properties could be violated and the system could fail to provide its mission. It is the tradeo between eciency and fault tolerance of transmission and coding strategies that is investigated in
this paper. In particular, we use multimedia systems as a widely emerging eld of application that typically exhibit real-time and fault-tolerance properties to demonstrate our approach. Measures have to be expressed in terms of multimedia speci c quality of service (QoS) parameters which are either network or application oriented. Analyses of particular network subsystems have shown that bounds for QoS related performance parameters can be derived analytically [1]. In [2] a translation of QoS parameters between network performance and more application oriented QoS measures is investigated. The possibilities of network congestion control and trac shaping is analyzed in [3] and mechanisms of adaptation { namely rerouting { is discussed and quantitatively analyzed in [4]. In [5] an OS kernel is presented that is able to schedule computational tasks that are needed to transform multimedia data connections into an actual human perceptible presentation. The kernel is guaranteed { by analysis { to meet the speci ed timeliness of the repetitive schedule. In section 2 we introduce the notion of quality of service. In section 3.1 our concept of model based multimedia system management is motivated. By way of example we investigate video coding techniques to show a typical tradeo analysis in section 3.2. Finally, the paper is concluded in section 4.
2 The quality of service paradigm for multimedia applications
Unlike conventional communication systems where throughput, delay and reliability are by far the most important measures of quality, the more sophisticated services in multimedia systems demand a quality of service (QoS) that is de ned by complex and stringent real-time constraints. QoS parameters are negotiated between service user and service provider at connection establishment time. The type of QoS parameters is strongly dependent on the type of communication application [6]. User requirements have to be speci ed in terms of appropriate QoS parameters, i.e., parameters that are typically de ned on a per-connection basis as seen by the end-user (application). It has to be veri ed that the stringent requirements are (will be) met by the communication system 1 University of Hamburg, CS-Department-RO, Vogt-Koelln-Str. o ering the network services. This can be done by quali30, 22527 Hamburg, Tel. +49 40 54715347, Fax. +49 40 54715328, tative reasoning, measurements, or performance prediction email: frichter,
[email protected] by means of models. Since the requirements are - partially 1
quantitative in nature, both for reliability - or dependability - and performance issues, modeling and measurement studies are mandatory. The ever growing need for modeling and management of QoS for multimedia systems arises from two novel facts. Systems that support multimedia applications are usually designed to cover a high diversity of di erent application types. Examples are video conferencing, voice communication, or hi audio and picture transmissions. Also systems have to be designed and implemented for future applications not yet known. Hence systems have to be created that support a highly exible dynamic system management, including network management. The necessary adaptability is a novelty for computer and communication systems and makes the management a challenging task. Furthermore, already current applications imply the need to support guarantees for parameters with opposite e ects. Network transmission delays have to be limited, delay jitters to be bounded, and rather low tolerance levels for corruption and loss of data to be maintained. Clearly timeliness and fault tolerance properties have to be provided on the same time. Hence real-time performance and dependability issues play both a crucial role for the quality of the provided services. For assurance of reliable functional properties sophisticated modeling, management, and veri cation techniques, that interplay dynamically in a complex manner, are necessary. The picture becomes even more complicated due to the limited resources which are often overcommitted for reasons of eciency. Therefore strategies are helpful to optimize switching between di erent modes of operation. Examples are to switch between forward and backward error recovery techniques to tolerate transmission errors, or to choose dynamically among di erent coding techniques as a response to varying timeliness properties.
3 Model based management and decision making
3.1 QoS modeling and management
to a decision whether and when a recon guration should be initiated. In general, multiple options may be available to recon gure a system for recovery in case of a severely degraded QoS performance: QoS parameters may be renegotiated, the policy of resource scheduling and usage can be changed, activities may be aborted, etc. The evaluation of the alternative con gurations nally includes the task of decision making. In order to support an adaptive quality management and veri cation an approach is proposed that is based on the integrated evaluation and optimization of performability measures. The studies are pursued by means of the software tool PENELOPE2 . PENELOPE provides a comfortable experimental environment and is implemented under Motif and X-windows. It supports the model construction and solution process as well as the analysis of the results by graphical representations. PENELOPE can be preferably applied in the context of adaptive, dynamically recon gurable systems [7]. Our approach is to provide a performability management system during run-time. To achieve this goal the following tasks must be completed: To nd a comprehensive set of meaningful measures and QoS parameters to satisfy user requirements that can be mapped accurately on system properties. Due to the layered system structure, this has to be done in a hierarchical way or separately for self-contained subsystems whenever possible. To develop meaningful models which relate system properties appropriately to earlier de ned measures. This includes, of course, to provide numerical, analytical, simulative, or hybrid solution procedures for the chosen models. To unify di erent dimensions and categories of QoS. Note that certain dimensions are only applicable at certain system layers. So research will focus on how they translate and a ect each other across layers and in di erent categories: { dependability measures like MTBF, MTTR, percentage loss of data units in a ow, or bit error rate in ATM cell transmission, { timeliness measures which re ect real-time performance issues like latency, jitter, etc., on different levels, { performance measures like video frames delivered per second, ATM cell peak rate, or throughput , etc.
It is most likely that resources will be overcommitted in multimedia systems, i.e., resources may be statistically multiplexed among services. Even though the system may support each single service, or subset of services, according to the negotiated QoS parameters, there might be a non-zero risk to violate possibly given guarantees. Hence a tradeo analysis is necessary to optimize the statistical gain due to overcommitment versus the statistical loss due to a degraded QoS observed. Other reasons for dynamic violations of QoS are due to possible failures of system components, changing trac patterns, or overload conditions | though "overload" is to some extend related to the scenario of overcommitting of resources. To cope with a changing environment the system 2 dePENdability EvaLuation and the Optimization of has to be self-adapting. Thus in all cases the di erent run- PErformability time conditions of the system have to be evaluated to come 2
3.2 Video coding as an example
compressed data
MPEG encoding
frame i frame i+1 frame i+2 . . .
Parameter
QoS Management
AAL: packetization / traffic shaping / ... Network Management
3.2.1 Preliminaries
One example application scenario we have analyzed is video conferencing. Video conferencing systems impose relatively narrow constraints on real-time behavior of the underlying communication system because of a maximum tolerable delay that must not be exceeded. The mutually agreed-upon communication delay speci es an upper bound. A violation of the contract can be considered either as an annoying disruption or simply as a failure of the service. Therefore, we consider it as appropriate to provide soft real-time guarantees. In order to obey real-time constraints, resources like network bandwidth have to be reserved exclusively for requesting applications. But since a high utilization is also desirable a most ecient usage of the resources is mandatory. Hence multiple compression techniques are incorporated to reduce the redundancy of the voluminous video data. Unfortunately, this is not the whole picture. Due to uncertainties induced by possible component or transmission failures, even reservations of resources do not fully guarantee the timely delivery of a service. But even worse, as we will show, the coding technique itself has an obstructive impact on the mean transmission failure rate. In particular, highly compressed data units are generally more error prone than less compressed ones. It is the tradeo between eciency, as a prerequisite of timeliness, and fault tolerance that we will investigate in what follows. Fig. 1 sketches the investigated setting. Among other trac types, video pictures, i.e., frames, are coded according to negotiated QoS parameters. The management component is responsible for an appropriate choice of the parameters, in order to guarantee real-time, performance, and reliability characteristics from a user point of view. The compressed video frames are further disassembled and processed, before the resulting data units are sent over the network. A network management component which interacts with the QoS manager is responsible for the controlled usage of the network resources.
Performance/ Reliability Management
frame i . . .
Video
Audio
Application other data
The types of values in each dimension might also di er with respect to the following categories: { advisory values, { mandatory values, { upper and lower bounds. To develop an integrated, hybrid modeling approach which includes analytical, numerical, measurement related, and simulative methods.
Network
Figure 1: Integration of QoS management in a communication architecture
The MPEG [9] video compression technique used in our analysis reduces redundancy of the user data (the video stream) according to the user QoS speci cation in two ways: First, homogeneously structured regions of a picture are strongly compressed via quantization and discrete cosine transform [10], resulting in a reduction of intra-frame redundancy. Second, several predictive encoding modes are utilized to homogenize parts of the picture by subtraction of nearly identical parts of preceding and/or succeeding video frames3, which corresponds to a reduction of inter-frame redundancy. Note that because of the rst compression method the pictures are not exactly reproducible, but only up to the speci ed level according to the agreed upon QoS parameters. The second method exploits a property of "real life" video sequences: since in video sequences frame n is almost identical to frame n+1 for nearly all n, encoding of only the (small) di erence information reduces the volume of data massively. But, this elimination of inter-frame redundancy introduces dependencies of decodability within the stream of compressed frames. For transmission of compressed video streams over a lossy channel this imposes the e ect of error propagation. In an MPEG coded video sequence, each frame is coded according to one of three encoding modes: I, P, and B. While I-frames (intra-frame) are coded without references to any other frame, P-frames (predictive) are coded as difference pictures from the last I- or P-frame. The compression potential of P-frames is higher than the compression potential of I-frames. B-frames (bi-directional predictive) are coded as di erences from an interpolation of the pre3.2.2 MPEG data coding and compression tech- ceding and the succeeding I- or P-frame. B-frames compress potentially even higher than P-frames, however, the niques actual ratio of compression factors is highly dependent on Due to limitation of space we will provide a brie ng of the 3 'Video frame' is used in this article as a synonym for 'picture'. results which are originally discussed in full detail in [8]. 3
the characteristics of the coded video stream. Nevertheless, B-frames are never used as a reference point for di erential encoding. Because the succeeding frame has to be known when a B-frame is compressed, a sequence of B-frames introduces an additional encoding delay, proportional to its length. To determine the encoding mode for each frame in the sequence, an IPB pattern is repeatedly applied to the sequence. The IPB pattern can be chosen arbitrarily and is usually de ned by two parameters, N and M, where N is the length of the pattern which also de nes the distance of I-coded frames and M de nes the distance of I- or P-coded frames. M must be a divider of N. As an example, the parameters N=9 and B=3 de ne the pattern I B B P B B P BB. The choice of N and M in uences the overall compression factor as well as the error propagation characteristics of the encoded video stream. In absence of transmission errors, the choice is (nearly) orthogonal to the visual quality of the stream. This property is maintained by a loop-back within the encoder: any encoded I- or P-frame is again decoded in the encoder to serve as the reference picture for subsequently encoded P- or B- frames. This limits the effect of quantization error propagation to a small amount introduced by possible di erences within the arithmetic routines of the encoder and the decoder. Our experiments have shown that encoding with parameters N up to 120 and beyond leads to degradations in picture quality that are only hardly perceptible. The worst Signal-to-Noise ratio for the luminance channel reported by the software encoder did not show any dependencies on the parameter N and the worst SNR for the chrominance channels degraded by only 5% for all pictures regardless of the actual encoding type 4 in the experiment evaluated below. However if a frame is lost in transmission, many more frames, although received correctly, may not be decodable, depending on the encoding mode of the lost frame and the structure of the IPB pattern. For our analysis we use a very generic model of a real time transport system. The only assumptions are: 1. The average delay is known in advance. 2. The maximum delay jitter is known in advance. 3. Intervals between transmission errors can be modeled as a Markov process. However, the term 'transmission failure' has to be clari ed: the term denotes the event, that a subsequence of the bit stream in transmission is not delivered correctly within the time window de ned by the instant of sending time, the average delay and the maximum delay jitter. Typically, the failures are caused by congestion within the communication system and they result in a longer burst of undeliverable bits. This includes e ects of untimely scheduling of
communication software processes anywhere in the system, including the application itself. Because of the burstiness of error conditions we assume in case of an error a frame is lost completely5. The goal of our analysis is to nd an optimal IPB pattern in a given setting. To minimize error propagation the simple IPB pattern I , i.e., N=1 and M=1 resulting in intraframe compression only, would be optimal. On the other hand, the simple I pattern inherently reveals the lowest compression factor and is therefore costly. The highest compression factor could potentially be achieved with an in nite pattern of Type I ((B k )P )1 with k only bound by the demands for a bounded delay. This pattern, though, is very vulnerable to transmission errors: if one of the P-coded frames gets lost, the rest of the sequence is undecodable. The optimal IPB pattern is thus de ned as the IPB pattern that maximizes the overall throughput of decodable frames through the multiplexed channel. This maximizes the overall contentment of all users. On the other hand, if one user is willing to pay for privileged service he or she can require a better QoS (in terms of frame loss probability) and the QoS manager is then able to calculate a parameter set (namely N) and a load characteristic that can be rated and billed by the transport system.
3.2.3 The results
Our analysis starts with the computation of the probability for a failure that corresponds to the obstruction of a single frame. Since the error behavior of the channel is of Markovian nature and a frame cannot be transferred correctly if the fault event occurs before the frame has been transmitted completely, the probability P fF ailureig for a transmissionR failure of frames with index i in the IPB pattern6 1 equals 0 (1 ? e?t=MT BF )fLength (t)R dt, where M T BF is the inverse of the error rate of the channel, fLength is the (empirical) probability density function of the number of bits constituing a frame with index i, and R is the transmission rate of the channel. Assuming the frame lenghts to be independent, exponentially distributed, results in 1=MT BF P fF ailureig = R=E fLength g+1=MT BF , where E fLengthi g is the average number of bits in all frames i. The probability of a frame i to be decodable is Q P fDeci g = j 2!i(1 ? P fF ailureg), where ! i is the set of indices the decodability of frame i depends on, including the frame itself. This set is implicitly induced by the structure of the IPB pattern and can easily be computed following the dependency rules stated above. With these probabilities, the fraction of decodable frames for a given IPB pattern can be computed as = 1 PN P fDeci g. And nally a measure, the e ective i=1 N i
i
i
5 This corresponds to a scenario where packets with deadlines that have expired are discarded by the communication system somewhere between sender and receiver, assuming that all packets disassembled from one video frame have the same deadline 6 In each repetition of the IPB pattern there is one occurence of 4 2 [26 5 ? 28 1] for M=2 and N=2 as well as N=120, 2 [9 34 11 7] for M=2 and N=2 and 2 a frame with index . Therefore, for each index , the is one random variable for the number of bits in a frame with index . [8 9 10 8] for N=120 SN RY
SN RU;V : ;
:
:
:
dB
:
;
:
dB
dB
i
SN RU;V
4
Lengthi
i
i
P frame length, LFeff = 1 N1 Ni=1 E fLengthi g can be de- account. The use of compression techniques could be over-
ned. The optimization mentioned above minimizes LFeff by choosing an optimum IPB pattern. The number of timely delivered data units, which are nevertheless lost due to undecodability because of inter-frame dependencies to lost or corrupted frames, are taken into consideration. In Fig. 2 thee plots are shown that demonstrate the effect of error propagation. While the nominal average frame length LFnom decreases with increasing N , the e ective average frame length LFe , i.e. the number of bits needed for a successful and decodable transmission of one frame, reaches a minimum at about N = 20. The plot tagged LFe emp is the result of the evaluation of the empirical frame length distribution, while LFe synth shows the e ect under the assumption of exponentially distributed frame lengths which is necessary for an analytical solution of the model as presented above. It can be seen that the error introduced by this assumption is negligible. The plots result from experimental series investigating a sequence of 248 frames of 352x288 pixels. Following the MPEG-1 standard, luminance Y is sampled with 8 bit/pixel and the two chrominance channels U and V are subsampled at a ration of 1:2 in both dimensions also with 8 bit. The sequence shows one of the authors during a simulated video conference presenting some objects to the camera7. Parameters for the calculation of the error propagation e ect were: a transmission rate of 10 Mbit/sec and a M T BF of 2 sec. M was held constantly at 2 to allow for a small compression delay. The presented results highlight the need for
valued if error propagation e ects, which were implicitely induced by the compression technique itself, were totally neglected. They have an increasing impact on the overall performance as the length N of the pattern increases.
4 Conclusions In this study we investigated the relation between performance and dependability properties of multimedia systems. There do exist rigid performance constraints on the delivery of services in multimedia systems. At the same time stringent bounds on reliability measures must be obeyed. We have shown that coding and compression techniques can have opposite e ects on timeliness and fault tolerance properties. On the one hand, real-time constraints, together with cost limitations, impose the need for a highly ecient usage of the resources. On the other hand, highly compressed video data, for example, appeared to be more error prone due to error propagation e ects than less compressed data. Therefore, a tradeo analysis is proposed that provides parameter dependent optimum coding strategies, a basis for a recon gurable, adaptive system.
Acknowledgements The authors would like to thank Bernd Wol nger for his knowledgeable review.
References
average frame length [bit]
[1] Wu, G.-L., and J. Mark, "Discrete time analysis of leaky-bucket congestion control", Computer Networks 70000 and ISDN Systems, Vol. 26, No. 1, (1993) pp. 79-94. 69000 [2] Ferrari, D., "Client Requirements for Real-Time Com68000 munication Services", ICSI Technical Report, TR-90-007, (1990). 67000 66000 [3] Gun, L., and R. Guerin, "Bandwidth management and congestion control framework of the broadband network 65000 architecture", Computer Networks and ISDN Systems, 64000 Vol. 26, No. 1, (1993) pp. 61 - 78. 63000 [4] Parris, C., H. Zhang, and D. Ferrari, "Dynamic management of guaranteed-performance multimedia 62000 connections", Multimedia Systems, Vol. 1, No. 6, (1994) N pp. 267-283. 0 50 100 [5] Je ay, K., D. Stone, and D. Smith, "Kernel support for live digital audio and video", Computer Communications, Figure 2: Nominal and e ective average frame length Vol. 15, No. 6, (1992) pp. 388 - 395. an optimization procedure that takes multiple factors into [6] Moran, M., and B. Wol nger, "Design of a Continu7 The sequence as well as the exact setting for the experiments can ous Media Data Transport Service and Protocol", ICSI be requested from the authors. Technical Report, TR-92-019, (1992). 5 71000
LFnom LFeff_emp LFeff_synth
[7] De Meer, H., K.S. Trivedi, and M. Dal Cin, "Guarded repair of dependable systems", Theoretical Computer Science Vol. 128, (1994) pp. 179{210. [8] Richter, J.-P., "Optimale Wahl des IPB Musters bei der Uebertragung von MPEG-kodierten Videostroemen ueber verlustbehaftete Uebertragungsmedien", Technical Report, University of Hamburg, (1995). [9] ISO/IEC 11172-2, "Information technology { coding of movig pictures and associated audio for digital storage media up to about 1,5 Mbit/s. Part2: Video", ISO/IEC International Standard, (1993). [10] Steinmetz, R., "Data compression in multimedia computing { principles and techniques", Multimedia Systems, Vol. 1, No. 4, pp. 166-172 and Vol. 1, No. 5 pp. 187-204, (1994).
6
