A general joint source-channel rate allocation scheme for a dependent video ... Chellappa [6] considered the problem of optimal operational rate allocation for ...
Dependent Source and Channel Rate Allocation for Video Transmission Maja Bystrom and Thomas Stockhammer
Abstract A general joint source-channel rate allocation scheme for a dependent video coding environment is presented. It is assumed that the source coders rely on motion-compensated prediction, and thus error propagation will contribute significantly to the degradation of the reconstructed video. The rate allocation methodology is based upon generation of operational distortion-rate characteristics. In order to reduce computational complexity, these surfaces and the channel code performance are modeled. An analytic method for computing optimal rate allocation across frames in a video sequence is then introduced. Comparisons are made between equal error protection and unequal error protection over video frames. Results are shown for H.263 compressed video used in conjunction with rate-compatible punctured convolutional codes.
Index terms: Combined source/channel coding, resource allocation, video compression
I. I NTRODUCTION Although Shannon’s separation theorem states that it is possible to design an optimal transmission system by separately optimizing each component [1], within the past few decades it has been shown that for practical systems a joint source-channel coding approach may be preferred. Typically, combined source-channel coding has been focused on either optimizing channel coding with respect to the source, source-optimized channel coding, or on optimizing source coding with respect to the channel, channel-optimized source coding. A brief overview of combined source-channel coding methods is provided in [2]. An excellent analytical This work was supported in part by NSF award #9870418. M. Bystrom is with the Department of Electrical and Computer Engineering, Boston University. T. Stockhammer is with the Institute for Communications Engineering, Munich University of Technology.
treatment of the tradeoffs between vector quantization and block channel coding is given in [3] for the binary symmetric channel. In the case of source-optimized channel coding, the source coding scheme is fixed, and an optimal channel code is designed on the basis of the source code. This method has been used in such work as [4] in which Modestino, et al. provide a rationale for combined source-channel coding using DCT-coded images with convolutional codes on AWGN channels. This work was re-visited in [5] for two-dimensional DPCM with short-constraint length low-rate convolutional codes on fading channels. In 1998 Srinivasan and Chellappa [6] considered the problem of optimal operational rate allocation for subband/vector-quantization coded images with rate-compatible punctured convolutional (RCPC) codes transmitted over fading channels. They showed that with an adaptive scheme which varies with the channel state, reasonable video quality can be achieved at low overall rates. The rate is selected by minimizing an overall operational distortion, found through a combination of simulation and analysis. In [7] an analytical expression for the end-to-end transmission distortion is found for the case of a scalable subband coding scheme with RCPC codes and unconstrained optimization is performed based on Lagrange multipliers. It is shown that due to the scalability of the source encoder, excellent results may be obtained by providing unequal error protection (UEP) to different bits in the compressed bit stream. A similar method, based on operational rate allocation was proposed in [8]. In this work the authors presented a general algorithm for distortion minimization using operational distortion-rate characteristics. The second case, channel-optimized source coding, has been considered in such work as [9] in which quantizers are optimized for transmission over noisy channels. Similar work has been investigated in [10] in which the authors consider trellis-coded vector quantization with convolutional codes based upon earlier work by Fischer and Marcellin [11] in joint trellis-coded quantization and modulation. Both of these works rely on design of the trellis-coded quantizer and then efficient mapping of the quantizer to the channel coder. In 1998 Goldsmith and Effros [12] present three methods of combined source-channel coding which involve source-optimized channel coding, channel-optimized source coding, and a joint technique employing both of the previous methods. It is shown that these methods, employing selected forms of vector quantization and 2
convolutional codes, can outperform traditional channel-optimized vector quantization. It can be seen that there exist a variety of schemes for performing joint source-channel coding. With the exception of [3] and [8], these schemes have been generally developed in the framework of specific source and channel coders. One of the goals of this work is to develop a method for optimal rate allocation for standard video encoders which can be employed without modification of the encoders. Furthermore, previous results in rate allocation for video coding have generally assumed that video frames are independently coded and have neglected error propagation between frames. However, error propagation can be a significant source of degradation in reconstructed video frames. In [13] Ramchandran, et al. were the first to introduce bit allocation during source coding assuming dependence between coding units such as image blocks or video frames. They showed that optimal source coding rate allocation is possible in an operational ratedistortion framework, providing motivation for considering the problem of dependent joint source-channel rate allocation. Whether for the independent or dependent assumption, the goal of optimal rate allocation is to distribute the available rate between source and channel coding such that the distortion between the sequence at the input to the video encoder and the sequence at the output of the video decoder is minimized. Ideally, the effects of error propagation would be mathematically modeled and these models employed to determine the optimal rate allocation. However, due to the structure of the encoded bitstream such modeling is prohibitively complex. Instead, we turn to operational allocation to distribute rate among frames in a sequence. Our approach is based upon results in [8], which presented a methodology of allocating rate between video frame components under an independent assumption, that is, it assumed video frames were independent and neglected the effects of error propagation. In [8] the argument is made that due to the large number of parameters involved, that a direct calculation of the optimal allocation of rate between video components is computationally infeasible, and that operational rate allocation is best performed using distortion-rate characteristics. To summarize this argument, in order to perform direct evaluation of the optimal rate allocation for a select set of channel conditions and a given 3
video sequence, it is first necessary to calculate the distortion between the original and the decoded sequence for each combination of desired source coding and channel coding rates. Given a restriction on the overall rate supplied by the channel, the source and channel coding rate allocation resulting in the minimum sequence distortion is selected as the optimal allocation for a fixed parameter set. If the selected set of rates or channel conditions change, then the allocation process must be repeated. Thus, this straightforward process is obviously computationally prohibitive. Therefore, it was proposed that distortion-rate characteristics be employed to make the rate allocation problem tractable. These characteristics are developed for representative video sequences, which are source coded at selected rates, and indicate the sensitivity of the compressed sequences to channel errors. When combined with typical channel coder characteristics, the well-known bit error rate versus
curves, it is
a simple matter to rapidly compute the optimal rate allocation for a given set of channel conditions and coder parameters. As channel conditions or parameters vary, little computation is required to re-evaluate the optimal rate allocation. In this work a general method of optimal source and channel rate allocation is introduced based upon the assumption of dependence between video frames. The use of distortion-rate characteristics is particularly suited to the problem of dependent rate allocation, since the number of parameters in this case is significantly larger than for the independent case. The rate allocation method provides for unequal error protection for video frames subject to a constraint on the overall rate available and selected channel and coder parameters. To limit the complexity of this otherwise difficult problem, the proposed method is based on models of the operational distortion-rate characteristics as well as models of the channel code bit error rate as a function of the bandwidth extension. The methodology is illustrated with a H.263 coded video segment, consisting of one
-frame and one or more
-frames, of the Susie video sequence. 1 The compressed video is channel coded
using rate-compatible punctured systematic recursive convolutional (RCPSRC) codes and transmitted using
In standards such as H.263 and MPEG-2 this entity is referred to as a group of pictures (GOP), while in the MPEG-4 case it is a
group of video object planes (GOV).
4
BPSK over an AWGN channel. However, the proposed rate allocation methodology is sufficiently general to be applicable to all source coding schemes which encode frames in the form of an anchor frame followed by some number of unidirectionally predicted frames, that is, an
structure, and to any channel coding
scheme. The arrangement of this paper is as follows. In the following section an overview of the rate allocation problem is presented. Section III presents the rate-compatible punctured systematic recursive convolutional codes and an exponential model of these codes. In Section IV the generation of distortion-rate surfaces for both frames and sequences is discussed. A model for the characteristics is then presented and the benefits of the model are discussed. In Section V a rate allocation method is presented and the use of the distortion-rate characteristics is clarified. Finally, in Section VI we summarize the work.
II. F ORMULATION
OF THE
R ATE A LLOCATION P ROBLEM
We introduce a general method of rate allocation in the framework of a generic video transmission system. An
-frame video sequence is input to the video encoder. The I-frames are independently coded; however, the
following P-frames are compressed using motion-compensated prediction between the frames. It is assumed that no bi-directional prediction is used. Each frame may then be channel coded separately. Furthermore, it is assumed that there is an ideal interleaver/deinterleaver between the source and channel coders so that channel decoding errors appear independent to the source decoder.
The standard video codecs under consideration assign a quantization parameter to each macroblock, thus trading between compression rate and source distortion subject to an overall source rate constraint. While quantization parameters may be assigned individually to macroblocks, in this work we concentrate on a framelevel rate allocation and select a single quantization parameter per frame. However, it can be noted that the proposed allocation method can be extended to the case of quantization parameter adaptation as well as the selection of coding modes such as slices and intra-MBs. The vector vector for an
-frame GOP where
#"%$&(')*+),-' 5
!
is the quantization
is the quantization parameter for the
.0/21
frame. The
.
source rate function for frame , denoted as the
.=/21
358: 4 "76 9 " 3 84 2" 6 9 "A; "7?@
-frame sequence is given as
bits
Similarly, the channel encoder assigns a selected channel coding rate denotes the channel code rate selection for the
B(" to each frame. Then C!B(DEB# )EB#
-frame sequence, where
B("
is the channel code rate for the
F8(4 "7GI6 H 9 ) "JEBKEB" 2 " 6 ' > w ' 8 9 F 4 "A;àM "2?@ 7" ?@ 13
>" \ H 4 "26 9 ) "JEBUD#EB"=; w ¼F Å ?@
> 8 "26 9 ' F 4 [;[M w "2?@ > 8 "26 9 ' F 4 [;[M "2?@ 2" 6 2" 6 Recall that î Å 4 and ÿ Å 4 implicitly depend upon .
' > > " îSÅ4 "26 "2?@ Å ?@!ûü ' > > " î Å4 "26 "2?@ Å ?@ ûü
øÌÀ ñ ýþ ' 7á âKc Å § ï ð ° ø À ñ ýþ B Å uN± §ï ð
(13)
The goal of the rate allocation algorithm, as stated in Formulation 1 is to find the source and channel coding vectors to minimize the average sequence distortion subject to the constraint on overall rate given by (3). Despite the decoupling of the effects of source errors and the channel errors, the second term in (13) is a function of both the source and channel coding rates, thus determining the optimum rate allocation is still an intractable problem. Therefore, we divide the optimization problem into two steps. First, for the selected quantization vectors,
, we determine the corresponding source rates
F H 9 R C);
3 84 2" 6 9 " ;pT.é')#
, and minimize the channel distortion
subject to the overall bandwidth constraint given by (3). We repeat this for each available
and for each desired rate constraint. The optimum rate allocation is then the combination of minimizes the overall sequence distortion for the available overall rate,
3 8(GIH
and
C
and ,
C
which
.
For a fixed , the first step can be performed by a Lagrangian multiplier approach with cost function
Àñ t ' > > " î Å4 "26 ° u± ø ýþ Mo 3 8(GIH u > 3 84 "76 9 " ; BÅ §ï(ð "2?@ Å ?@!ûü "7?@ B" v t ø À ñ ýþ > > Å "26 ° > 3 84 "76 9 "A; ' ( 8 I G H Å î 4 B Å u± Mo 3 u (14) §ï(ð "7?@ B" v ?@ "2? Å ûü n Taking the partial derivatives of 9 CWpo«; with respect to C and o yields the following system of equations Àñ ¶6 9 n G x6 > 2 " 6 2 " 6 ° ' ° 3 "
