An FEC-based error control scheme for wireless MPEG ... - IEEE Xplore

An FEC-Based Error Control Scheme for Wireless MPEG-4 Video Transmission Jianfei Cai* Qian Zhang+ Wenwu Zhu+ Chang Wen Chen* Abstmct- In this paper, we propose an FEC-based error control scheme for the transmission of low bit rate MEPG-4 video over wireless channels. The proposed error control scheme, through integrating with MPEG-4 inherent error resilient techniques, divides MPEG-4 bitstream into several classes. Such division of the bitstream not only facilitates unequal error protections for different classes of data but also allows local reorganization of the bitstream into a flxedlength structure. This reorganization enables robust decoding at the receiving end, however, results in little side information and causes negligible delay. T h e proposed scheme can also be combined with motion-compensation based error concealment for effective post processing. Experimental results demonstrate that the performance of the proposed scheme is much better than that of the simple equal error protection scheme, not only in PSNR but also in visual quality, in terms of the reconstructed video frames at the receiver.

I. INTRODUCTION Supporting wireless video communications is a challenging task primarily because of two factors: (1) limited bandwidth; (2) time-varying error-prone environment. Since video signals usually have a huge volume of data that is in strong contrast with limited bandwidth of the wireless channels, it is very important that we apply highly efficient compression schemes t o video signals before transmission over wireless channels. However, it is well known that the compressed video bitstreams are extremely sensitive to transmission errors. Therefore, we need to add controlled redundancy t o the compressed bistreams so as to provide a reliable transmission in the error-prone wireless environment. It is natural that there exists a tradeoff between bandwidth efficiency and transmission reliability. MPEG-4 is an audio-visual coding standard being developed by ISO/IEC Moving Picture Experts Group (MPEG), and is initially targeted for low bit rate video communications [l]. In order to make the compressed bitstream more robust to channel errors, the MPEG-4 video compression standard incorporates several error resilient tools, i.e., resynchronization markers, data partitioning, reversible variable length coding (RVLC) and header extension codes (HEC). Although with these tools, an MPEG-4 video This research is supported by University of Missouri Research Board Grant URB-98-142. 'Dept. of Electrical Engineering, University of Missouri-Columbia, Columbia, MO, 65211,USA. Email: {cai,cchen}Oee.missouri.edu +Microsoft Research, No. 49 Zhichun Road, Haidian District, Beijing, 100080,China. Email: {qianz,wwzhu}Omicrosoft.com

0-7803-6596-8/00/$10.000 2000 IEEE

stream can be decoded with acceptable video quality at a bit rate of or lower [2], wireless channels typically have a higher bit error rate (BER). In addition, wireless channels are generally time-varying, resulting bursty pattern of bit error occurrence. Therefore, error control techniques such as forward error correction (FEC) and automatic repeat request (ARQ) are necessary t o ensure high quality wireless video transmission. Because of the strict delay constraints for real-time video transmission, it is often considered more beneficial to apply FEC than to use ARQ.

It is well-known that different portions of MPEG-4 bitstreams have different importance t o the quality of the reconstructed video. Therefore, unequal .error protection (UEP) is more appropriate for such compressed bitstreams. To accomplish the UEP, the original compressed bitstream would need t o be classified into different classes and different error protections are needed for different classes of the video bitstream. However, the implementation of such a UEP scheme is not a trivial task since different class data is unevenly distributed within the bitstream. The length and position of each portion cannot be easily determined. In order to separate different classes within the bitstream for UEP, we need to label the boundary of each portion of each class data. This would lead to significant side information that needs to be transmitted to the receiver for decoding purpose. In addition, new problems would arise when we need t o guarantee that the side information is transmitted error-free and t o synchronize the transmission of the side information with the video bitstream itself. An alternative would be to cluster the same class of data from the entire video sequence into a single portion so as to avoid the side information. However, this is not feasible for many delay-constrained real-time video applications. Recently, Heinzelman et al. introduced a UEP scheme for MPEG4 video transmission in [3]. Nevertheless, this problem of synchronization has not been addressed. In this paper, we proposed a novel unequal error protection scheme that is able to incorporate UEP for reliable wireless MPEG-4 video transmission with little side information and negligible delay. In this scheme, an MPEG-4 bitstream is divided into many segments, and each segment is reorganized into a fixed-length structure. Then, different FEC protections are applied to different classes of data based on their relative importance with respect t o video frame reconstruction at the receiver. In this case, there is 1243

no need to transmit the side information that indicates the boundaries between different classes of data, and the delay caused by such data reorganization is negligible. STATEMENT 11. PROBLEM In this research, we focus on the simple profile of MPEG4 standard in error-resilient mode, where data partitioning (DP) is enabled. As shown in Fig. 1, there is only one video object (VO) and one video object layer (VOL). Each VOL consists of many video object planes (VOP), and each VOP consists of many video packets (VP). Each VOP starts with a start code (STC), each VP starts with a resynchronization marker (RESYN), and motion and texture data in each VP are separated by a motion boundary marker (MBM).

This is because the problem of synchronization between different clusters still exists. For example, when we separate Class l data from Class 2 data within N , VPs, all the Class 1 data contains the header information while Class 2 data does not. In other words, Class 1 data contains the absolute address information that is included in the header, while Class 2 data contains no such information. Therefore, we will have t o repeat part of the header information in each Class 2 data. Otherwise, even if the cluster of Class 2 data loses synchronization for a short period such as missing only one of Class 2 data segment, the decoder will put all the following Class 8 data in wrong position. To overcome the problems related to UEP as it applies t o MPEG-4 video transmission, we propose a novel unequal error protection scheme in which, instead of clustering the bitstream data, we locally reorganize the bitstream into a fixed-length structure so that the size of the side information can be greatly reduced.

111. A FEC-BASEDERROR CONTROL SCHEME The proposed wireless video transmission system is shown in Fig. 2, with a network transmission point of view. An error control layer is inserted between MPEG-4 video applications and the network. Notice that we assume a UDP-like protocol is employed in this system in which there is no checksum operation and the corrupted packets are presented directly t o the application layer. We choose this UDP-like protocol instead of TCP because, in a highly noisy environment, the error control scheme of Fig. 1. Error-resilient MPEG-4 bitstream structure. TCP will cause long delay and the unrecovered packets will be discarded. With this UDP-like protocol, we can For the purpose of simplicity in analysis, we classify implement error control in the application layer, where the the bitstream into two classes. Class 1 Data includes source characteristics can be fully considered. The function I-frames, VOP header, VP header, motion and scene- of this error control (EC) layer is to reorganize the MPEG-4 changed frames. Texture data belongs t o Class 2 Data. compressed bitstream or add necessary control redundancy VP is chosen as the basic processing unit. An MPEG-4 according t o current network or channel conditions so that bitstream can be viewed as a huge collection of VPs. No- the output bitstream of EC layer will be more robust to tice that the first VP in a VOP is started with VOP header channel errors. At the receiver end, after the processing of instead of VP header. In a UEP system, the channel cod- EC layer, the output bitstream will be compatible with the ing rates vary with different classes of data. Therefore, the MPEG-4 syntax. channel decoder needs t o know the boundaries between two different channel coding rates. As shown in Fig. 1, the boundaries of different classes of data are determined I by RESYN, MBM or VOP STC. However, these special codes cannot be used directly to determine the boundaries of packets adopted for wireless transmission, because they have also been protected by channel coding. As we have discussed in Section I, we need t o insert tremendous Fig. 2. System structure. amount of synchronization codes identifiable by the channel decoder for successful channel decoding. A possible solution to such problem is to locally cluster the same class The basic idea of this proposed UEP-based error control data by VOP or a group of VPs. However, the method scheme is shown in Fig. 3. The MPEG-4 bitstream is reorof locally clustering data is still not an efficient approach. ganized into the EC frame structure. Each EC frame is a

\o UDWIP

UDP/IP

Wile,-%

1244

Nctwolk

segment of the original bitstream. Each EC frame consists of an EC Header and N , Slots. Each Slot consists of Class 1 Subslot and Class 2 Subslot. The length of EC Header is k e d for all the EC frames. The length of each Class 1 Subslot and Class 2 Subslot can be determined for each EC frame. There is one Slot per VP. The EREC approach [4] is applied t o assign different classes of data from N , VPs into the N , Slots of an EC frame. First, we put into each subslot as much data as possible from corresponding class. Then, the leftover data is assigned t o the remaining space in the subslots of the same class. In this way, the starting position of Class 1 Subslot in a Slot is the same as the starting position of Class 1 Data in the corresponding VP. Similarly, the starting position of Class 2 Subslot in a Slot is the-same as the starting position of Class 2 Data in the corresponding VP. Furthermore, all the Class 1 Data has been put in Class 1 Subslots, while all the Class 2 Data has been put in Class 2 Subslots.

C l a n 1 Subrla

I

The definition of the EC Header is shown in Fig. 4 and Table I. The Class and Change fields act as switches to choose different approaches. For example, if all data in the current EC frame belongs t o one class, such as I-frame data, equal error protection (EEP) will be applied. N , indicates the number of VPs in this EC frame, while T, indicates the length of this EC frame. If there are two classes of data in the current EC frame, UEP will be applied. T,1 indicates the length of Class 1 Data, while Ts2 indicates the length of Class 2 Data. If the channel coding rates are not the same as the previous EC frame, an extended EC header will follow. This extended EC header contains the new channel coding rates. Notice that in the case of UEP, if the number of VPs in an EC frame is not equal t o a default value, N , will be also sent in the extended EC header. Having defined N,, T,1 and T,2 in the EC header, it is necessary to clarify how to determine the length for each subslot. Let us denote Slj,j = 1 , . . . ,N , - 1 as the length

I

I

I

I

13

16

Fig. 4. The error control (EC) header structure. TABLE I THEDEFINITIONOF ERROR CONTROL

HEADER.

of the j-th Class 1 Subslot. Slj can be defined as "j

Fig. 3. The proposed unequal error protection scheme.

I

2

Bits

=

+ { SI, SI,

1 0 5 j < (T,1 mod N,) (T,1 mod N , ) 5 j < N ,

where S1, = L?]. The length of Class 2 Subslot S2j can be determined in the same way. With such configuration of EC frame, data are reorganized into a fixedlength structure. The order of bit transmission remains unchanged as from left t o right and from top to bottom. UEP and packetization can be easily applied t o such fixedlength structure. At the receiver, after decoding an EC header, the decoder knows where t o chop a fixed-length packet for channel decoding. Therefore, the proposed error control scheme is able to incorporate the unequal error protection for high performance transmission, and, with a small overhead, determine the boundaries of different class of data for appropriate channel decoding. For example, in the case of video sequence in QCIF format, if we assume each EC frame contains 10 slots, then, we will need 24 bits of overhead side information for each EC header. However, in the case of local clustering approach, in addition to the EC header, it will also need 7 bits for each Class 2 Data t o indicate the absolute address of the first MB in a VP. Therefore, the total overhead for local clustering approach will be 7 x 10 24 = 94 bits. The amount of overhead in this case is almost 4 times of the overhead in the case of the proposed EC scheme. The proposed UEP-based error control scheme can also be effectively integrated with an error concealment scheme at the MPEG-4 decoder to provide an improved quality of received video. Usually, when a corrupted video packet

+

1245

Y Component

that has not been fully corrected in the process of channel decoding is detected, the entire video packet is discarded. The lost frame content is often replaced by corresponding pixels from the previous frame. When UEP is applied, motion data are received much higher protection than texture data. The error probability of motion data becomes much lower than that of texture data. Therefore, at the MEPG-4 decoder, motion-compensated error concealment will become more dominant than error concealment without motion-compensation. In particular, instead of being replaced by corresponding pixels from the previous frame, the corrupted frame content can be replaced by motioncompensated pixels from the previous frame. Such error concealment scheme will certainly provide better video quality at the receiving end. Moreover, the proposed system is an adaptive scheme so that the channel coding rates can be designed to change at each EC frame. Because the length of each M P E G 4 VP is approximately the same, the length of each EC frame will also be approximately the same. Therefore, the adaptive rate control can be updated regularly t o synchronize with the MPEG-4 VP structures.

? C

35

30 25 20 15

10

_ ...................

................................

‘

I

I

I

110 120 130 1 4 0 T o t a l Bandwidth (kbps)

100

U Component 40 p: 35

z

01 P,

30 ................

25 20 100

110 120 130 140 T o t a l Bandwidth (kbps)

IV. EXPERIMENTAL RESULTS We have conducted experiments based on 150 frames of “Foreman” in QCIF format as the test video sequence t o verify the performance of the proposed UEP-based error control scheme. The frame rate is 15 frames/s, and the source bit rate R is 100 kbps. The designed VP length is 93 bytes, and there is one I-VOP per 50 VOPs. A specific GEC model [5] is adopted to generate bursty characteristics of wireless channels. The average BER of the channel is 0.01, and the average burst length is 8 bits. We employ RCPC/CRC channel codes to protect the EC headers with interleaving. The design of the interleaving is based on each EC frame so that the length of the interleaving is equal t o N,. With such error protection procedures, we may assume the EC headers are received error-free. Furthermore, the VOL header and the first I-frame are assumed received error-free. Such assumption is reasonable because in practice we can always find a timing to start, or we may use ARQ to guarantee that the short time interval at the beginning of the transmission is error-free. The rest of the MPEG-4 bitstream are protected with ReedSolomon codes suitable for wireless bursty channels. Fig. 5 shows the results of average PSNR over 30 simulations. The z axis represents the total bandwidth after channel coding. For example, total bandwidth of 110 kbps represents 100 kbps of fixed source coding rate with 10 kbps of channel coding rate. In this case, the channel coding redundancy is 10%. The redundancy of other cases can be derived similarly. The 100 kbps points in each of the Y, U, and V components represent the results obtained without

I

V Component

40 35 d z 30 ZI) P, 2 5

.......

20

15 1. O O

110 120 130 140 T o t a l Bandwidth (kbps)

6

Fig. 5. Average PSNR results of transmiting QCIF Foreman Over the bursty channel.

channel coding. The experimental results demonstrate that in bandwidth-stringent cases, UEP is much better than EEP with the average gain of about 5 dB at 10% redundancy. When the channel coding redundancy increases to 20% or more, the performance of EEP is nearly the same as that of UEP. This is because high redundancy channel coding provides virtually error free environment for all classes of the video bitstream so that the advantage of unequal error protection is diminishing. Fig. 6 shows several reconstructed VOPs with 17.2% redundancy. The same error pattern generated by the GEC model is applied to both UEP and EEP schemes. In the 3rd, 7th,and 42nd VOP, there exist corrupted motion data and corresponding error propagation. Such error corruption and propagation 1246

overhead and negligible delay. The experimental results demonstrate much improved performance comparing with the equal error protection-based scheme. Future work will include several design issues related t o the characteristics of the wireless channels and video signals. First, the power constraint at a wireless mobile terminal needs t o be taken into account when the UEP is applied since UEP will generally require more power consumption than EEP. This is particularly necessary when the bandwidth is not quite stringent and the gain of UEP over EEP is not so significant. Therefore, a decision needs t o be worked out t o determine whether the UEP shall be adopted under a given channel condition. Second, the optimal allocation of channel coding rates among different classes of video data is still an open problem since an analytical relationship of the error sensitivity among different class of video data is unknown. Without such analytical relationship, an optimal allocation cannot be easily designed.

ACKNOWLEDGMENTS The authors would like t o thank Mr. Brandon Schwartz, Dr. Inald Lagendijk and Mr. Lei Cao for proofreading and helpful discussion.

REFERENCES

Fig. 6. Reconstructed Foreman images with 17.2% redundency. Lee: EEP; Right: UEP. From top to bottom: 3’d VOP; Ph VOP; 42nd VOP; 50th VOP.

cannot be recovered by the EEP and error concealment at the decoder leads t o the distorted faces. However, for UEP-based error control scheme, since the motion data are highly protected, the channel errors in motion data can be corrected. Although there exist corrupted texture data, it can be well concealed by the motion-compensation-based error concealment as discussed earlier. For the 50th VOP, this is an I-VOP. Each I-VOP in the UEP scheme is highly protected, while the EEP scheme treats I-VOP the same way as other VOPs. Therefore, the channel errors in these segments of data lead to a few lost blocks in the case of EEP scheme.

[l] ISO/IEC JTC 1/SC 29/WG 11, “Information technology generic coding of audio-visual objects - part 2: visual,” W G l l N2688, March 1999. [ Z ] S. Gringeri, R. Egorov, K. Shuaib, A. Lewis, and B. Basch, “Robust compression and transmission of MPEG4 video,” in ACM MM 2000 Electronic Proceedings, June 2000, http: //woodworm.cs.uml.edu/ rprice/ep/gringeri. [3] W. R. Heinzelman, M. Budagavi, and R. Talluri, “Unequal error protection of M P E G 4 compressed video,’’ in Pmceedings of IEEE ICIP99, Oct. 1999, Japan. [4] D. W. Redmill and N. G. Kingsbury, “The EREC: an errorresilient technique for coding variable-length blocks of data,” IEEE h n s . on Image Processing, vol. 5, No.4, April 1996. [5] H. S. Wang and N. Moayeri, “Finitestate Markov channel - a useful model for radio communication channels,” IEEE h n s . on Vehicular Technology, vol. 44, No.1, pp. 163-171, Feb. 1995.

V. CONCLUSION AND DISCUSSION In this paper, we proposed a novel unequal error protection scheme for MPEG-4 video transmission over wireless channels. This scheme can be implemented with very small 1247