REDUNDANT MOTION VECTORS FOR IMPROVED ERROR ...

REDUNDANT MOTION VECTORS FOR IMPROVED ERROR RESILIENCE IN H.264/AVC CODED VIDEO M.B.Dissanayake, C.T.E.R. Hewage, S.T.Worrall, W.A.C.Fernando, and A.M.Kondoz CCSR, University of Surrey, Guildford, GU2 7XH, UK {Maheshi.Dissanayake, E.Thushara, S.Worrall, W.Fernando, A.Kondoz}@surrey.ac.uk conceal the errors in the reconstructed frame. However, the introduction of error resilient tools to the encoded stream increases the amount of redundancy present and decreases the compression efficiency of the encoded data [3]. This in turn decreases the bandwidth utilization. Moreover, in the case of error free transmission, the error resilient information is discarded at the receiving end. Therefore, it is essential to maintain the right balance between the redundant information and video quality, to obtain performance gains under fluctuating network conditions. One of the key error resilient techniques supported in H.264/AVC is redundant picture coding [3]. In JSVM this is achieved by duplicating the primary coded data as presented by J. Jia and H Kim in [4]. However, this adaptation generates large amount of redundant data and overheads. Therefore, the primary picture quality has to be sacrificed in order to find room for the redundant information in constant bit rate applications. Thus, this scheme is potentially an inefficient error resilient technique, specifically in low bandwidth and low packet loss rate environments. Hence, the main focus of the current research is to reduce the amount of redundant information while achieving better error resilience. According to Baccichet et. al.’s proposal [5] the bandwidth utilization can be improved by employing a coarse quantization parameter for the redundant picture coding. The unequal key picture protection method proposed by C. He et. al. [6] achieves this by prioritizing the error resilient scheme and transmitting redundant information only for the key pictures. Then D. Y. Suh and G. H. Park’s proposal based on an XOR mechanism [7] led to many other [8, 9] variations on key picture protection, based on XOR logic. Moreover in the Systematic Lossy Error Protection (SLEP) scheme, parity information is transmitted instead of the whole redundant frame to maximize the bandwidth utilization [10]. Other work has shown how redundant slices can be used for Multiple Description Coding schemes [11]. However, most of the above discussed schemes increase the processing time and the complexity of the codec. Moreover some of them introduce additional delays to the system while others give less priority to the non-key pictures, which might contain important information, such as sudden changes in the visual data.

ABSTRACT This proposal presents a new error robust strategy for encoding redundant pictures for the H.264/AVC standard. The method is based on providing motion vectors as redundant data, i.e. providing extra protection to the motion information of the encoded stream. The proposed system is implemented based on the existing redundant coding algorithm of the scalable extension of H.264/AVC. The performance of the algorithm is evaluated using various objective quality measurements under both error free and error prone Internet Protocol (IP) packet network environments. The proposed algorithm increases the bandwidth utilization with slight degradation in the primary picture quality for error free conditions, compared to the existing redundant coding method of JSVM (Joint Scalable Video Model). Furthermore, the simulation results under packet loss environments show that the proposed algorithm outperforms the existing redundant picture coding of JSVM. Index Terms— Redundant picture coding, Error resilient techniques, Motion vectors, H.264/AVC 1. INTRODUCTION In multimedia communication scenarios, due to the dynamic nature of the transmission channels, errors such as packet losses, bit errors and burst errors are almost inevitable. These errors degrade the quality of the reconstructed video frames and create a mismatch between the transmitted and reconstructed frames which may propagate spatially and temporally [1]. Hence, error resilient mechanisms are employed in video communication systems, to mitigate the visual artifacts caused by packet losses. Depending on the functional characteristics, there are four types of error resilient tools: localization, data partitioning, redundant coding and concealment techniques [1]. In localization the temporal dependency between the frames is removed and in data partitioning the encoded data are grouped according to relative importance. Then in redundant coding, redundant information is encapsulated in the encoded bit-stream to enable robust decoding. In concealment driven techniques [2], measures are taken to

978-1-4244-2571-6/08/$25.00 ©2008 IEEE

25

Authorized licensed use limited to: IEEE Xplore. Downloaded on November 6, 2008 at 07:13 from IEEE Xplore. Restrictions apply.

ICME 2008

data partition 0, it increases the scope of the application of error resilience in video transmission compared to the latter because data partitioning is implemented only for the extended profile of H.264/AVC, while the redundant data transmission is implemented in the baseline profile. During the JSVM decoding process, if the primary data is lost then redundant data is used to recover the error. In the proposed method if the NALU of the primary picture is lost then the motion information is recovered using the uncorrupted redundant NALU. As residual information is discarded during the reconstruction process, the reconstructed image quality of the proposed method is lower compared to the JSVM redundant picture coding strategy. However, the redundant MV NALU of the proposed method is more likely to survive in error prone wireless environments than the duplicated primary picture NALU which is larger in size [12]. The proposed method provides error resilience to low bitrate applications and more effective in recovery process as residual information is highly compressed. As the proposed method follows the same cycle as the existing redundant coding algorithm, it is not necessary to include any new functionality to the existing codec. Moreover, it reduces the complexity, buffering delay and the processing time of the redundant coding process. For example, in the proposals described in [7] and [11], additional storage is required for the buffering of NALUs before applying the proposed scheme, which introduces a delay at the encoder and decoder ends. Since this proposed scheme doesn’t require such buffering it can be employed in real time applications as well as in offline applications.

This paper proposes a novel technique of providing error resilience by means of redundant Motion Vectors (MV). The proposed algorithm increases the bandwidth utilization and provides a significant improvement in the primary picture quality, specifically under error free conditions. The paper is organized as follows: Section 2 elaborates on the algorithm of the proposed system and Section 3 discusses the simulations results. Finally Section 4 addresses the future work and the conclusion. 2. PROPOSED ERROR RESILIENT METHOD USING REDUNDANT MOTION INFORMATION The proposed method incorporates motion information to provide robust error resilience to the transmitted data. As shown in Fig. 1, the entropy coded data of the frame consists of coded residual data and motion information. In motion compensation the current video frame is represented in terms of Marcoblocks (MB) of closely matched reference pictures. I.e. it exploits the correlation between the frames of the video sequence. Then the error of the motion compensated MB and the current MB is transmitted as the residual information. Hence of these two, the motion information plays a more important role when it comes to displaying the image properly. Therefore, the technique proposed in this paper highlights the effect of providing error resilience using redundant motion information. Input

--

Quantized transform coefficients

QT

(QT)-

1

+

Intra mode

Intra prediction

Entropy coder

3. SIMULATION RESULTS

De-blocking filter

To evaluate the proposed technique, simulations are carried out using the modified JSVM version 8.9 [13]. The simulations are performed only on the base layer level, which is H.264/AVC compatible. The encoded bit-streams with and without the existing redundant coding method of JSVM 8.9 are used as the benchmark for the analysis of the results. The simulations are carried out using three sequences, for 4000 frames at QCIF resolution. The sequences exhibit different kinds of motion and spatial complexity. For example, News features slow motion, and Soccer contains high motion. Also the channel bandwidth and frame rates are varied according to the specifications given in [14], such as Soccer with 30fps at 256 kbps, News with 10fps at 48 kbps and Foreman with 7.5 fps at 144 kbps and 64 kbps. The experiments are carried out with a Group of Picture (GOP) size of 8. The intra period is set to twice the GOP size and the maximum number of frames to be used for

Motion data

Inter mode

Motion compensation

Reference picture

Motion estimation estimati

Fig. 1. The structure of the JSVM encoder

In JSVM, redundant pictures are coded similarly to the primary pictures as shown in Fig. 1. In the proposed method, only the motion information is transmitted as redundant data. Neither motion compensation nor residual coding is performed. Hence, the redundant pictures Network Abstraction Layer Units (NALU) contain only the MVs of the primary picture. Accordingly the proposed method reduces the amount of redundant information transmitted and in turn saves the bandwidth for high quality primary data transmission. Although this scheme is similar to repeating

26 Authorized licensed use limited to: IEEE Xplore. Downloaded on November 6, 2008 at 07:13 from IEEE Xplore. Restrictions apply.

On the other hand in the proposed method the bit penalty is dependent on the type of the motion in the sequence. So for sequences with high motion, such as Soccer, the bit penalty is nearly half of that of the existing scheme. However, even in sequences with low motion, like News, the bit penalty of the proposed method is 25% less than that of the existing method. The reason attributed is that the size of the residual is considerable for sequences with high motion information.

reference pictures lists 0 and 1 is set to one. ContentAdaptive Variable Length Coding (CAVLC) is employed as the entropy coder. An IP packet Loss simulator is used to simulate packet loss scenario [15]. In the simulations, the probability of occurrence of packet loss is independent of the length of the packet and depends on the amount of congestion present in the channel. The comparisons are made in terms of the Peak Signal to Noise Ratio (PSNR), bit rate, rate distortion (R-D) relationship and the performance in the error prone channels. For the analysis, the bit penalty is calculated using the Equation (1).

Table 1. Bit penalty at average PSNR = 37 dB Sequence

§ Bit spent on error resilient inf ormation · ¸¸ × 100% -(1) bit penalty = ¨¨ © Bit spent on primary picture data ¹

Existing redundant coding method Proposed redundant coding method

3.1. Performance in error free environments

Average PSNR /(dB)

News_ proposed scheme News_ existing redundnat coding scheme News_ without redundant coding scheme foreman_ proposed scheme foreman_ exisiting redundnat coding scheme foreman_ without redundant coding scheme soccer_ proposed scheme soccer_ with exisiting redundnat coding scheme soccer_ without redundant coding scheme

30 1000

1500

2000

2500

100.0%

100.0%

100.0%

63.2%

75.6%

46.0%

In this simulation setup, if the primary NAL unit is lost then the redundant NAL unit is used to recover the error. However, if both the redundant and primary NAL units are lost, frame copy is used to conceal the lost frame. In the frame copy method the previous key frame is copied in place of the lost frame to conceal the error. According to Figs. 3 to 6, the proposed system exhibits a significant improvement, compared to the standard scheme with redundant data, for all of the packet loss rates (PLR) tested. For example, according to Fig. 5, the proposed scheme outperforms the existing redundant coding method by 1.25dB at 20% PLR. Therefore as observed from Figs. 3, 5 and 6, by using redundant motion information in sequences with relatively high motion a considerable improvement in video quality can be achieved at higher PLRs. This is due to the fact that in high motion sequences the MVs plays a vital role while the frame copy error concealment fails to recover lost details from neighboring frames. The proposed method outperforms the scheme without redundant coding at higher loss rates except in the sequence News. According to Fig. 4, in slow motion sequence at low bandwidth, frame copying achieves good quality due to its low scene motion. Because of the high correlation present between adjacent frames in slow moving sequences, it is impractical to get any gain in PSNR by means of redundant pictures. Therefore it appears to be unnecessary to employ redundant coding in such sequences.

50

500

Soccer

3.2. Performance in packet loss environments

60

0

News

Consequently the bit overheads for the proposed redundant motion vector coding method are comparatively low compared to the traditional redundant picture coding method. Therefore, by means of the proposed techniques the bit consumption as well as the primary picture quality can be significantly improved.

Fig. 2 shows the R-D curve for the sequences, Soccer, Foreman and News. The results are taken for Quantization Parameters (QP) between 5 and 45. From the results it is evident that the redundant coding method of JSVM 8.9 [4] requires a high bit rate. Although in the proposed scheme the bit rate consumption for a given PSNR value is moderately high compared to the scheme without any redundant information, for a given bit rate the degradation in primary picture quality is rather less compared to that with the existing redundant coding technique. For example in the Soccer sequence, for a predefined bit rate, the average PSNR of the sequence with the redundant pictures take a lower value compared to the proposed scheme. This is due to the fact that more texture information is coded in the redundant NAL Unit, since the correlation between adjacent frames is moderately low in high motion sequences.

40

Foreman

3000

bit rate /(kbps)

Fig. 2. Rate distortion curves

Table 1 indicates the bit penalty for both existing and proposed redundant picture coding methods at an average PSNR of 37 dB. According to Table 1, it is evident that the existing redundant coding scheme has a bit penalty of 100% independent of the characteristics of the encoded sequence.

27 Authorized licensed use limited to: IEEE Xplore. Downloaded on November 6, 2008 at 07:13 from IEEE Xplore. Restrictions apply.

40

where the bit penalty is reduced up to 46% at 37 dB compared to the existing redundant coding system. It also exhibits a significant gain in terms of average PSNR in a range of packet loss rates. Moreover, in the future this technique can be extended to enhancement layers, and tested in wireless environments such as WiMAX to establish the findings for scalable video in wireless transmission systems.

Average PSNR /( dB)

Proposed redundant coding method Existing redundant coding method W ithout redundant coding method 38

36

34

32 0

5

10

15

20

5. REFERENCES

Packet loss rate /(%)

Fig. 3. Results of sequence Soccer @ 256 kbps and 30 fps

[1] A.Vetro, J. Xin and H. Sun, “Error Resilience Video Transcoding for Wireless Communications”, IEEE Wireless Communications, 12(4), pp. 14-21, Aug. 2005. [2] L. Liu, S. Zhang, X. Ye, Y. Zhang “Error Resilience Schemes of H.264/AVC for 3G Conversational Video Services”, The Fifth International Conference on Computer and Information Technology, (CIT 2005), pp.657-661, 2005 [3] T.Stockhammer, M.M. Hannuksela, “ H.264/AVC video for wireless transmission”, Wireless Communications, IEEE, vol.12 ,pp. 6- 13issue 4, Aug. 2005. [4] J. Jia, H Kim, “Implementation of redundant pictures in JSVM” , JVT-document, JVT-Q054, Oct. 2005 [5] P Baccichet, A. Chimienti, “Error resilience by means of coarsely quantized redundant descriptions”, JVT-document, JVT-S046, Apr. 2006 [6] C. He, H. Liu, H. Li, Y. Wang, M. Hannuksela “ Redundant pictures in SVC” , JVT-document, JVT-W049, Apr. 2007 [7] D. Y. Suh, G. H. Park, “Proposal for extension of redundant picture (XRP) against loss of key pictures”, JVT-document, JVT-S015 , Apr. 2006 [8] Hendry, M. Kim, “SVC/AVC Sliding Window XOR-based Redundant Picture for Error Resilience”, JVT-document, JVT-X056, July. 2007 [9] B. S. Choi, T. S. Park,D. Y. Suh, J. S. Park, G. H. Park, K. H. Kim, C. B. Park, D. Y. Kim,Y. H. Lee, “Cross XORed Redundant picture”, JVT-document, JVT-X045, July. 2007 [10] S. Rane, P. Baccichet, B. Girod, “ Systematic Lossy Error Protection based on H.264/AVC Redundant Slices and Flexible Macroblock Ordering”, JVT-document, JVT-S025, Apr. 2006 [11] P. Schelkens, A. Gavrilescu, A. Munteanu, Yves D'Hondt, K. Vermeirsch, R. Van de Walle, O. Guye, Stewart Worrall, A. Navarro, “Error-Resilient Transmission of H.264 SVC Streams Over DVB-T/H and WiMAX Channels with Multiple Description Coding Techniques”, 15th European Signal Processing Conference, Sept. 2007 [12] P. Ferre, A. Doufexi, J. Chung-How, A. R. Nix, D. R. Bull, “Robust Video Transmission Over Wireless LANs” IEEE Transactions on Vehicular Technology, Accepted for future publication [13] JSVM 8.9 software, downloaded from CVS server, garcon.ient.rwth-aachen.de/cvs/jvt. [14] Y. Wang, S. Wenger, M.M. Hannuksela, “Common condition for SVC error resilience testing”, JVT-document, JVT –P026, July 2005. [15] Y. Guo, H. Li, Y. Wang, “SVC/AVC loss simulator donation”, JVT-document, JVT-Q069, Oct. 2005

40

Proposed redundant coding method Existing redundant coding method W ithout redundant coding method

Average PSNR /(dB)

39

38

37

36

35

34 0

5

10

15

20

Packet loss rate /(%)

Fig. 4. Results of sequence News @ 48 kbps and 10 fps 42

Proposed redundant coding method Existing redundant coding method W ithout redundant coding method

Average PSNR/ (dB)

40

38

36

34

32 0

5

10

15

20

Packet loss rate /( %)

Fig. 5. Results of sequence Foreman @ 64 kbps and 7.5 fps 46

Proposed redundant coding method Existing redundant coding method

Average PSNR /(dB)

44

W ithout redundant coding method 42

40

38

36 0

5

10

15

20

Packet loss rate /(%)

Fig. 6. Results of sequence Foreman @ 144 kbps and 7.5 fps

4. FUTURE WORK AND CONCLUSION An error resilient scheme which improves the bandwidth utilization with limited primary picture quality degradation is proposed for H264/AVC. This proposal uses motion information as redundant data to achieve the above objectives. The experimental results show that the proposed technique outperforms the existing redundant picture coding scheme of JSVM approximately by 1.25 dB at 20% PLR in high motion sequences. Furthermore, this algorithm can be effectively used for encoding highly dynamic sequences,

28 Authorized licensed use limited to: IEEE Xplore. Downloaded on November 6, 2008 at 07:13 from IEEE Xplore. Restrictions apply.