an efficient error concealment implementation for mpeg–4 video streams

AN EFFICIENT ERROR CONCEALMENT IMPLEMENTATION FOR MPEG–4 VIDEO STREAMS C´ecile Dufour, Fran¸coise Groli`ere, Daniel Snook and St´ephane Valente Video and Communication Group LEP / Philips Research

1

Introduction and Prior Art

One of the goals of the MPEG–4 standard is the transmission of audio–visual data on lossy networks, like radio channels or the Internet [1]. Although the norm describes error resilience tools, errors are still likely to damage the visual quality of the decoded streams. Therefore, efficient error concealment techniques are needed on the decoder side to mask such impairements and ensure the consumer acceptance of MPEG–4 devices. Because these techniques are optional, they also represent one of the possible sources of added value in MPEG–4 decoders. Many references and surveys can be found about the video error concealment problem, like [2]. However, most do not address the specificities of MPEG–4 streams, which need • low–cost algorithms, to fit on mobile terminals and sustain real–time display rates; • to interpolate 16×16 macroblocks and not simply 8×8 blocks; • and to cope with burst errors, which can span several rows of macroblocks. In sections 2 and 3, we present our current implementation specifically designed for MPEG–4 streams, and in section 4, we demonstrate how our concealment implementation can benefit from an optimized state–of–the–art encoder, compared to the reference ACTS–MoMuSys implementation.

2

Implementation

Our concealment implementation relies on the residual spatial and temporal redundancies found in the decoded bitstream. Spatial Concealment: depending on the spatial activity of the decoded neighborhood, we interpolate the missing data in INTRA VOPs

using variations of bilinear interpolation [3] and restoration in the frequency domain [4] (see figure 1). Temporal Concealment: we interpolate the neighboring motion vectors and use them to compensate the damaged macroblocks with data from previous VOPs [5] (see figure 2). The techniques that we retained were chosen both for their interpolation capabilities and their computational efficiencies. When burst errors occur, groups of consecutively damaged macroblocks are processed one by one, and the concealment algorithms are applied bidirectionally from the first and the last macroblocks of the group. This strategy progressively propagates the valid neighboring information inside the missing areas.

3

Results

We compared our concealment to a more basic one, which consists of spatial copy in INTRA VOPs and concealment by zero motion vectors in INTER VOPs. The results measured on 61 seconds of video (1525 VOPs at 25 Hz) mixing several sequences are found in table 1. The sequence was encoded with the MoMuSys reference encoder (June 2000 version) using CIF resolution, 800 kbits/s, 25 Hz, INTRA VOP every 50 VOPs and video packets of 500 bytes. Neither data partitioning nor reversible variable length codes were used. 1%, 5% and 15% of video packet loss rates were simulated. At the highest loss rate, we obtain a 3.76 dB increase over the basic technique.

4

Encoder Impact

The quality of the encoder can also influence the quality of the concealment. To compare several

Bitstream Basic concealment Efficient concealment Gain over basic

Table 1: PSNR results of 1525 concealed VOPs. 1% of damaged VPs 5% of damaged VPs 15% of damaged VPs 32.15 27.07 22.40 33.13 30.20 26.16 +0.98 +3.13 +3.76

bitstreams, the sequence of section 2 was re– compressed with the encoder internally developed at Philips (which is now found in commercial solutions such as [6]) using the same encoding parameters. Table 2 shows that the Philips encoder has a 0.54 dB better PSNR than MoMuSys when no errors occur. In presence of errors, the PSNR gain gets even larger, because of various encoding optimizations (including smarter macroblock coding decisions and motion vector estimation) which help the concealment process and stop the temporal propagation of errors.

References [1] Information technology — coding of audiovisual objects — part 2: Visual — amendment 1: Visual extensions, December 1999. ISO/IEC 14496-2:1999/Amd.1:2000(E). [2] Y. Wang and Q.-F. Zhu. Error control and concealment for video communication: A review. Proceedings of the IEEE, 86(5):974–997, 1998. [3] S. Aign and K. Fazel. Temporal & spatial error concealment techniques for hierarchical MPEG–2 video codec. In Proceedings of Globecom’95, pages 1778–1783, 1995. [4] Y. Wang, Q.-F. Zhu, and L. Shaw. Maximally smooth image recovery in transform coding. IEEE Transactions on Communications, 41(10):1544–1551, 1993. [5] J.-W. Suh, E.-T. Kim, S.-J. Choi, H.-B. Park, and Y.-S. Ho. Motion vector recovery for error concealment. In SPIE Conference on Visual Communication and Image Processing, volume 3653, pages 667–676, San Jose, CA, January 1999. [6] Philips WebCine. 4player.com

URL http://www.mpeg-

Figure 1: Spatial concealment of an INTRA VOP — from left to right: damaged, concealed and error–free reconstruction.

Figure 2: Temporal concealment of an INTER VOP — from left to right: damaged, concealed and error–free reconstruction.

Table 2: PSNR results of similar bitstreams produced by different encoders. Bitstream Basic concealment Efficient concealment

Error–free MoMuSys Philips 34.37 34.91 34.37 34.91

gain +0.54 +0.54

1% of damaged VPs MoMuSys Philips gain 32.15 33.36 +1.21 33.13 34.09 +0.96

5% of damaged VPs MoMuSys Philips gain 27.07 28.05 +0.98 30.20 30.84 +0.64

15% of damaged VPs MoMuSys Philips gain 22.40 24.41 +2.01 26.16 27.30 +1.14