USING DISTRIBUTED SOURCE CODING AND DEPTH IMAGE ...

USING DISTRIBUTED SOURCE CODING AND DEPTH IMAGE BASED RENDERING TO IMPROVE INTERACTIVE MULTIVIEW VIDEO ACCESS Giovanni Petrazzuoli, Marco Cagnazzo, Frederic Dufaux, B´eatrice Pesquet-Popescu TELECOM-ParisTech, TSI department 46 rue Barrault, F-75634 Paris Cedex 13, FRANCE ABSTRACT view demands

Multiple-views video is commonly believed to be the next significant achievement in video communications, since it enables new exciting interactive services such as free viewpoint television and immersive teleconferencing. However the interactivity requirement (i.e. allowing the user to change the viewpoint during video streaming) involves a trade-off between storage and bandwidth costs. Several solutions have been proposed in the literature, using redundant predictive frames, Wyner-Ziv frames, or a combination of them. In this paper, we adopt distributed video coding for interactive multiview video plus depth (MVD), taking advantage of depth image based rendering (DIBR) and depth-aided inpainting to fill the occlusion areas. To the authors’ best knowledge, very few works in interactive MVD consider the problem of continuity of the playback during the switching among streams. Therefore we survey the existing solutions, we propose a set of techniques for MVD coding and we compare them. As main results, we observe that DIBR can help in rate reduction (up to 13.36% for the texture video and up to 8.67% for the depth map, wrt the case where DIBR is not used), and we also note that the optimal strategy to combine DIBR and distributed video coding depends on the position of the switching time into the group of pictures. Choosing the best technique on a frame-to-frame basis can further reduce the rate from 1% to 6%. Index Terms— distributed video coding, interactive TV, 3D-TV, multiview video plus depth, depth map, depth image based rendering, inpainting 1. INTRODUCTION In recent years, progress in cinema and TV is increasingly pushing towards immersive television [1] where users have the impression of being present at a real event. A very promising application is free viewpoint television (FTV) [2]. It gives to the user the 3D perception of the captured scene with the possibility of changing his viewpoint. This can be obtained by using 𝑁 Z-cameras that capture the color image as well as a so-called depth map (multiview video plus depth, MVD). The latter can be used to generate intermediate views by depth image based rendering (DIBR) [3]. Obviously, MVD has a huge redundancy: not only in time, as ordinary mono-view video, but also among views (inter-view correlation) and between views and depth maps. All these kinds of redundancies have to be exploited in order to reduce the storage space on the server and the bandwidth used for transmission [4]. These two requirements are equivalent in the context of non-interactive scenarios (like TV broadcasting), when all the video stored on the server will be sent to the user. For example, all the views are sent when MVD is used on a autostereoscopic display. On the contrary, the interactive multiview video streaming (IMVS) [5] is a paradigm that

Multiview video

Video

requested video

server

view demands

encoder

user 1

user N

requested video

Fig. 1: In the IMVS paradigm the video is first pre-encoded, stored in a video server and afterwards sent to the users, according to their requests

enables the client to select interactively the view that he/she wants to display. Given this constraint, the server will send only the data needed to display the views according to the switch pattern decided by the user. However, the video is first encoded and stored in a server and afterwards it is sent to the clients (see Fig. 1). We would like to minimize both the storage space demanded by the compressed video and the bandwidth needed to interactively send the requested view to the user. These requirements are conflicting in the case of IMVS [5], which makes the problem challenging. In this paper we deal with the problem of IMVS in the special case of MVD, which has been only sporadically treated in the literature. The problem background and the state of the art are given in section 2. Then, in section 3 we propose a number of techniques based on DIBR and on distributed video coding. Finally, in section 4 we compare them and we determine the best strategy for allowing view-switching in our context. In section 5 we draw some conclusions and present possible future works. 2. STATE OF THE ART Let us consider a client switching from the view 𝑣1 to 𝑣2 . The images of 𝑣2 cannot be encoded using previous images from the same view, since the decoder will not have them. Therefore, two contrasting approaches emerges: on the one hand, we could reduce the bandwidth requirement if for any view, any image is coded 𝑁 times, using any other views as reference. In this case the storage requirement is multiplied by a factor 𝑁 (since at the encoding time we do not know which view the user will choose at any time), but the required bandwidth is minimized, since we can send, for the current image, only the residual with respect to the images we have already sent. This approach is called Redundant P-frames. On the other hand, we could encode each image only once but as an Intra frame, thus reducing the storage space. However in this case the bandwidth requirements are more demanding, since I-frames need much more bits to be encoded than P’s do for the same quality. This approach is called I-frames [5].

An alternative approach exploits principles coming from distributed video coding (DVC) [6]. Each view is coded (independently from others) with a DVC technique, such as DISCOVER [7]: some images are intra-coded (they are called Key Frames, KF), while for some other (Wyner-Ziv Frames, WZF) we only send the parity bits of a systematic channel code. At the decoder side, KFs are decoded as usual, and used to estimate the missing WZFs. For example, one out of four frame is a KF, and the three middle WZFs are estimated using motion-compensated image interpolation between current and next KF (such as DISCOVER [7] or HOMI [8]). We explicitly remark that two images are needed in order to perform image interpolation. Finally, the parity bits are used to improve the quality of the estimation, which can be seen as a “noisy” version of the actual WZF. The differences between the estimation (called side information, SI) and the WZF are corrected using the parity bits. This approach can be effectively used in IMVS. When a user switches to a novel view, the next image to be displayed can either be a KF or a WZF. In the first case, it is decoded as usual; in the second one, we just have to adapt the mechanism of SI production, but we do not change the encoded representation of the frame stored on the server, i.e. the parity bits of the frame. This is interesting, since on one hand we improve the storage cost (only one version of any view is stored on the server), and on the other hand we do not send I-frames but only WZFs, which in theory could require as few bits as a P-frame, while in practice have an intermediate coding rate between I’s ans P’s [6]. Of course, any future novel and better DVC scheme will have an immediate impact on this kind of scheme for IMVS. Cheung et al. [5] propose to insert the Wyner-Ziv Frames, used as M-Frames (Merge Frames), in the video stream. In this case, the side information is the previous frame available at the decoder. They find an optimal combination of I-frames, Redundant P-frames and M-frames for the view switching, but they are interested in the case of multiview video coding without the depth information. To the best of our knowledge, only a few papers deal with IMVS in MVD with the constraint of ensuring that the video playback is not interrupted during switching. In MVD the depth maps are usually coded independently of the texture images. Kurutepe et al. [9] propose to send to the user also lower bit-rate versions of a set of adjacent views (texture plus depth) in addition to the currently required view, in order to alleviate adverse effects of the unavoidable delay between the time a client requests a new stream and the time it becomes available. Yang et al. [10] propose to estimate the disparity map between different views. Differently from DIBR, disparity based algorithm can be applied also when camera parameters are not available. With this algorithm, they can simply generate virtual views in real time. We conclude this section with some remarks about DIBR and the problem of occlusion filling. Without loss of generality, we consider a pair of cameras. Let (𝑢1 , 𝑣1 ) be the projection of a point on the camera 1 image plane. The position (𝑢2 , 𝑣2 ) of the same point on the camera 2 can be obtained by the knowledge of the intrinsic and extrinsic parameters of the two cameras. This allows to obtain a synthesis of the depth map and texture for the second view given the first one. The second camera can be a real camera or a virtual viewpoint. In this paper we refer only on the first case: then, this synthesization will be used only as estimation for the second camera. Obviously, there are points of the second view that are not visible in the first view (occlusion points): this causes the presence of occlusion areas in the synthesized image. These areas can be filled by inpainting techniques [11]. In particular, the depth maps, due to their smooth nature, can be inpainted by Bertalmio technique [12], which is based on isotropic diffusion by using the Navier-Stokes and fluid dynamics

Name 𝑚 𝑘 𝑁 𝐼𝑛 𝐽𝑛 𝐼˜𝑛 𝐼ˆ𝑛

Description switching time: user wants view 1 up to 𝑚 − 1 and then view 2 time of the KF of the GOP affected by the switch on view 2. 𝑘 ≤ 𝑚 GOP size; for simplicity we only consider the case 𝑁 = 4 decoded frame for the first view at instant 𝑛 an estimated frame of the target view, taken at time 𝑛 and used as reference for motion interpolation for the remaining WZFs of the GOP estimation of the second view at time 𝑛 obtained by depth-aided DIBR [14] on 𝐼𝑛 𝐼˜𝑛 corrected by parity bits

Table 1: Notation

equations. On the other hand, for the texture image, one of the most popular technique is the Criminisi inpainting [13]: in this algorithm the texture is inpainted in the isophote direction. In the case of large baseline between the two cameras, Daribo and Pesquet-Popescu [14] modify the Criminisi inpainting by introducing a term related to the depth: in fact, the depth helps to distinguish if a pixel belongs to foreground or background and give higher priority to the patch that overlays at the same depth level. We will use this technique in order to perform an enhanced DIBR in our proposed methods. This will allow a better quality for both the texture image and the depth map. 3. PROPOSED METHODS FOR INTERACTIVE MVD For the sake of simplicity, we consider the case where we have two Z-cameras, there is a KF out of 𝑁 images in time domain (i.e. a GOP size of 𝑁 ), and the KFs on the two views are shifted by half a GOP. For each view the texture and the depth maps are coded independently from the other view, e.g. using DISCOVER. This will assure an efficient use of server storage, since DISCOVER performances are better than all-Intra coding [7]. However we note that, since the encoding process is performed off-line, at that time we do not know which views will be used by the decoder to estimate the WZFs, so we do not know how many bits it will demand. Therefore we have to store all the parity bits, which can be more than the bits that will actually be demanded at the decoding time. However this is unavoidable in the Wyner-Ziv Frame approach. Still, this represents less than storing all possible P residuals in a classical coding framework. We have to choose what information is send to the decoder when the user makes a switch between two views, and how this is used. We propose six different techniques, exploiting DIBR and DVC. In order to describe the proposed methods, we introduce the following notation. We call 𝑚 the instant of the switch: that is the user wants view 1 up to the time 𝑚 − 1, and then he/she wants the other view. We call 𝑘 the instant when the GOP of frame 𝑚 begins in view 2: in other words the previous KF for the target view happens to be in 𝑘. Because of the periodical GOP structure, we have to consider only the cases 𝑚 = 𝑘, 𝑘 + 1, 𝑘 + 2, . . . 𝑘 + 𝑁 − 1, namely the cases when the switch corresponds to a KF or to one of the 𝑁 − 1 WZF of the GOP. However, since the GOP structures of the views are shifted by half a GOP, the frames globally concerned by the switch are those from 𝑘−𝑁/2 (beginning of the current GOP on view 1) to 𝑘+𝑁 −1 (end of the current GOP on view 2). Moreover, we call 𝐽𝑛 the reconstructed image for the second view used for creating the side information. This image will not necessary be displayed, but will be used, together with the KF 𝑘 +𝑁 that will always be sent (𝑚 < 𝑘 + 𝑁 by definition), to produce the estimation for the WZF of the current GOP in the target view. We call this image the reference image. Finally we denote by 𝐼˜𝑛 the estimation of the target view at time 𝑛 obtained by only using

𝑚

𝑘 0000000 1111111 1111111 0000000 0000000 1111111 0000000 1111111 0000000 1111111 0000000 1111111 0000000 1111111

0000000 1111111 1111111 0000000 0000000 1111111 0000000 1111111 0000000 1111111 0000000 1111111 0000000 1111111

(a)

(b) 𝑚

𝑘

𝑚

𝑘

𝑚

𝑘

000000 111111 111111 000000 000000 111111 000000 111111 000000 111111 000000 111111

(d) 𝑚

𝑚

𝑘

1111111 0000000 0000000 1111111 0000000 1111111 0000000 1111111 0000000 1111111 0000000 1111111 0000000 1111111

(e)

time for reference image 𝑚−1 𝑚−1 𝑚 𝑚 𝑘 𝑘

(f)

Fig. 2: Solutions for 𝑁 = 4 and 𝑚 = 𝑘 + 2: (a) A-DIBR; (b) ADIBRc; (c) I-DIBR; (d) I-DIBRc; (e) GOPp; (f) noDIBR. The KFs are in red and the WZFs in blue. The frames with green background are displayed. The filled frames are sent by the video server to the user. The black arrow shows that DIBR is applied.

depth-aided DIBR [14] on the first view, and with 𝐼ˆ𝑛 an improved version of this image, obtained by using the parity bits of the second view image to correct 𝐼˜𝑛 . The introduced notation is summarized in Table 1 for ease of reading. Now we can conceive several methods for IMVS using DIBR. They are shown in Fig. 2 for 𝑁 = 4 and 𝑚 = 𝑘 + 2, but this can be easily generalized to any 𝑁 and 𝑚. A first one, that we call Advance DIBR (A-DIBR) consists in computing 𝐽𝑚−1 at the decoder by using DIBR on the last received image from the first view. Next images in the GOP will be interpolated using 𝐽𝑚−1 = 𝐼˜𝑚−1 and the next KF on the target view, and this side information will be corrected with the parity bits of the target view. A variation of this method, that we can use only when 𝑚 ∕= 𝑘 + 1 consists in using the parity bits in order to improve the reference image: 𝐽𝑚−1 = 𝐼ˆ𝑚−1 . This second method is not necessarily better than the first one, since the higher reference quality is traded-off with a higher coding rate (the parity bits were not sent in the previous case). We call this method Advance DIBR + correction (A-DIBRc). Another couple of methods is obtained if we perform DIBR at switch time instead of the previous instant. This means that we send to the decoder the parity bits (or the key frame) of the first view at time 𝑚, so that it can reconstruct the latter and use it to perform DIBR, without (Immediate DIBR, I-DIBR) or with sending the parity bits for view 2 at time 𝑚 (Immediate DIBR + correction, I-DIBRc). We can also think about preserving the GOP decoding structure in the second view. If we compute the previous KF on the GOP of the target view by DIBR, then we can use it to perform image interpolation in the GOP. In this case we just need to compute 𝐽𝑘 as 𝐼˜𝑘 . We call this method GOP preserving with DIBR (GOPp). The last method does not use DIBR. This method consists in directly sending the key frame of the current GOP for the target view. With respect to the GOPp method, it demands more rate but provides a better representation of the reference frame. This last method is called GOP preserving without DIBR (noDIBR). The proposed methods are summarized in Table 2, by describing the time chosen to compute the reference image and the technique used to perform this operation. The operation mode of the six meth-

computation of the reference image 𝐼˜𝑚−1 𝐼ˆ𝑚−1 𝐼˜𝑚 𝐼ˆ𝑚 𝐼˜𝑘 KF of target view

Table 2: Summary of methods

0000000 1111111 1111111 0000000 0000000 1111111 0000000 1111111 0000000 1111111 0000000 1111111

(c) 𝑘

method A-DIBR A-DIBRc I-DIBR I-DIBRc GOPp noDIBR

ods for the case 𝑚 = 𝑘 + 2 is shown in Fig. 2(a). Note that not all the methods can be applied for each 𝑚, in fact A-DIBRc (resp. I-DIBRc) are applicable only if in 𝑚 − 1 (resp. 𝑚) there is a WZF for the second view. Now we compute the rate and the distortion provided by the different method for the frames concerned by the switch, namely from (𝑖) 𝑘 − 𝑁/2 to 𝑘 + 𝑁 − 1. We note with 𝑅𝑛 the rate for the frame at instant 𝑛 of the 𝑖-th view, with 𝑅ST the rate needed for storage, and with 𝑅BW the bandwidth needed for sending the requested video, and with 𝑅SW the frames that are sent to the decoder during the switch, but not necessarily visualized from the user:

𝑅ST =

𝑘+𝑁 ∑−1

(1) 𝑅𝑛 +

𝑚−1 ∑ 𝑛=𝑘−𝑁/2

(2) 𝑅𝑛

(1)

𝑛=𝑘−𝑁/2

𝑛=𝑘−𝑁/2

𝑅BW =

𝑘+𝑁 ∑−1

(1) 𝑅𝑛 +

𝑘+𝑁 ∑−1

(1)

(2) 𝑅𝑛 + 𝑅𝑘+𝑁/2 + 𝑅SW

(2)

𝑛=𝑚+1

Eq. (1) accounts for the fact that all encoded frames have to be stored (independently of the IMVS method). However, only some of them have to be sent (see Eq. (2)), namely frames from 𝑘 − 2 to 𝑚 − 1 for the first view and from 𝑚 + 1 to the end of the GOP for the second. Moreover, we always send the next KF of the first view, since it is needed to obtain 𝐼𝑘−2 , . . . , 𝐼𝑚−1 . Then, according to the chosen technique, some other frames should be sent (see 𝑅SW in Eq. (1)). In particular, we always send the encoded frame of target view at time 𝑚 (excepted for I-DIBR, where it is obtained as 𝐼˜𝑚 ). Moreover we send the parity bits for the frame obtained by DIBR in methods A-DIBRc and I-DIBRc (if the corresponding frame is a WZF), the KF 𝑘 of second view for the noDIBR method and possibly, for GOPp, frame 𝑚 − 1 of the second view 1 . As far as the distortion is concerned, we consider only frames that are visualized by the user. Therefore, the distortion is computed ∑𝑘+𝑁 −1 (2) ∑ (1) 𝐷𝑛 . as 𝐷 = 𝑚−1 𝑛=𝑚 𝑛=𝑘−𝑁/2 𝐷𝑛 + 4. EXPERIMENTAL RESULTS In our tests we use the first 100 frames of the MVD sequences Ballet and Breakdancers (views 3 and 4). The texture sequence and the depth maps are independently encoded for both cameras with the DISCOVER algorithm [7]. In order to compare the algorithms, five switchs from one view to the other are performed for each sequence. The results are presented in Table 3, where we report the Bjontegaard metric [15] of the first five methods with respect to the sixth (noDIBR). We remark that, according to the position of the switching point within the GOP (that is supposed to be equal to 4), the performance of the methods change, and some of them become equivalent, while 1 Only if it is needed in hierarchical temporal interpolation, e.g. for 𝑚 = 𝑘 + 3 and 𝑁 = 4.

method

ΔR [%]

I-DIBR/GOPp

13.40

A-DIBR/GOPp I-DIBR I-DIBRc

4.81 10.07 3.24

A-DIBR A-DIBRc I-DIBR I-DIBRc GOPp

0.54 -0.42 7.34 -2.22 -0.77

A-DIBR A-DIBRc I-DIBR I-DIBRc GOPp

-5.53 -5.82 18.29 8.93 -1.53

I-DIBR/GOPp

1.52

A-DIBR/GOPp I-DIBR I-DIBRc

-6.21 0.89 -4.42

A-DIBR A-DIBRc I-DIBR I-DIBRc GOPp

-5.76 -5.77 -3.55 -11.01 -10.98

A-DIBR A-DIBRc I-DIBR I-DIBRc GOPp

-9.26 -13.36 12.38 5.18 -10.48

ballet texture ΔPSNR [dB] 𝑚=𝑘 -0.80 𝑚=𝑘+1 -0.27 -0.59 -0.18 𝑚=𝑘+2 -0.02 0.02 -0.43 0.12 0.07 𝑚=𝑘+3 0.33 0.34 -1.04 -0.50 0.09 breakdancers 𝑚=𝑘 -0.08 𝑚=𝑘+1 0.33 -0.04 0.22 𝑚=𝑘+2 0.29 0.29 0.18 0.56 0.58 𝑚=𝑘+3 0.48 0.68 -0.59 -0.22 0.54

depth ΔR [%] ΔPSNR [dB]

method A-DIBRc I-DIBRc GOPp no DIBR

ΔR [%] 0.88 6.04 2.94 4.22

ΔPSNR [dB] -0.06 -0.44 -0.17 -0.22

13.76

-0.93

1.76 9.69 1.73

-0.13 -0.62 -0.11

Table 4: Rate-distortion performance for texture video (averaged over the two sequences) by the Bjontegaard metric [15] wrt the optimal combination.

-1.54 -2.11 4.58 -3.57 -1.67

0.08 0.13 -0.29 0.21 0.09

5. CONCLUSIONS AND FUTURE WORK

-6.67 -6.67 17.21 8.65 -3.05

0.41 0.41 -1.05 -0.52 0.18

0.48

-0.01

-5.82 -0.30 -4.38

0.36 0.03 0.26

-4.64 -5.01 -2.88 -8.04 -4.94

0.28 0.29 0.19 0.50 0.33

-8.42 -8.67 17.60 12.80 -5.18

0.50 0.55 -0.99 -0.71 0.34

Table 3: Rate-distortion performance of all techniques wrt noDIBR by Bjontegaard metric [15]

some other are not available. For example, for 𝑚 = 𝑘 the best solution is noDIBR, which was expected: when the switching point coincides with a KF, the best is to send it directly. Moreover in this case, the A-DIBR/A-DIBRc methods would not be possible, since the frame 𝑚 − 1 belongs to another GOP. For 𝑚 = 𝑘 + 1 the best solution is A-DIBR/GOPp (The two methods coincide since the frame in 𝑚 − 1 is the previous KF). For 𝑚 = 𝑘 + 2 the best solution is I-DIBRc: in this case A-DIBR is less effective because the reference frame is farther apart. For 𝑚 = 𝑘 + 3 the most effective method is A-DIBRc because it demands the smallest number of non-displayed frames. Note that, when parity bits are available, it is always better to send them since the quality of the frame 𝐽𝑛 used for creating the side information for the rest of the GOP is improved. Therefore next estimated frames are improved as well less parity bits are needed to correct them. So, I-DIBRc and A-DIBRc should be used instead of I-DIBR and A-DIBR, respectively, whenever the parity bits are available. As a conclusion, we remark that no technique is always the best. We highlight this by comparing the 6 proposed methods (see Table 4) with the optimal combination which uses the best method for each value of 𝑚, that is I-DIBR/GOPp for 𝑚 = 𝑘, A-DIBR/GOPp for 𝑚 = 𝑘 + 1, I-DIBRc for 𝑚 = 𝑘 + 2 and ADIBRc for 𝑚 = 𝑘 + 3, and averaging over all possible values of 𝑚. We see that if we want to use only one method, it is better to use A-DIBRc. If we want to gain a further almost 1% we have to use the optimal combination.

In this paper, we explore interactive multiview video video access in the context of free viewpoint television where both the texture images and the depth maps are coded. Our goal is to assure that during the switching the temporal playback is not interrupted. The video stream is coded by DVC techniques. Since the depth maps are available, DIBR can be applied in order to have an estimation of the other view. Several algorithms are proposed and compared w.r.t. a solution that does not use DIBR algorithm. The results are encouraging: the rate reduction for texture image is up to 13.36%. We have also shown that there is not an optimal method for any switching instant: the best solution is choosing a different method according to it. As future work, we want to apply extrapolation techniques for side information generation instead of interpolation techniques in order to reduce the number of frames that are sent to the decoder and are not displayed by the user, because interpolation introduces a small delay for the reconstruction at the decoder. 6. REFERENCES [1] C. Fehn, P. Kauff, O. Schreer, and R. Sch¨afer, “Interactive virtual view video for immersive TV applications,” in Proc. of Intern. Broadcast Conf, 2001, pp. 14–18. [2] M. Tanimoto, M.P. Tehrani, T. Fujii, and T. Yendo, “Free-viewpoint TV,” Signal Processing Magazine, IEEE, vol. 28, no. 1, pp. 67 –76, 2011. [3] C. Fehn, “A 3D-TV Approach Using Depth-Image-Based Rendering (DIBR),” in Proceedings of 3rd IASTED Conference on Visualization, Imaging, and Image Processing, Benalm´adena, Spain, Sept. 2003, pp. 482–487. [4] M. Flierl and B. Girod, “Multiview video compression,” Signal Processing Magazine, IEEE, vol. 24, no. 6, pp. 66–76, Nov. 2007. [5] G. Cheung, A. Ortega, and N. M. Cheung, “Interactive streaming of stored multiview video using redundant frame structures,” Image Proc., IEEE Trans., 2010. [6] B. Girod, A. Aaron, S. Rane, and D. Rebollo-Monedero, “Distributed video coding,” Proc. IEEE, vol. 93, no. 1, pp. 71–83, Jan. 2005. [7] X. Artigas, J. Ascenso, M. Dalai, S. Klomp, D. Kubasov, and M. Ouaret, “The DISCOVER codec: Architecture, techniques and evaluation, picture coding symposium,” in Coding of Audio-Visual Objects, Part 10: Advanced Video Coding, 1st Edition, 2007. [8] G. Petrazzuoli, M. Cagnazzo, and B. Pesquet-Popescu, “High order motion interpolation for side information improvement in DVC,” in Acoustics Speech and Signal Processing (ICASSP) IEEE Int. Conf. on, 2010, pp. 2342 –2345. [9] E. Kurutepe, M. R. Civanlar, and A. M. Tekalp, “Interactive transport of multiview videos for 3dtv applications,” in Packet VideoWorkshop, Hangzhou, China, 2006. [10] Y. Yang, M. Yu, G.Y. Jiang, and Z.J. Peng, “A transmission and interaction oriented free viewpoint video system,” Int. J. Circuits Syst. Signal Process, vol. 1(4), pp. 310–316, 2007. [11] M.Bertalmio, G. Sapiro, V. Caselles, and C. Ballester, “Image inpainting,” in Proc. ACM Conf. Comp. Graphics (SIGGRAPH), New Orleans, USA, 2000. [12] M. Bertalmio, A.L. Bertozzi, and G. Sapiro, “Navier-stokes, fluid dynamics, and image and video inpainting,” in Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on, 2001. [13] A. Criminisi, P. Perez, and K. Toyama, “Region filling and object removal by exemplar-based image inpainting,” Image Pro., IEEE Trans., vol. 13, no. 9, pp. 1200 –1212, 2004. [14] I. Daribo and B. Pesquet-Popescu, “Depth-aided image inpainting for novel view synthesis,” in Multimedia Signal Processing (MMSP), 2010 IEEE International Workshop on, 2010, pp. 167 –170. [15] G. Bjontegaard, “Calculation of average PSNR differences between RD-curves,” in VCEG Meeting, Austin, USA, Apr. 2001.