Direct Estimation of Dense Scene Flow and Depth ...

Direct Estimation of Dense Scene Flow and Depth from a Monocular Sequence Yosra Mathlouthi1 , Amar Mitiche1 , and Ismail Ben Ayed2 1

Institut National de la Recherche Scientifique (INRS-EMT), Montr´eal, QC, Canada 2 GE Healthcare, London, ON, Canada

Abstract. We propose a method that uses a monocular sequence for joint direct estimation of dense scene flow and relative depth, a problem that has been generally tackled in the literature with binocular or stereo image sequences. The problem is posed as the optimization of a functional containing two terms: a data term, which relates three-dimensional (3D) velocity to depth in terms of the spatiotemporal visual pattern and an L2 regularization term. Based on expressing the optical flow gradient constraint in terms of scene flow velocity and depth, our formulation is analogous to the classical Horn and Schunck optical flow estimation although it involves 3D motion and depth rather than 2D image motion. The discretized Euler-Lagrange equations yield a large scale sparse system of linear equations, which we order so that the corresponding matrix is symmetric positive definite. This implies that Gauss-Seidel iterations converge, point-wise or block-wise, and afford highly efficient means of solving the equations. Examples are given to verify the scheme and its implementation.

1

Introduction

Three-dimensional scene flow is the velocity field of the visible environmental surfaces. Computing dense scene flow from image sequences is still acknowledged a challenging problem [1], in spite of the substantial progress made in 2D optical flow computation [2,3]. Unlike optical flow, which has been intensively researched in the last 30 years, it is only recently that 3D scene flow attracted a significant research effort [1,4,5,6,7,8,9]. Scene flow estimation techniques can be categorized into direct and indirect. The indirect methods express scene flow in terms of more elementary variables. For instance, [10], [11] and [12] follow the rigidity assumption so as to describe the motion of an environmental surface by the rotation and translation parameters of a kinematic screw, or by rigid body motion essential parameters, whereas [13] uses local affine descriptions. Subsequently, the surface scene flow is recovered from the representation parameters and depth. Techniques that use such parametric descriptions can be cast in a motion segmentation framework, where the image domain is divided into segments according to the movement of real objects [10]. Another example is the representation of scene flow jointly by the optical velocity and disparity fields. In this case, scene flow is determined by G. Bebis et al. (Eds.): ISVC 2014, Part I, LNCS 8887, pp. 107–117, 2014. c Springer International Publishing Switzerland 2014 

108

Y. Mathlouthi, A. Mitiche, and I. Ben Ayed

these fields via stereokinematic constraint between optical velocity and disparity [14]. The latters can be recovered concurrently in binocular image sequences via the stereokinematic constraints [5,6,7]. The approach in [4] expresses scene flow in terms of depth and 3D motion, and uses scene flow regularization with total variation. Then, it estimates all the unknowns jointly via a 3D extension of the optical-flow method of Brox et al. [15]. The study in [16] suggests to regularize scene flow with a local rigidity constraint, which favors solutions that include mainly rigidly moving parts. The work in [1] models the 3D scene by a set of local, rigidly moving planar segments and state the problem as the joint computation of segment membership functions, 3D position, normal vector and rigid motion parameters of a plane for each segment. Direct methods reference scene flow directly and explicitly in the formulation, without any use of representational variables. As scene flow is a function of depth and optical flow [17], direct methods were generally developed for multiple viewpoint sequences, which allow constraints from multiple viewpoint optical flow and provide means to compute depth by correspondence [9,18]. The study in [9] was the first to address direct scene flow computation. It investigated the special cases of known depth and known stereoscopic correspondence, and treated the general case using a voxel tessellation of space and photometric tests of visibility. The method in [18] does not reference optical flow but morphs an initial surface approximation, obtained at the onset and independently of scene flow, to fit the image across multiple viewpoints and at two consecutive instants. Direct estimation removes the need for an analytical expression of the movements or surfaces in space. This is an advantage when such expressions are not available and practical models cannot be assumed. Articulated human motion recovery is an example where direct methods are relevant. If an analytical expression or a model of the motions in space is available, an indirect method may be advantageous. For instance, if the environmental surfaces are rigid, a parametric representation can afford both added accuracy and computational efficiency, particularly when the problem can be posed as a motion segmentation [10]. In any case, scene flow expressed directly in terms of three-dimensional pointwise velocities, without a model, is a useful general description of motion in space and the issue is whether its recovery can be efficiently formulated. This is a real issue because the significantly large number of variables involved in the analysis of multiple viewpoint sequences can quickly overwhelm a recovery method. This study investigates a method of direct joint scene flow and relative depth estimation from a monocular image sequence. This is unlike all previous methods, which use binocular or stereo sequences to recover scene flow. We propose a variational formulation with a functional of two terms: a data term which relates three-dimensional velocity to depth in terms of spatiotemporal variations and a regularization term. Based on expressing the optical flow gradient constraint in terms of scene flow and depth, our formulation is analogous to the classical Horn and Schunck optical flow estimation [19,20] although it involves 3D motion and depth rather than 2D image motion. We will develop a large, significantly sparse system of linear equations from a discretization of the Euler-Lagrange

Direct Estimation of Dense Scene Flow and Depth

109

equations corresponding to the functional. We will show that the matrix of this system is symmetric positive definite and block tridiagonal. This implies that Gauss-Seidel iterations, point-wise or block-wise, afford convergent and highly efficient means of solving the equations.

2

Formulation

Let I : (x, y, t) → I(x, y, t) be an image sequence, where (x, y) are the spatial coordinates defined on the bounded image domain Ω, and t ∈ R+ is the time coordinate. Let u and v be the coordinates function of optical flow. The Horn and Schunck optical flow gradient constraint relates the coordinate functions u and v to the spatiotemporal visual pattern: Ix u + Iy v + It = 0

(1)

where Ix , Iy , It are the image spatiotemporal partial derivatives.

Fig. 1. The viewing system is modeled by an orthonormal coordinate system and central projection through the origin

We will not explain this equation [19] because it has been detailed in numerous studies. Now, let (X, Y, Z) the coordinates of a point P in space and (x, y) its image coordinates. The reference system and imaging geometry are shown in Y Fig. 1. Time derivation of the projective relations x = f X Z and y = f Z , where f is the focal length, gives the coordinates u, v of optical velocity: u=

f U − xW dy f V − yW dx = ; and v = = dt Z dt Z

(2)

dY dZ where Z is depth (Fig. 1) and (U, V, W ) = ( dX dt , dt , dt ) is the scene flow at P. Substitution of (2) in (1) and multiplication of the left hand side by Z = 0 gives the linear constraint:

f Ix U + f Iy V − (xIx + yIy )W + It Z = 0

(3)

110

Y. Mathlouthi, A. Mitiche, and I. Ben Ayed

This constraint is a homogeneous linear equation in the variables of scene flow and depth. The homogeneity is a manifestation of the aperture problem because multiplication of motion and depth (and structure thereof) by a constant (scale) maintains the constraint integrity. One can remove this uncertainty of scale by choosing depth to be relative to the fronto-parallel plane Z = Z0 , for some positive depth Z0 . Therefore, Eq. (3) becomes: f Ix U + f Iy V − (xIx + yIy )W + It (Z − Z0 ) + It Z0 = 0

(4)

For notational simplicity, and economy, we will reuse the symbol Z to designate depth relative to the fronto-parallel plane Z = Z0 , in which case we write Eq. (4) as: (5) f Ix U + f Iy V − (xIx + yIy )W + It Z + It Z0 = 0 In this basic formulation, we will estimate scene flow and relative depth by minimizing the following L2 smoothness regularization functional:  1 (f Ix U + f Iy V − (xIx + yIy )W + It Z + It Z0 )2 dxdy E(U, V, W, Z|I) = 2 Ω   β α 2 2 2 (∇U  + ∇V  + ∇W  )dxdy + ∇Z2 dxdy + 2 Ω 2 Ω (6) where α and β are positive constants balancing the contributions of the smoothness terms.

3

Optimization

The Euler-Lagrange equations associated with functional (6) are the following coupled partial differential equations: f Ix (f Ix U + f Iy V + (−xIx − yIy )W + It Z + It Z0 ) − α∇2 U = 0 f Iy (f Ix U + f Iy V + (−xIx − yIy )W + It Z + It Z0 ) − α∇2 V = 0 (−xIx − yIy )(f Ix U + f Iy V + (−xIx − yIy )W + It Z + It Z0 ) − α∇2 W = 0 It (f Ix U + f Iy V + (−xIx − yIy )W + It Z + It Z0 ) − α∇2 Z = 0

(7)

with the Neumann boundary conditions on the boundary ∂Ω of Ω ∂U = 0, ∂n

∂V = 0, ∂n

∂W = 0, ∂n

∂Z =0 ∂n

(8)

∂ where ∂n is the differentiation operator in the direction of the normal n of ∂Ω. Let Ω be discretized via a unit-spacing grid D and the grid points indexed by the integers {1, 2, ..., N }. The pixel numbering is according to the lexicographical order, i.e., by scanning the image top-down and left-to-right. If the image is of size n × n then N = n2 . Let a = f Ix , b = f Iy , c = −(xIx + yIy ), d = It .

Direct Estimation of Dense Scene Flow and Depth

111

For all grid point indices i ∈ {1, 2, ..., N }, a discrete approximation of the EulerLagrange equations (7) is:  a2i Ui + ai bi Vi + ai ci Wi + ai di Zi + ai di Z0 − α (Uj − Ui ) = 0 j∈Ni

bi ai Ui + b2i Vi + bi ci Wi + bi di Zi + bi di Z0 − α



(Vj − Vi ) = 0

j∈Ni

ci ai Ui + ci bi Vi + c2i Wi + ci di Zi + ci di Z0 − α



(Wj − Wi ) = 0

(9)

j∈Ni

di ai Ui + di bi Vi + di ci Wi +

d2i Zi

+

d2i Z0

−α



(Zj − Zi ) = 0

j∈Ni

where (Ui , Vi , Wi , Zi ) = (U, V, W, Z)i is the scene flow vector at grid point i; ai , bi , ci , di are the values at i of a, b, c, d, respectively, and Ni is the set of indices of the neighbors of i. For the 4-neighborhood, card (Ni ) = 4 for points interior to the discrete image domain, and card (Ni ) < 4 for boundary points. The Laplacian 2 ∇ Q, Q ∈ {U, V, W, Z}, in the Euler-Lagrange equations has been discretized as 1 j∈Ni (Qj − Qi ), with α absorbing the factor 1/4. 4 Rewriting (9), and where ni = card (Ni ), we have the following system of linear equations, i ∈ {1, ..., N }:  ⎧ 2 (ai + αni )Ui + ai bi Vi + ai ci Wi + ai di Zi − α Uj = −ai di Z0 ⎪ ⎪ ⎪ ⎪ j∈Ni ⎪ ⎪ ⎪  ⎪ ⎪ 2 ⎪ b a U + (b + αn )V + b c W + b d Z − α Vj = −bi di Z0 ⎪ i i i i i i i i i i i i ⎪ ⎨ j∈Ni (S)  ⎪ ⎪ ci ai Ui + ci bi Vi + (c2i + αni )Wi + ci di Zi − α Wj = −ci di Z0 ⎪ ⎪ ⎪ ⎪ j∈Ni ⎪ ⎪  ⎪ ⎪ 2 ⎪ Zj = −d2i Z0 ⎪ ⎩ di ai Ui + di bi Vi + di ci Wi + (di + αni )Zi − α j∈Ni

Let q = (q1 , ..., q4N )t ∈ R4N be the vector with coordinates q4i−3 = Ui , q4i−2 = Vi , q4i−1 = Wi , q4i = Zi , i ∈ {1, ..., N }, and r = (r1 , ..., r4N )t ∈ R4N the vector with coordinates r4i−3 = −ai di Z0 , r4i−2 = −bi di Z0 , r4i−1 = −ci di Z0 , and r4i = −d2i Z0 , i ∈ {1, ..., N }. System (S) of linear equations can be written in matrix form as: Aq = r (10) where A is the 4N ×4N matrix with elements A4i−3,4i−3 = a2i +αni ; A4i−2,4i−2 = b2i +αni ; A4i−1,4i−1 = c2i +αni ; A4i,4i = d2i +αni ; A4i−3,4i−2 = A4i−2,4i−3 = ai bi ; A4i−3,4i−1 = A4i−1,4i−3 = ai ci ; A4i−3,4i = A4i,4i−3 = ai di ; A4i−2,4i−1= A4i−1,4i−2 = bi ci ; A4i−2,4i = A4i,4i−2 = bi di ; A4i−1,4i = A4i,4i−1 = ci di ; for all i ∈ {1, ..., N }, and A4i−3,4j−3 = A4i−2,4j−2 = A4i−1,4j−1 = A4i,4j = −α, for all i, j ∈ {1, ..., N } such that j ∈ Ni , all other elements being equal to zero. System (S) is a large scale sparse system of linear equations. Such systems are best solved by iterative methods designed for sparse matrices [21,22]. Because

112

Y. Mathlouthi, A. Mitiche, and I. Ben Ayed

A is symmetric positive definite (We omit the details on the proof here), pointwise and block-wise Gauss-Seidel and relaxation iterative methods for solving system (10) converge (We use convergent 4 × 4 block-wise Gauss-Seidel iterations). This is a standard result in numerical linear algebra and a proof can be found in numerical analysis textbooks such as [21,22]. For a 4 × 4 block division of matrix A, the Gauss-Seidel iterations consist of solving, for each i ∈ {1, ..., N }, the following 4 × 4 linear system of equations, where k is the iteration number: (a2i + αni )Uik+1 + ai bi Vik+1 + ai ci Wik+1 + ai⎛ di Zik+1 = −ai di Z0   + α⎝ Ujk+1 + j∈Ni ;ji

j∈Ni ;ji

which can be done efficiently by the singular value decomposition method [23].

4

Estimation of the Spatiotemporal Derivatives

The purpose is to estimate the spatiotemporal derivatives Ix , Iy , It from two consecutive images of a sequence. The computation of a function derivative from inaccurate data is an ill-posed problem because, as it is easy to demonstrate, small changes in the function values can result in significantly large changes in the derivative. Following the formulas in the Horn and Schunck paper on optical estimation [19], motion analysis studies have generally used forward first differences to represent derivatives, locally averaged to counter the effect of noise: 1 { I(r + Δr, c + 1, 0) − I(r + Δr, c, 0) Ix (r, c) ≈ 14 Δr=0 +I(r + Δr, c + 1, 1) − I(r + Δr, c, 1) } 1 1 { I(r + 1, c + Δc, 0) − I(r, c + Δc, 0) Iy (r, c) ≈ 4 Δc=0 It (r, c) ≈

1 4

1 Δr=0

1 Δc=0

+I(r + 1, c + Δc, 1) − I(r, c + Δc, 1) } { I(r + Δr, c + Δc, 1) − I(r + Δr, c + Δc, 0) } (11)

Direct Estimation of Dense Scene Flow and Depth

113

where I0 is the current image and I1 the next. The spatial derivatives have sometimes been estimated using averages of central differences.

5

Experimental Results

This section presents various experiments on synthetic and real sequences, which demonstrate the validity and quality of our scheme/implementation. Using the recovered scene flow and relative estimated depth, we depict for each sequence a 3D view of scene flow and an anaglyph of the estimated depth [12]. Computed from an image of the monocular sequence and its corresponding depth map, an anaglyph affords a convenient way for 3D appraisal. When viewed with chromatic (red−cyan) glasses, it gives viewers a strong sense of depth1 . In addition to 3D scene flows/anaglyphs, we further depict for each test the optical flow velocities recovered from the obtained 3D scene flow via (2), and juxtapose them to those computed by the classical Horn-and-Schunck method. For each example, we used the first two consecutive frames. All parameters were determined empirically, with the distances measured in pixels and fronto-parallel plane variable Z0 fixed equal to 6 × 104 pixels. Coefficients α and β are given in the caption of each figure. The camera focal length f is fixed equal to 600 pixels [24], and the number of Gauss-Seidel iterations to 500. The initial value of scene flow at each point is set to 0. A. First example: Fig. 2 depicts the results we obtained on Marbled-block, a synthetic sequence from the database of KOGS/IAKS Laboratory, Germany. This sequence contains three blocs, two of which are moving. Each moving block has a distinct translation, with the larger one moving in depth/left direction and the smaller one in forward/left direction. 3D interpretation of this sequence is challenged by several reasons. The blocks cast shadows that weaken apparent boundary movement, and are rendered by a macro texture along with weak spatio-temporal intensity variations within the textons. The top of the blocks have a texture profile very similar to that of the background, weakening two of the image-occluding edges. The left-positioned source of light shadows the rightmost faces of the blocks, affecting significantly the intensity variations on these faces. The first column depicts the anaglyph, and the second a 3D viewpoint of the recovered scene flow. In the third column, we display the 2D optical flow velocities recovered from the obtained 3D scene flow via (2), and juxtapose them to those computed by the classical Horn and Schunck method (last column). Observe that the 2D optical flow we obtained is quite similar to Horn-andSchunk’s, which demonstrates the validity of our scheme/implementation. B. Second example: Fig. 3 shows the results we obtained on a real image sequence containing three moving objects (courtesy of Debrunner and Ahuja, who built the sequence in [25] for the purpose of sparse 3D reconstruction). The sequence depicts (i) a cylindrical surface rotating around the vertical axis at a 1

Anaglyphs are best perceived on good-quality photographic paper. When viewed on standard screens, EPS formats (which we use here) are better perceived with full-color resolution.

114

Y. Mathlouthi, A. Mitiche, and I. Ben Ayed

Fig. 2. Marbled-block results (better perceived when enlarged on the screen). First column: Anaglyph; Second: 3D scene flow; Third: optical flow recovered from 3D scene flow via (2); Last: Horn-and-Schunck optical flow. α = 6 × 107 ; β = 102 .

velocity of one degree per frame, and moving laterally to the right at an image rate of about 0.15 pixel per frame; (ii) a box moving to the right at an image rate of about 0.30 pixel per frame and (iii) a flat background moving to the right (parallel to the box motion) at 0.15 pixel per frame. The 3D motion of this sequence makes its 3D recovery challenging. The movement of the box is quite similar to the background’s, and both movements are very small, which makes them regarded as a single-object motion [25]. The anaglyph (first column) reveals a strong sense of depth when viewed with chromatic glasses. The second column depicts a 3D viewpoint of the recovered scene flow. The last two columns illustrate the consistency between the 2D optical flow velocities computed from our 3D scene flow (third column) via (2) and those computed by the Horn-andSchunck method (last column).

Fig. 3. Cylinder results (better perceived when enlarged on the screen). First column: Anaglyph; Second: 3D scene flow; Third: optical flow recovered from 3D scene flow via (2); Last: Horn-and-Schunck optical flow. α = 6 × 106 ; β = 104 .

C. Third example: The third example uses the Berber data (Fig. 4), a real sequence which depicts a statue rotating around a nearly vertical axis and moving forward to the left (in a static environment). Unlike the previous examples, this test undergoes several complex variations in depth. The surfaces of most elements in the scene are smooth and firm in texture, with the background containing some areas of weak textures. The boundary placements are accurate in some places, and depth shows sharp discontinuities at some occlusion boundaries. The first column depicts the anaglyph; the second shows a 3D viewpoint of the recovered 3D scene flow. We also display the 2D optical flow velocities recovered from the obtained 3D scene flow (third column) via (2), and juxtapose them to those

Direct Estimation of Dense Scene Flow and Depth

115

computed by the classical Horn and Schunck method (last column). Again, the 2D optical flow velocities we obtained are quite similar to Horn-and-Schunk’s.

Fig. 4. Berber results (better perceived when enlarged on the screen). First column: Anaglyph; Second: 3D scene flow; Third: optical flow recovered from 3D scene flow via (2); Last: Horn-and-Schunck optical flow. α = 6 × 107 ; β = 5 × 104 .

D. Fourth example: The test in Fig. 5 uses the Pharaohs data, a real sequence which depicts two figurines. The one on the right side rotates about a nearly vertical axis whereas the other translates left and forward, both moving in a static environment. Similarly to the previous example, there are several variations in depth and 3D reconstruction in this example is challenging. The right-positioned source of light shadows the left-most face of a figurine, causing weak intensity variations on this face. The background is steady and contains areas of weak textures. Figurine surfaces are smooth and firm in texture. The boundary placements are accurate in some places. Fig. 5, first column, shows the anaglyph. A 3D viewpoint of the recovered 3D scene is depicted in the second column. The last two columns juxtapose the 2D optical flow velocities recovered from our 3D scene flow (third column) to those computed by Horn-and-Schunck method (last column), illustrating the similarities between both 2D velocity fields.

Fig. 5. Pharaohs results (better perceived when enlarged on the screen). First column: Anaglyph; Second: 3D scene flow; Third: optical flow recovered from 3D scene flow via (2); Last: Horn-and-Schunck optical flow. α = 6 × 107 ; β = ×102 .

E. Fifth example: The last example uses the Rock sequence, a real scene from the CMU/VASC image database. This sequence is recorded by a camera moving horizontally and to the right. Here, depth variations makes 3D interpretation more challenging than the previous examples. The rocks have rough textures, where several areas are either textureless or characterized by weak

116

Y. Mathlouthi, A. Mitiche, and I. Ben Ayed

spatio-temporal variations and fine textures. The first column of Fig. 6 shows the anaglyph, and the second illustrates a 3D view of the recovered scene flow. The 2D optical flow recovered from our scheme, as well as the one obtained with the Horn-and-Schunk method, are displayed in the third and fourth columns respectively. In this example, the optical flow recovered from our scene flow scheme is smoother and more aligned than Horn-and-Schunck’s. This can be explained by the fact that our scheme accounts for 3D information.

Fig. 6. Rock results (better perceived when enlarged on the screen). First column: Anaglyph; Second: 3D scene flow; Third: optical flow recovered from 3D scene flow via (2); Fourth: Horn-and-Schunck optical flow. α = 6 × 107 ; β = ×105 .

References 1. Vogel, C., Schindler, K., Roth, S.: Piecewise rigid scene flow. In: IEEE International Conference on Computer Vision, ICCV (2013) 2. Baker, S., Scharstein, D., Lewis, J.P., Roth, S., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. International Journal of Computer Vision 92(1), 1–31 (2011) 3. Sun, D., Roth, S., Black, M.J.: Secrets of optical flow estimation and their principles. In: CVPR, pp. 2432–2439 (2010) 4. Basha, T., Moses, Y., Kiryati, N.: Multi-view scene flow estimation: A view centered variational approach. International Journal of Computer Vision 101(1), 6–21 (2013) 5. Wedel, A., Brox, T., Vaudrey, T., Rabe, C., Franke, U., Cremers, D.: Stereoscopic scene flow computation for 3d motion understanding. International Journal of Computer Vision 95(1), 29–51 (2011) 6. Rabe, C., M¨ uller, T., Wedel, A., Franke, U.: Dense, robust, and accurate motion field estimation from stereo image sequences in real-time. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part IV. LNCS, vol. 6314, pp. 582–595. Springer, Heidelberg (2010) 7. Huguet, F., Devernay, F.: A variational method for scene flow estimation from stereo sequences. In: IEEE International Conference on Computer Vision (ICCV), pp. 1–7 (2007) 8. Pons, J.P., Keriven, R., Faugeras, O.D.: Multi-view stereo reconstruction and scene flow estimation with a global image-based matching score. International Journal of Computer Vision 72(2), 179–193 (2007) 9. Vedula, S., Baker, S., Rander, P., Collins, R., Kanade, T.: Three-dimensional scene flow. IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 475–480 (2005)

Direct Estimation of Dense Scene Flow and Depth

117

10. Mitiche, A., Ben Ayed, I.: Variational and level set methods in image segmentation. Springer (2010) 11. Mitiche, A., Sekkati, H.: Optical flow 3D segmentation and interpretation: A variational method with active curve evolution and level sets. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(11), 1818–1829 (2006) 12. Sekkati, H., Mitiche, A.: Concurrent 3D motion segmentation and 3D interpretation of temporal sequences of monocular images. IEEE Transactions on Image Processing 15(3), 641–653 (2006) 13. Zhang, Y., Kambhamettu, C.: Integrated 3d scene flow and structure recovery from multiview image sequences. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 674–681 (2000) 14. Mitiche, A., Letang, J.M.: Stereokinematic analysis of visual data in active, convergent stereoscopy. Journal of Robotics and Autonomous Systems 705, 43–71 (1998) 15. Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004) 16. Vogel, C., Schindler, K., Roth, S.: 3d scene flow estimation with a rigid motion prior. In: IEEE International Conference on Computer Vision (ICCV), pp. 1291–1298 (2011) 17. Longuet-Higgins, H., Prazdny, K.: The interpretation of a moving retinal image. Proceedings of the Royal Society of London, B 208, 385–397 (1981) 18. Pons, J.P., Keriven, R., Faugeras, O., Hermosillo, G.: Variational stereovision and 3d scene flow estimation with statistical similarity measures. In: IEEE International Conference on Computer Vision (ICCV), pp. 597–602 (2003) 19. Horn, B., Schunk, B.: Determining optical flow. Artificial Intelligence 17(17), 185–203 (1981) 20. Mitiche, A., Mansouri, A.: On convergence of the horn and schunck optical flow estimation method. IEEE Transactions on Image Processing 13(11), 1473–1490 (2004) 21. Ciarlet, P.: Introduction a l’analyse numerique matricielle et a l’optimisation, 5th edn., Masson (1994) 22. Stoer, J., Bulirsch, R.: Introduction to Numerical Analysis, 3rd edn. Texts in Applied Mathematics, vol. 12. Springer, New York (2002) 23. Forsythe, G., Malcolm, M., Moler, C.: Computer methods for mathematical computations. Prentice-Hall (1977) 24. Sekkati, H., Mitiche, A.: A variational method for the recovery of dense 3D structure from motion. Journal of Robotics and Autonomous Systems 55, 597–607 (2007) 25. Debrunner, C., Ahuja, N.: Segmentation and factorization-based motion and structure estimation for long image sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence 20, 206–211 (1998)