An Efficient Linearisation Approach for Variational Perspective Shape From Shading Daniel Maurer1, Yong Chul Ju1, Michael Breuß2, Andr´es Bruhn1 1
Institute for Visualization and Interactive Systems University of Stuttgart, Germany 2 Applied Mathematics and Computer Vision Group Brandenburg University of Technology Cottbus-Senftenberg, Germany {maurer,ju,bruhn}@vis.uni-stuttgart.de
[email protected]
Abstract. Recently, variational methods have become increasingly more popular for perspective shape from shading due to their robustness under noise and missing information. So far, however, due to the strong nonlinearity of the data term, existing numerical schemes for minimising the corresponding energy functionals were restricted to simple explicit schemes that require thousands or even millions of iterations to provide accurate results. In this paper we tackle the problem by proposing an efficient linearisation approach for the recent variational model of Ju et al. [14]. By embedding such a linearisation in a coarse-to-fine Gauß-Newton scheme, we show that we can reduce the runtime by more than three orders of magnitude without degrading the quality of results. Hence, it is not only possible to apply variational methods for perspective SfS to significantly larger image sizes. Our approach also allows a practical choice of the regularisation parameter so that noise can be suppressed efficiently at the same time.
1
Introduction
The recovery of the 3-D shape of an object from a single image given only information on the illumination direction and the surface reflectance – so called Shape from Shading (SfS) – is one of the classical tasks in computer vision. In particular in scenarios, in which huge baselines or space constraints do not allow the use of a stereo setup with two or more cameras, monocular SfS can be a highly appealing alternative to traditional stereo. Moreover SfS, in contrast to other 3-D shape reconstruction methods, does not rely on the presence of texture. Hence, it is not surprising that SfS has a wide field of applications, covering large scale reconstruction problems such as astronomy [24] and terrain reconstruction [4] as well as small scale tasks such as dentistry [1] and endoscopy [31]. Further important applications are the reconstruction of archaeological findings [9] and the visual inspection of manufactured parts [17]. Most of the classical methods for SfS have been developed in the context of astronomy and are hence based on a simple orthographic projection [11, 12]. However, in recent applications in endoscopy or macro photography, camera and light source are relatively close to the photographed scene, so that the consideration of a perspective camera [8, 18, 22] is required. The corresponding camera model not only improves the
2
Daniel Maurer, Yong Chul Ju, Michael Breuß, Andr´es Bruhn
results in such applications, it also offers a decisive theoretical advantage compared to the orthographic model: When considering a point light source at the optical centre and combining the resulting model with a physically motivated light attenuation term based on a quadratic intensity fall-off, the resulting SfS model is well-posed in the viscosity sense [23], such that concave-convex ambiguities inherent to orthographic models [29] are dissolved to some extent [5]. This makes explicit that the use of a perspective camera model is very beneficial from both a practical and a theoretical viewpoint. Taking a closer look at the underlying modelling framework, most approaches for perspective SfS are based on the solution of a hyperbolic partial differential equation (PDE) of Hamilton-Jacobi type; see e.g. [23, 27]. While such approaches allow the application of efficient numerical solvers such as fast marching schemes [15, 26, 30], they are prone to noise and missing data. In particular, they have no mechanisms to handle such cases, since they rely completely on the correctness of the input data. In this context, variational methods have proven to be very useful [13, 14]. Since such methods are based on the minimisation of an energy functional that complements a data fidelity term with a smoothness term, they do not strictly enforce the consistency with the input data, as they also regularise by adaptively averaging the information. However, when it comes to numerical schemes for the minimisation, the literature on perspective SfS is restricted so far to the application of simple explicit schemes [2, 13, 14, 31, 32]. Explicit schemes have the advantage that they are easy to code, but the computation of the minimiser as steady state of an artificial time evolution typically requires thousands or even millions of iterations to provide useful results. This slow convergence does not only pose a problem for large image sizes. It also turns out to be problematic in the presence of noise and missing data, since a larger amount of regularisation typically requires a significant decrease of the time step size. As a consequence, the number of iterations has to be increased even further which makes the application of explicit schemes inefficient if not infeasible even for small image sizes. Summarising: While variational methods for perspective SfS offer a high degree of robustness under noise and missing data, their long runtimes make them hardly applicable in practice. Our Contributions. Using the recent model of Ju et al. [14] that extends previous work of the authors [13] by a depth parametrisation along the optical axis, our paper contributes to the design of efficient solvers for variational perspective SfS in three ways: (i) On the one hand, we propose an efficient numerical scheme that embeds a linearisation approach based on a lagged upwind discretisation into a Gauß-Newton like coarse-to-fine solver. Compared with the alternating explicit scheme in [14] that already relies on a coarse-to-fine estimation with the same discretisation, this solver allows to speed up the computation by more than three orders of magnitude. (ii) On the other hand, when linearising the reflectance model in the data term, we propose to compute its derivatives numerically. As a consequence, the proposed approach can be extended in a straightforward way to more advanced reflection models such as the Oren-Nayar model for rough surfaces [19] or the Phong model for specular reflections [21]. (iii) Finally, we demonstrate that the proposed numerical scheme is highly useful in those cases where large parts of the input information is missing or when a large amount of regularisation is needed, e.g. due to noise. Also significant larger image sizes can be handled than with the alternating explicit scheme from [14].
An Efficient Linearisation Approach for Variational Perspective SfS
3
Related Work. Since the field of variational perspective SfS is rather new, there exist only a few works that address the problem of the efficient computation. On the one hand, Ju et al. [14] propose to embed an alternating simplified explicit scheme into a coarse-to-fine estimation. However, the overall convergence is still rather slow and the algorithm needs hundreds of thousands of iterations to provide accurate results. On the other hand, Abdelrahim et al. [2] propose to speed up the computation by initialising the explicit scheme with the result of a PDE-based approach. This, however, contradicts the idea of using variational methods to render the estimation more robust, since PDEbased approaches have no mechanisms to handle noise or missing information. First approaches to linearise the reflectance model go back to the early works of Pentland [20] and Tsai and Shah [28] in the context of local methods for orthographic SfS. Recently, also Barron and Malik [3] suggested to perform such a linearisation within a joint variational approach for estimating shape, illumination, and reflectance. However, also in this case, the camera model was assumed to be orthographic. The only linearisation approach for SfS in the context of a perspective camera so far was proposed by Lee et al. [16]. Their method, however, was specifically designed for a triangular element surface model and did not consider any explicit form of regularisation. Finally, none of the aforementioned approaches considered any form of upwind schemes for discretising occurring derivatives. While such schemes are not often used in computer vision, they are highly important for obtaining a stable numerical method. Organisation. In Section 2 we review the recent model of Ju et al. [14] as a representative for variational perspective SfS. In Section 3 we then show how this model can be discretised appropriately and minimised efficiently using a linearised coarse-to-fine approach. A qualitative and quantitative evaluation of the model and the minimisation framework is conducted in Section 4. Finally, Section 5 concludes with a summary.
2
Variational Perspective SfS
In this section, we review the variational method of Ju et al. [14] that serves as a prototype for the development of our efficient linearisation scheme in Section 3. To this end, we first derive the surface parametrisation as well as the underlying model assumptions and then discuss their embedding into a variational framework. Surface Parametrisation and Model Assumptions. Let us start by discussing the parametrisation of the surface. Assuming a perspective camera the unknown surface > S : Ωx → R3 can be parametrised as S (x, z(x)) = [z x/f, z y/f, −z] , where > 2 x = (x, y) ∈ Ωx is the pixel position in the image plane Ωx ⊂ R , f denotes the focal length of the camera and z(x) the depth orthogonal to the image plane. Furthermore, assuming a Lambertian reflectance model and a light attenuation term that follows the inverse square law, the resulting brightness equation reads [23]: I=
1 (N · L) , r2
(1)
where I = I(x) is the recorded image, N is the surface normal, L = L(x) is the direction of incoming light and r = r(x) denotes the distance from the light source to the surface. For a point light source located in the camera centre at the origin, this dis-
4
Daniel Maurer, Yong Chul Ju, Michael Breuß, Andr´es Bruhn
tance r as well as the direction of the incoming light L read −x −x z 1 Q −y = −y , r(x) = , L(x) = p (2) Q f |x|2 + f2 f f p with Q(x) = f/ |x|2 + f2 being the conversion factor between the radial depth r and the Cartesian depth z. Note that light rays and optical rays have opposite direction. Moreover, the surface normal can be computed by taking the normalised crossproduct of the partial derivatives Sx , Sy of the parametrised surface in x- and y-direction: fzx Sx × Sy 1 , fzy N(x, z, ∇z) = = (3) |Sx × Sy | W (∇z · x) + z q 2 2 (4) W (x, z, ∇z) = f2 |∇z(x)| + [(∇z · x) + z(x)] . Finally, plugging the surface normal (3) and the light direction (2) into the brightness equation (1), we obtain the following constraint for perspective SfS [14]: I(x) −
Q(x)3 = 0. zW (x, z, ∇z)
(5)
Variational Model. Following [14], we embed the previous constraint as quadratic data term into a variational framework and complement it with a discontinuity-preserving second order smoothness term. Please note that from a theoretical viewpoint first order data terms are not advisable for SfS, since the data term already contains first order derivatives. Consequently, we use a second order smoothness term based on the Hessian and compute the unknown depth z as minimiser of the following energy Z E (z(x)) = c(x) D(x, z(x), ∇z(x))2 + α Ψ S(Hess(z)(x))2 dx , (6) {z } | | {z } Ωx Data term Smoothness term where D of the data term and S of the smoothness term are given by Q(x)3 D(x, z, ∇z) = I(x) − , (7) z W (x, z, ∇z)) q S (Hess(z)(x)) = zxx (x)2 + 2zxy (x)2 + zyy (x)2 , (8) p respectively. Here, the penaliser Ψ (s2 ) = 2λ2 1 + s2 /λ2 is the Charbonnier function [7] with the contrast parameter λ, the weight α is the regularisation parameter that steers the amount of smoothness of the surface, and c : x ∈ Ωx ⊂ R2 → {0, 1} is a confidence function that allows to exclude unreliable image regions from the data term.
3
An Efficient Linearisation Approach
Let us now discuss how the minimiser of the previous energy functional in Eq. (6) can be computed efficiently. To this end, we first approximate the spatial derivatives in the
An Efficient Linearisation Approach for Variational Perspective SfS
5
data term using a similar scheme as the the one proposed by [28] in the context of non-variational orthographic SfS. In a second step, we then linearise the corresponding constraint in the data term and deduce a numerical scheme motivated by [6] that make use of two nested fixed point iterations and a coarse-to-fine strategy. Approximation. In order to minimise the proposed energy functional, we first introduce approximations for zx and zy using the upwind scheme from [25]. Please note that standard finite differences schemes (e.g. central differences) are not appropriate, due to the hyperbolic nature of the SfS data term. Employing grid spacings hx , hy in x- and y-direction, respectively, the approximation for zx reads zex = max D− z, −D+ z, 0 , (9) z(x + hx , y) − z(x, y) z(x, y) − z(x − hx , y) , D+ z = , (10) hx hx where, for the simplicity of our presentation, z(·, ·) is identified with the corresponding grid values. Since the forward difference D+ z enters Eq. (9) with a negative sign, one has to restore the correct sign afterwards via [5, 14] −e zx if zex = −D+ z , zx ≈ (11) zex else . D− z =
After approximating zy accordingly and replacing all derivatives in the data term with the corresponding approximations, we obtain the following expression for D that only depends on z (at the expense of including values at neighbouring locations): D(x, z, ∇z) ≈ D(x, y, z(x, y), z(x − hx , y), z(x + hx , y), z(x, y − hy ), z(x, y + hy )) . (12) Minimisation. According to the calculus of variations [10] any stationary point (and in particular the minimiser) of the approximated energy functional must fulfil the associated Euler-Lagrange equation [10]. In order to write down this equation compactly, let us introduce the following abbreviations Dxy = D x, y, z(x, y), z(x − hx , y), z(x + hx , y), z(x, y − hy ), z(x, y + hy ) , −
y
= D x − hx , y, z(x − hx , y), z(x − 2hx , y), z(x, y), z(x − hx , y − hy ), z(x − hx , y + hy ) ,
+
y
= D x + hx , y, z(x + hx , y), z(x, y), z(x + 2hx , y), z(x + hx , y − hy ), z(x + hx , y + hy ) ,
Dx
Dx
−
+
(13)
with Dxy and Dxy being defined analogously. Here, the superscripts denote the central point of the approximation. Moreover, we use the same style of notation for the abbreviations of the confidence function c and the smoothness term S. Then, the EulerLagrange equation of our approximated energy is given by h − i h + i − − + + 0 = cxy Dxy [Dxy ]z + cx y Dx y Dx y + cx y Dx y Dx y (14) h iz h iz − − − + + + + cxy Dxy Dxy + cxy Dxy Dxy z z 0 0 0 xy 2 xy 2 +α Ψ (S ) zxx xx + 2 Ψ (S ) zxy yx + Ψ (S xy )2 zyy yy ,
6
Daniel Maurer, Yong Chul Ju, Michael Breuß, Andr´es Bruhn
where [·]∗ denotes partial derivatives of the enclosed expressions. Moreover, the derivap tive of the penaliser function Ψ (s2 ) reads Ψ 0 (s2 ) = 1/ (1 + s2 /λ2 . Due to the ap∂ ∂ proximation of ∇z, the data term contributions ∂x [D]zx and ∂y [D]zy stated in [14] do not arise. Instead they are replaced by four terms considering additional points in the neighbourhood. With the purpose of obtaining a linear system of equations, we now introduce a first fixed point iteration on z, with the iteration index k, using a semi-implicit scheme in the terms related to the data term and an implicit scheme in the terms related to the smoothness term. Then z k+1 can be obtained as the solution of h − i h + i − − + + 0 = cxy Dxyk+1 Dxyk zk + cx y Dx yk+1 Dx yk k + cx y Dx yk+1 Dx yk k h iz h iz xy − xy − k+1 xy − k xy + xy + k+1 xy + k +c D D D +c D zk zk 0 xyk+1 2 k+1 0 xyk+1 2 k+1 +α Ψ (S ) zxx xx + 2 Ψ (S ) zxy yx k+1 + Ψ 0 (S xyk+1 )2 zyy . (15) yy In the first iteration, z 0 is initialised as suggested in [14]. In order to remove the nonlinearity of the terms related to the data term we furthermore linearise Dxyk+1 around Dxyk using a first order Taylor expansion: − Dxyk+1 = Dxyk + Dxyk zxyk dz xyk + Dxyk zx− yk dz x yk + − + + Dxyk zx+ yk dz x yk + Dxyk zxy− k dz xy k + Dxyk zxy+ k dz xy k . Here, dz denotes the unknown depth increment dz k = z k+1−z k and the superscripts are − used analogously to those of the previous terms. Accordingly, we linearise Dx yk+1 , + − − Dx yk+1 , Dxy k+1 and Dx yk+1 . Thus we introduce an incremental computation as in [6] and only compute the increment dz in each iteration. Please note that our data term is substantially different from linearised data terms in optical flow estimation and stereo reconstruction, since it includes neighbouring locations. To remove the last remaining nonlinearity in Ψ 0 we introduce a second fixed point iteration on dz with the iteration index l, which we initialise with dz k,0 = 0. For the argument of the Ψ 0 -function dz k,l is employed and for the other terms dz k,l+1 . After plugging the linearised expressions in Eq. (15), we finally obtain a linear system of equations with respect to dz k,l+1 . Coarse-to-Fine Scheme. As in [6, 14] we embed the first fixed point iteration in a coarse-to-fine scheme to overcome local minima and thus better approximate the global minimizer. To this end, we introduce the parameter η ∈ (0, 1) that specifies the downsampling factor between two consecutive resolution levels and the parameter κ that specifies after how many iterations k the resolution level is adopted. Computation. Finally, we compute the derivatives of D with respect to z numerically. To this end, we vary the current z estimate by ±hz and re-evaluate the D terms. This proceeding allows to compute the derivatives of D with a standard central difference scheme. In addition the contributions of the smoothness term are discretised using standard central differences. In order to solve the sparse linear system of equations in dz k,l+1 efficiently we apply the successive over-relaxation method (SOR). After
An Efficient Linearisation Approach for Variational Perspective SfS
7
Fig. 1. Synthetic images. From left to right: Sombrero, Suzanne, Stanford Bunny and Dragon.
Fig. 2. Impact of the smoothness term under increasing α using the Stanford Bunny test image. From left to right: Input image, reprojected image with α = 1, α = 20 and α = 100.
sufficient solver iterations as well as sufficient fix point iterations l we crop the computed increments dz k,lmax such that |dz k,lmax | ≤ dzlimit and then update the depth via z k+1 = z k +dz k,lmax . This avoids that erroneous increments misdirect the computation in case the linearisation provides a poor approximation in x. Let us note that controlling the size of updates is a standard procedure in many numerical algorithms.
4
Experimental Evaluation
To investigate the performance of our algorithm, we made use of the synthetic test images shown in Fig. 1 that have already been used in Ju et al. [14]. For a suitable comparison we also employ the same error measures as in their original paper: the relative surface error (RSE) which determines how well the reconstructed surface matches the ground truth and the relative image error (RIE) that indicates how well the reprojected image fits the input image. For the purpose of using a minimum number of parameters and of demonstrating that the proposed algorithm does not need a time-consuming fine tuning we choose a set of parameters which will be used throughout the following experiments, namely a downsampling factor of η = 0.9, κ = 5, lmax = 9, 10 SOR iterations, dzlimit = 0.01, hz = 10−12 and a contrast parameter of λ = 10−3 . Further, the confidence function c is set to 0 at the background pixels and otherwise set to 1. Impact of the Smoothness Term. In our first experiment we investigate the impact of the smoothness term under increasing values of the regularisation parameter α. In Fig. 2 the original input image is shown as well the reprojected images of computations with increasing α. As one can see, increasing α leads to an estimation of a smoothed surface while details are gradually eliminated. This is not only a very important property in the presence of noise, it also allows to specify the level of detail of the reconstruction.
8
Daniel Maurer, Yong Chul Ju, Michael Breuß, Andr´es Bruhn
Comparison with other Methods. In the second experiment we compare our method with the PDE-based approach of Vogel et al. [30] with Lambertian reflectance model (= baseline model of Prados et al. [23]) and the variational approach of Ju et al. [14]. In order to demonstrate the advantages and shortcomings of these approaches we consider the original input images as well as noisy versions (Gaussian noise with σ = 20). The computed error measures are listed in Tab. 1. It can be seen that for the original input images our new linearised approach yields slightly higher error values, especially in the case of the Dragon. However, except for the Dragon, the errors are below one percent. Moreover, the results for the noisy test images show the advantage of variational methods that include a regularisation mechanism contrary to PDE-based SfS approaches. While the PDE-based approach produces strongly deteriorated results due to its crucial dependence on the initialisation at singular points, both variational approaches achieve much lower surface errors. Furthermore, our novel scheme does not suffer from the problem that the regularisation parameter α has to be chosen sufficiently small to allow convergence within a reasonable number of iterations - as in the case of the alternating explicit scheme of Ju et al. [14]. This explains why our optimal parameters do not necessarily have to coincide with the optimal parameters of Ju et al. [14] that have been tuned for a fixed number of 106 iterations (to keep runtimes within a day). We repeated the noise experiment with an additional pre- and post-processing step, respectively. While for the pre-processing a variational image denoising method with TV-regularisation was used, we employed a similar method with second order smoothness term corresponding to our regulariser in (6) for the post-processing. The outcome of this experiment is shown in Tab. 2. Although the approach of Vogel et al. [30] benefits significantly from both steps (in particular from denoising the input image) our variational model still yields better results (even without the corresponding steps). This clearly demonstrates the usefulness of the built-in regularisation of variational methods. Table 1. Comparison between the PDE-based method of Vogel et al. [30], the variational method of Ju et al. [14] and our approach in terms of error measures (RSE, RIE) for the four test images without and with noise (σ = 20). The parameters of our approach are: Sombrero (α = 0.003), Stanford Bunny (α = 0.08), Dragon (α = 0.2), Suzanne (α = 0.04), Noisy Sombrero (α = 0.02), Noisy Stanford Bunny (α = 3), Noisy Dragon (α = 1), Noisy Suzanne (α = 2). For the other two approaches the same parameters have been used as in the original papers. Vogel et al. [30] RSE RIE
Ju et al. [14] RSE RIE
our method RSE RIE
Sombrero Stanford Bunny Dragon Suzanne
0.00301 0.00266 0.00422 0.00253
0.00495 0.00154 0.00255 0.00082
0.00318 0.00439 0.01376 0.00251
0.00209 0.00007 0.00028 0.00002
0.00768 0.00928 0.02904 0.00696
0.00925 0.00327 0.02333 0.00224
Noisy Sombrero Noisy Stanford B. Noisy Dragon Noisy Suzanne
0.19530 0.10973 0.12240 0.12134
0.27254 0.17347 0.19409 0.16783
0.05118 0.03235 0.05395 0.01256
0.13239 0.15279 0.18767 0.14302
0.01542 0.01359 0.03391 0.00826
0.03851 0.12285 0.17732 0.12038
An Efficient Linearisation Approach for Variational Perspective SfS
9
Table 2. Comparison of the method of Vogel et al. [30] and our approach under noise when using an additional pre-processing step (image denoising) or post-processing step (depth smoothing). Vogel et al. [30] Noisy Sombrero Noisy Stanford B. Noisy Dragon Noisy Suzanne
our method
orig. RSE
pre-p.
post-p.
orig. RSE
pre-p.
post-p.
0.19530 0.10973 0.12240 0.12134
0.02008 0.01434 0.04623 0.01245
0.19197 0.06164 0.08226 0.06169
0.01542 0.01359 0.03391 0.00826
0.01741 0.01470 0.03322 0.00917
0.01538 0.01357 0.03390 0.00824
Fig. 3. Inpainting of the Suzanne image: Deteriorated input image, reprojected image (α = 0.5).
Reconstruction with Inpainting. Our third experiment considers the inpainting capabilities of the smoothness term given via Eqs. (6) and (8). A similar experiment has been carried out in [13, 14]. However, the inpainted domains were rather small in those works. The explicit schemes used there would have needed more iterations to fill in larger regions resulting in tremendous computation time. In contrast, our new approach may inpaint larger regions without significant increase in runtime. To demonstrate this, we defined a degraded domain for inpainting considerably larger than those in [13, 14] and set the confidence function to 0 in the degraded domain. As shown in Fig. 3 our reprojected image of the reconstruction looks quite reasonable in spite of the huge amount of missing information. In case of rather flat regions no differences to the original image are noticeable, whereas missing regions with varying surface orientations, as e.g. at the ear, seem to be smoothed. However, an RSE of 0.00793 shows that the quality of the reconstruction is comparable to the RSE achieved by computing the reconstruction based on the original input image without missing regions (RSE = 0.00696). High-Resolution Image. Since variational SfS approaches mainly use simple explicit schemes, they usually require thousands or even millions of iterations to converge. Hence, the runtime becomes a critical issue. As can be noticed in Tab. 3, our approach achieves a speed up of approximately 2000 for small images compared to the explicit approach in [14]. In particular, it allows a reasonable computation time for high resolution images such as Blunderbuss Pete depicted in Fig. 4, where runtimes with explicit schemes become infeasible. Moreover, the quality is still highly competitive. As shown in Fig. 4, the reprojected image of our approach is close to the ground truth (RIE = 0.03414), whereas the difference plots reveal some difficulties at the cloak (RSE = 0.02930). In contrast, the approach of Vogel et al. [30] achieves an almost perfect reprojected image (RIE = 0.00067). With respect to the surface it shows similar errors at the cloak, while the reconstruction at the knee and torso is better (RSE = 0.01593).
10
Daniel Maurer, Yong Chul Ju, Michael Breuß, Andr´es Bruhn Table 3. Runtime comparison between Vogel et al. [30], Ju et al. [14] and our approach.
Sombrero (256 × 256) Stanford Bunny (256 × 256) Dragon (256 × 256) Suzanne (512 × 256) Blunderbuss Pete (1080 × 1920)
Vogel et al. [30]
Ju et al. [14]
our method
1s 1s 1s 2s 33s
29113s 23969s 25350s 48395s
17s 11s 12s 21s 340s
infeasible computation time
Fig. 4. From top to bottom: Reprojected image and colour-coded depth of the Blunderbuss Pete test image (3-D model by BenDasie). From left to right: Ground truth, our approach + difference plots, method of Vogel et al. [30] + difference plots.
5
Conclusion
In this paper we have introduced an efficient numerical scheme for variational perspective SfS based on a linearisation of the reflectance model. The proposed scheme not only yields speed ups of more than three orders of magnitude compared to standard explicit schemes without significantly compromising the quality of the reconstruction. It also allows to select sufficiently large values for the regularisation parameter without compromising the runtime, which enables us to deal adequately and efficiently with noise and missing information. Finally, the proposed numerical strategy is rather general such that it can be carried over to other variational models form the SfS literature that are based on standard explicit schemes so far. Acknowledgements. This work has been partly funded by the Deutsche Forschungsgemeinschaft (BR 2245/3-1, BR 4372/1-1).
An Efficient Linearisation Approach for Variational Perspective SfS
11
References 1. Abdelrahim, A.S., Abdelrahman, M.A., Abdelmunim, H., Farag, A., Miller, M.: Novel image-based 3D reconstruction of the human jaw using shape from shading and feature descriptors. In: Proc. British Machine Vision Conference, 1–11 (2011) 2. Abdelrahim, A.S., Abdelmunim, H., Graham, J., Farag, A.: Novel variational approach for the perspective shape from shading problem using calibrated images. In: Proc. IEEE International Conference on Image Processing, 2563–2566 (2013) 3. Barron, J. T., Malik, J,: Shape, illumination, and reflectance from shading. IEEE Transactions on Pattern Analysis and Machine Intelligence. To appear. 4. Bors, A.G., Hancock, E.R., Wilson, R.C.: Terrain analysis using radar shape-from-shading. IEEE Transactions on Pattern Analysis and Machine Intelligence 25, 974–992 (2003) 5. Breuß, M., Cristiani, E., Durou, J.-D., Falcone, M. , Vogel, O.: Perspective shape from shading: ambiguity analysis and numerical approximations. SIAM Journal on Imaging Sciences 5(1), 311–342 (2012) 6. Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Proc. European Conference on Computer Vision, 25–36 (2004) 7. Charbonnier, P., Blanc-F´eraud, L., Aubert, G., Barlaud, M.: Deterministic edge-preserving regularization in computed imaging. IEEE Transactions on Image Processing 6(2), 298–311 (1997) 8. Courteille, F., Crouzil, A., Durou, J.-D., Gurdjos, P.: Towards shape from shading under realistic photographic conditions. In: Proc. IEEE International Conference on Pattern Recognition, 277–280 (2004) 9. Fassold, H., Danzl, R., Schindler, K., Bischof. H.: Reconstruction of archaeological finds using shape from stereo and shape from shading. In:. Proc. Computer Vision Winter Workshop, 21–30 (2004) 10. Gelfand, I. M., Fomin, S. V.: Calculus of Variations. Dover, New York, NY (2000) 11. Horn, B.K.P.: Shape from Shading: A Method for Obtaining the Shape of a Smooth Opaque Object from One View. PhD thesis, Department of Electrical Engineering, MIT, Cambridge, Massachusetts, USA (1970) 12. Ikeuchi, K., Horn, B. K. P.: Numerical shape from shading and occluding boundaries. Artificial Intelligence 17(1–3), 141–184 (1981) 13. Ju, Y. C., Breuß, M., Bruhn, A.: Variational perspective shape from shading. In: Proc. International Conference on Scale Space and Variational Methods in Computer Vision, 538– 550 (2015). 14. Ju, Y. C., Maurer, D., Breuß, M., Bruhn, A.: Direct variational perspective shape from shading with Cartesian depth parametrisation. Perspectives in Shape Analysis, Mathematics and Visualization, Springer (2015). To appear. 15. Kimmel, R., Sethian, J.A.: Optimal algorithm for shape from shading and path planning. Journal of Mathematical Imaging and Vision 14(3), 237–244 (2001) 16. Lee, K. M., Kuo, C.-C. J.: Shape from shading with a generalized reflectance map model. Computer Vision and Image Understanding 67 (1997) 143–160 17. Nayar, S. K.: Shape recovery methods for visual inspection. Proc. IEEE Workshop on Applications in Computer Vision, 136–145 (1992) 18. Okatani, T., Deguchi, K.: Shape reconstruction from an endoscope image by shape from shading technique for a point light source at the projection center. Computer Vision and Image Understanding 66 (1997) 119–131 19. Oren, M., Nayar, S.; Generalization of the Lambertian model and implications for machine vision. International Journal of Computer Vision 14(3), 227–251 (1995)
12
Daniel Maurer, Yong Chul Ju, Michael Breuß, Andr´es Bruhn
20. Pentland, A.: Shape information from shading: a theory about human perception. In: Proc. IEEE International Conference on Computer Vision, 404–413 (1988) 21. Phong, B.T.: Illumination for computer-generated pictures. Communications of the ACM 18(6), 311–317 (1975) 22. Prados, E., Faugeras, O.: “Perspective shape from shading” and viscosity solutions. In: Proc. IEEE International Conference Computer Vision, 826–831 (2003) 23. Prados, E., Faugeras, O.: Shape from shading: a well-posed problem? In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, 870–877 (2005) 24. Rindfleisch, T.: Photometric method for lunar topography. Photogrammetric Engineering 32(2), 262–277 (1966) 25. Rouy, E., Tourin, A.: A viscosity solution approach to shape-from-shading. SIAM Journal on Numerical Analysis 29(3), 867–884 (1992) 26. Sethian, J.: Level Set Methods and Fast Marching Methods. Cambridge University Press (1999) 27. Tankus, A., Sochen, N, Yeshurun, Y.: Shape-from-shading under perspective projection. International Journal of Computer Vision 63(1), 21–43 (2005) 28. Tsai, P.-S., Shah, M.: Shape from shading using linear approximation. Image and Vision Computing 12(8), 487–498 (1994) 29. Vogel, O., Bruhn, A., Weickert, J., Didas, S.: Direct shape-from-shading with adaptive higher order regularisation. In: Proc. International Conference on Scale Space and Variational Methods in Computer Vision, 871–882 (2007) 30. Vogel, O., Breuß, M., Leichtweis, T., Weickert, J.: Fast shape from shading for Phong-type surfaces. In: Proc. International Conference on Scale Space and Variational Methods in Computer Vision, 733–744 (2009) 31. Wu, C., Narasimhan, S., Jaramaz, B.: A multi-image shape-from-shading framework for near-lighting perspective endoscopes. International Journal of Computer Vision 86, 211– 228 (2010) 32. Zhang, L., Yip, A. M., Tan, C. T.: Shape from shading based on Lax-Friedrichs fast sweeping and regularization techniques with applications to document image restoration. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, 1–8 (2007)
