Optic Flow from Multi-scale Dynamic Anchor Point Attributes

Optic Flow from Multi-scale Dynamic Anchor Point Attributes B.J. Janssen, L.M.J. Florack, R. Duits, and B.M. ter Haar Romeny Eindhoven University of Technology, Den Dolech 2, Postbus 513 5600 MB Eindhoven, The Netherlands {B.J.Janssen, L.M.J.Florack, R.Duits, B.M.terHaarRomeny}@tue.nl

Abstract. Optic flow describes the apparent motion that is present in an image sequence. We show the feasibility of obtaining optic flow from dynamic properties of a sparse set of multi-scale anchor points. Singular points of a Gaussian scale space image are identified as feasible anchor point candidates and analytical expressions describing their dynamic properties are presented. Advantages of approaching the optic flow estimation problem using these anchor points are that (i) in these points the notorious aperture problem does not manifest itself, (ii) it combines the strengths of variational and multi-scale methods, (iii) optic flow definition becomes independent of image resolution, (iv) computations of the components of the optic flow field are decoupled and that (v) the feature set inducing the optic flow field is very sparse (typically < 12 % of the number of pixels in a frame). A dense optic flow vector field is obtained through projection into a Sobolev space defined by and consistent with the dynamic constraints in the anchor points. As opposed to classical optic flow estimation schemes the proposed method accounts for an explicit scale component of the vector field, which encodes some dynamic differential flow property.

1 Introduction Typical optic flow estimation algorithms are based on the brightness constancy assumption initially proposed by Horn & Schunck [1]. This constraint is insufficient to determine optic flow unambiguously since constant brightness occurs on surfaces of codimension one (curves in 2D, surfaces in 3D, etc.). The intrinsic ambiguity has become known as the aperture problem. The ambiguity is typically resolved by adding extra constraints to the optic flow constraint equation, or similarly, through an appropriate regularizer in a variational formulation. Horn & Schunck used a quadratic regularizer. Ever since many alternative regularisation schemes have been proposed, essentially following the same rationale. Apart from variational methods [2,3,4] that are similar to the method proposed by Horn & Schunck, correlation-based [5,6], frequency-based [7] and phase-based methods [8] have been proposed. In order to cover large displacements several coarse-to-fine strategies of these techniques have been devised [9,10]. Werkhoven et al., Florack et al. and Suinesiaputra et al. [11,12,13] developed biologically inspired optic flow estimation methods incorporating “optimal” local scale selection. The work by Florack et al. A. Campilho and M. Kamel (Eds.): ICIAR 2006, LNCS 4141, pp. 767–779, 2006. c Springer-Verlag Berlin Heidelberg 2006 

768

B.J. Janssen et al.

shows that taking notion of scale may lead to superior performance compared to the optic flow estimation algorithms evaluated by Barron et al. [14]. Recent work by Brox et al. and Bruhn et al. [2,15] show impressive results of current state of the art variational optic flow estimation techniques. In this work we combine the strength of the variational rationale with the added value of the multi-scale framework, by proposing a new paradigm for optic flow estimation. To this end we investigate flow of so called anchor points in Gaussian scale space. These anchor points could be any type of identifiable isolated points in Gaussian scale space. In these points the aperture problem is nonexistent and therefore the flow can be unambiguously measured. If these points carry “sufficiently rich” dynamic information one may hypothesise that any high resolution dense optic flow field that is consistent with the constraints posed by the anchor points and their dynamic features will be a potential representative of the “underlying” optic flow field one aims to extract. This has the additional advantage that optic flow definition becomes independent of image resolution, as opposed to “constant brightness” paradigms. Note that we completely abandon the constant brightness ansatz, as the anchor points typically have non-conserved intensity attributes. The proposed optic flow estimation method which is depicted in Figure 1 is of a truly multi-scale nature since each anchor point lives at a certain scale. This leads to a method in which automatic scale selection is manifest as opposed to the approach by Florack et al. [11] which require an extrinsic criterion for scale selection. We apply our method to a specific set of anchor points. One should however notice that any set of intrinsic points can be used. Section 2 discusses the subject of anchor points.

Fig. 1. An overview of the proposed optic flow estimation algorithm. First anchor points are selected. The second step is the flow estimation that is done on the sparse set of selected anchor points that are present in the scale space of a single frame. Finally this sparse multi-scale vector field is converted to a dense vector field at grid scale.

The sparse multi-scale vector field constructed by the motion of the anchor points has to be converted to a dense high resolution vector field at scale s = s0 , in which s0 > 0 is related to grid scale. To this extent we apply a reconstruction method similar to the method proposed by Janssen et al. [16,17,18,19]. The case of stationary reconstruction is discussed in Section 3. In Section 4 the extension of the reconstruction algorithm to vector valued images and results of its application to flow reconstruction are discussed. We stress that the emphasis in this article is not on performance (which, indeed, is not quite state-of-the-art), but on a paradigm shift for optic flow extraction with great potential. Many improvements tot the ansatz proposed here are readily conceived of, such as the extension of intrinsic anchor pointset, utilization of higher order dynamic attributes

Optic Flow from Multi-scale Dynamic Anchor Point Attributes

769

(e.g accelerations), modification of Sobolev inner product space, slick exploitation of time scale, etc. . We believe that a thorough investigation of these suggestions will significantly improve performance.

2 Anchor Points A Gaussian scale space representation u(x; s) in n spatial dimensions is obtained by convolution of a raw image f (x) with a normalized Gaussian: u(x; s) = (f ∗ ϕs ) (x) , ||x||2 1 − 4s . ϕs (x) = √ n e 4πs

(1)

We aim to identify and determine the velocity of anchor points in the scale space of an image sequence. The scale space of an image sequence is a concatenation of the scale spaces of the images of the sequence. This means time scale is not taken as an explicit dynamic parameter. The type of anchor point that is discussed in this paper is the so called singular point. Apart from or in addition to these points any type of generically isolated point that is intrinsic in the scale space of an image could have been considered. An introduction to scale space theory can be found in a tutorial book by ter Haar Romeny [20] and monographs by Lindeberg [21] and Florack [22]. 2.1 Singular Points A singular point is a non-Morse critical point of a Gaussian scale space image. Scale s is taken as a control parameter. This type of point is also referred to in the literature as a degenerate spatial critical point or as a toppoint or catastrophe. Definition 1 (Singular Point). A singular point (x; s) ∈ Rn+1 is defined by the following equations.  ∇u(x; s) = 0 , (2) det ∇∇T u(x; s) = 0 . Here ∇ denotes the spatial gradient operator. The behavior near singular points is the subject of catastrophe theory. Damon studied the applicability of established catastrophe theory in a scale space context [23]. Florack and Kuijper have given an overview of the established theory in their paper about the topological structure of scale space images for the generic case of interest, and investigated geometrical aspects of generic singularities [24]. More on catastrophe theory in general can be found in a monograph by Gilmore [25]. Singular points can be found by following critical paths, which are curves of vanishing gradient. A generic singular point manifests itself either as a creation or annihilation event involving a pair of critical points.

770

B.J. Janssen et al.

2.2 Flow When tracking anchor points over time we have one extra state variable. Spatial nonMorse critical points follow a path through scale spacetime in the scale space of an image sequence. Such a path can be described by a parameterised curve p(t) as long as it is transversal to time frames. Along this curve, of which a visualisation can be found in Figure 2, the properties of the point as described in equation (2) – given that the t-transversality critereon holds1 – do not change.

p(t)

s v x t

Fig. 2. The path p(t) that a singular point travels through time t. In the frames the zero crossings of the image gradient are depicted as a function of scale s. These lines are called critical paths.

We are interested in finding a flow vector in each singular point that is present in the scale space of an image. Definition 2 (Flow Vector). The motion of a singular point is described by a flow vector v ∈ Rn+1 , ⎡ ⎤ x˙ (3) v=⎣ ⎦ . s˙ Here x˙ ∈ Rn denotes a spatial vector describing the flow’s spatial components and s˙ ∈ R describes the scale component. The (suppressed) time component2 of this vector is implicit and taken as t˙ = 1. In other words it is assumed that time t is a valid ∂ . parameter of the flow vector’s integral curve. A dot is shorthand for ∂t We ignore boundaries of the image sequence. By definition of scale space and the fact that we are studying natural image sequences it can be assumed that the image sequence of interest is at least piecewise continuously differentiable (even analytical with respect to spatial variables). By definition the properties of the tracked point along the curve p(t) (see Figure 2),   ∇u(t; x, s) =0, (4) det H(t; x, s) 1

2

This transversality condition is implicitly required in the constant brightness based optic flow rationale. This assumption is called the “temporal gauge” and reflects the transversality assumption.

Optic Flow from Multi-scale Dynamic Anchor Point Attributes

771

2

u(t;x,s) do not change. H(t; x, s) is the n × n matrix with components ∂ ∂x where a and a ∂xb b index the spatial dimensions. If the Jacobian determinant does not degenerate,

⎡

H(t; x, s)

⎢ det ⎣

∇ det H(t; x, s) . . . T

⎤ ∇ ∂u(t;x,s) ∂s ⎥ ..  0, ⎦= .

(5)

∂ det H(t;x,s) ∂s

the implicit function theorem can be applied to obtain the flow along the parameterised curve p(t) = (x(t), s(t)) trough time. Assuming that equation (5) holds, the movement of the anchor point in a sufficiently small neighborhood of the initial position is given by inversion of ⎤⎡ ⎤ ∇ ∂u(t;x,s) x˙ ∂s ⎥⎣ ⎦ ⎢ .. = ⎦ ⎣ . s ˙ ∂ det H(t;x,s) T ∇ det H(t; x, s) . . . ∂s ⎡ ∂u(t;x,s) ⎤ ∇ ∂t ⎦. −⎣ ⎡

H(t; x, s)

(6)

∂ det H(t;x,s) ∂t

This follows straight forward by setting   d ∇u(t; x, s) =0, dt det H(t; x, s)

(7)

d denotes the total time derivative along the flow. in which dt Problems arise when the point that is followed disappears, i.e. if the t-transversality condition fails: which may happen at isolated points in the image sequence (in-between image frames). At such an event the point in scale spacetime changes to a non-Morse critical point and the implicit function theorem does not hold anymore. This can be detected by checking if the Jacobian determinant degenerates. As such this can be used to evaluate the validity of the estimated flow vector.

2.3 Error Measure for Anchor Point Localisation and Tracking Because of noise the resulting vector of equation (6) will not point to the exact position of the anchor point in the next frame. A refinement of the solution can be made by using a Taylor expansion around the estimated position of the anchor point as proposed by Florack and Kuijper [24]. Evaluation of 

   x − det H(t; x, s) H−1 (t; x, s) ∂∇u(x;s) ∂s = s det H(t; x, s)

(8)

near a singular point results in a vector pointing to the actual position of the singular point. The prime in x and s denotes differentiation with respect to a path parameter p

772

B.J. Janssen et al.

ds implicitly defined by dp = det H(t; x, s). For a detailed derivation of equation (8) we refer to the original article [24]. Notice that the so-called “cofactor matrix”

˜ x, s) def = det H(t; x, s) H−1 (t; x, s) H(t;

(9)

is well defined even in the limit where det H(t; x, s) → 0. Equation (8) gives us an error correction mechanism that can be incorporated in tracking. Note that one can march this vector field in order to obtain a higher accuracy.

3 Stationary Reconstruction In order to obtain a dense flow field from singular points endowed with their dynamic attributes a so called dense flux field is generated that is consistent with these features. In this section we will study the essence of this so-called “reconstruction” algorithm in the simplified context of stationary scalar images. Optic flow will be discussed in the next section. Stationary reconstruction from differential attributes of singular points can be connected to generalized sampling theory as proposed by Unser and Aldroubi [16]. They find a consistent reconstruction of a signal from its integer shifted filter responses, i.e. a reconstruction that is indistinguishable from its original when observed through the filters the features were extracted with. The consistency requirement is adopted by Lillholm, Nielsen and Griffin [17,18] in pursue of stationary reconstruction from scale space interest points. They describe a variational framework that finds a consistent reconstruction that minimizes a so called prior. The reconstruction problem boils down to the selection of an instance of the metameric class consisting of g ∈ L2 R2 such that (ψi , g)L2 = ci , (i = 1...N )

(10)

with ψi denoting the distinct localized filters that generate the ith filter response ci = (ψi , f )L2 . In case the prior is formed by an inner product the reconstruction scheme aims to establish a reconstruction g that satisfies equation (10) and minimizes E(g) =

1 (g, g)H . 2

(11)

Here (·, ·)H denotes an inner product on a Hilbert space H (to be specified later). Since g satisfies equation (10) we may as well write g = argmin E(g) =

1 (g, g)H − λi ((κi , g)H − ci ) . 2

(12)

Summation convention applies to upper and lower feature indices i = 1...N . The first term in the Euler-Lagrange formulation, cf. equation (12), is referred to as the prior. The remainder consists of a linear combination of constraints, recall equation (10),

Optic Flow from Multi-scale Dynamic Anchor Point Attributes

773

with Lagrange multipliers λi . κi are the filters in H that correspond to the filters ψi in L2 , i.e. by definition we have, (κi , f )H = (ψi , f )L2 .

(13)

The solution to equation (12) can be found by orthogonal projection in H of the original image f on the linear space V spanned by the filters κi , i.e. g = PV f = (κi , f )H κi .

(14)

def

Here we have defined κi = Gij κj with Gramm matrix Gij = (κi , κj )H

(15)

and Gik Gkj = δji . We adopt the prior that is formed by the first order Sobolev inner product (f, g)H = (f, g)L2 + (γ∇f, γ∇g)L2 .

(16)

with γ ∈ R+ a scalar that controls the smoothness of the reconstructed function. Because of this choice the explicit form of the filters κi is as follows, κi = (I − γ 2 Δ)−1 ψi 

1 = F −1 ω −→ F (ψ )(ω) . i 1 + γ 2 ||ω||2

(17)

For two dimensions (n = 2) the convolution filter that represents the linear operator (I − γ 2 Δ)−1 equals  x2 + y 2 1 φγ (x, y) = ] (18) K0 [ 2 2πγ γ with K0 representing the zeroth order modified Bessel function of the second kind [26].

4 Flow Reconstruction The reconstruction algorithm that is described in the previous section will be used to obtain a dense flux field that explicitly contains the desired flow information. The velocity of the anchor points (recall equation (6)) is encoded in a multi-scale flux field J. 4.1 Proposed Vector Field Retrieval Method n Definition 3 (Dense Flux Field). The dense flux field j ∈ Lm 2 (R ) at scale s = s0 , in which s0 > 0 is related to the grid scale, is defined by

j=fv

(19)

n with f ∈ L2 (Rn ) the image intensity and v ∈ Lm 2 (R ) the dense optic flow field introduced in Definition 2.

774

B.J. Janssen et al.

In order to find a dense flux field from flux measurements in scale space the minin mization with respect to k ∈ Lm 2 (R ) of the following Euler-Lagrange formalism is proposed, in which P denotes the collection of anchor points, ⎤ ⎡ m m      1 def ⎣ (ka , ka )H + E[k] = cap (ka , φp )H − Jpa ⎦ = Ea [ka ] . (20) 2 a=1 a=1 p ∈P

With k = (k1 , . . . , km ) denoting the estimated dense flux field (gray value times scale space velocity) at scale s = s0 . The filter responses for each point p ∈ P are  j φp dV . (21) Jp = Rn

These measurements of the image flux in scale space can only be performed reliably in well defined anchor points. We restrict ourselves to the anchor points introduced in Section 2, namely singular points. The reader should note that there is no theoretical objection against the use of any other point type. Equation (20) is a sum of positive convex energies and can therefore be minimized for each component of k separately. The fact that equation (20) is decoupled for each spatial and scale component readily yields k = PV j ,

(22)

with V denoting the span of the filters {φp }p∈P and PV the component-wise linear projection onto V. Recall Section 3, notably equation (14).

5 Evaluation The results of the optic flow estimation are evaluated quantitatively by means of the angular error of the estimated vector field as proposed by Barron et al.[14]. The test sequence used for evaluation is the familiar Yosemity Sequence. Qualitative evaluation is performed by analysis of the flow field of the Translating Tree Sequence and the Hamburg Taxi Sequence. These image sequences and their ground truths, if available, are obtained from the University of Western Ontario (ftp://ftp.csd.uwo.ca in the directory /pub/vision/) as mentioned in a paper by Barron et al. [14]. The algorithm is implemented in Mathematica [27]. For the properties of the different sequences we refer to the original article [14]. The anchor pointset P is obtained in two steps. Initial guesses of the positions of the singular points are obtained using ScaleSpaceViz [28] which basically uses a zerocrossings method for solving (2). The locations of the points returned by this program are refined by iteration over equation (8) until the estimated error is below 10−3 pixels. In case a singular point position cannot be refined, which can be caused by a poor initial guess of its position, the point is discarded. The result of these two steps is a highly accurate representation of P . The dense flux field is estimated for each spatial component separately as described in equation (22). The scale component is, for the sake of simplicity, ignored as it has no

Optic Flow from Multi-scale Dynamic Anchor Point Attributes

775

counterpart in the algorithms evaluated in the comparison paper by Barron et. al.[14] or elsewhere. More on this new aspect will be discussed in section 6. For each of the test sequences the velocities of the singular points are estimated by inversion of equation 6. When the velocity of a singular point is unstable the point is not taken into account in the algorithm. The velocity of a singular point is considered stable as long as its stability measure (recall equation (5)) satisfies ⎡ ⎢ det ⎣

H(t; x, s) ∇T det H(t; x, s) . . .

⎤ ∇ ∂u(t;x,s) ∂s ⎥ .. ⎦> . .

(23)

∂ det H(t;x,s) ∂s

In the experiments is set to 10−5 . This settings of is an educated guess and should be subject of further research (it is likely to depend on the condition number of the linear system). The time derivatives in equation (6) are either taken as two-point central derivatives or as Gaussian derivatives with fixed time scale τ . The Gaussian derivatives are computed with a scale component in the time direction, notably τ . The performance of the reconstruction depends on the setting of the parameter γ introduced in equation (16). Since the goal of this section is to evaluate the feasibility of the proposed algorithm only a fixed sensible setting of γ = 16 pixels, is taken into account. In theory one obtains smoother reconstructions as γ tends to infinity, all consistent with the dynamic anchor point constraints. Definition 3 is applied to resolve the desired dense optic flow field. 5.1 Qualitative Evaluation A visualization of the angular component of the estimated flow vectors belonging to the Translating Tree Sequence with time scale set to τ = 3 frames is presented in Figure 3. In this case problems occur at the boundaries. However only those edges that possess appearing and disappearing singular points show problems. The results could be influenced negatively by the low number of features (95 singular points with an image size of 150 × 150 pixels) that were used in the algorithm. It is remarkable that the translating structure of the flow field is correctly detected by our algorithm without explicitly accounting for this structure. The optic flow field of the Hamburg Taxi Sequence superimposed on the image on which the optic flow estimation is performed is shown in Figure 4. It shows all vectors that possess a magnitude larger than 0.2 pixels/frame. Most notable is the fact that the pedestrian, present in the upper left of the image sequence, is detected by our algorithm. Also some false vectors are present in the scene. This could be caused by numerical errors that occur when high order derivatives are taken at fine scales. A more restricting setting of will remove these vectors but coarsens the set of anchor points too much. Boundary problems are visible at the location where the van enters the sequence. 5.2 Quantitative Evaluation A more objective approach to the evaluation of the quality of the estimated flow field is the average angular error [14]. The results of the experiments conducted on the Yosemity

776

B.J. Janssen et al.

Fig. 3. The estimated flow field of Translating Tree Sequence using time scale τ = 3 frames Table 1. Summary of the performance of the proposed optic flow retrieval method applied to the Yosemity Sequence Time Average Standard Deviation # Singular Scale Angular Error of Angular Error Points 1 24.93◦ 36.00◦ 352 2 19.19◦ 34.45◦ 303 3 19.97◦ 34.30◦ 271 4 22.94◦ 42.61◦ 251 5 25.00◦ 39.50◦ 226

Sequence are summarised in Table 1. It shows the angular error, standard deviation of the angular error and the number of singular points used in the reconstruction step for different time scales τ . When the time scale is increased less singular points can be taken into account in the reconstruction step of the algorithm. The setting of the time scale is a tradeoff between the number of features that can be taken into account (which decreases with time scale) and noise robustness (which increases with time scale). A histogram of the angular errors of the estimated optic flow in the Yosemity Sequence using a time scale of τ = 2 frames is shown in Figure 5. The figure shows that

Optic Flow from Multi-scale Dynamic Anchor Point Attributes

777

# vectors

4000

3000

2000

1000

50

Fig. 4. The estimated flow field of Hamburg Taxi Sequence using time scale τ = 2 frames superimposed on the image on which the calculations were applied. Only vectors that denote a velocity larger than 0.2 pixels/frame are displayed.

100

150

Angular Error degrees

Fig. 5. Histogram of the angular error of the estimated flow field belonging to the Yosemity Sequence

the number of erroneous vectors decreases rapidly with the magnitude of the angular error, which is a promising result. Similar results are obtained for different time scales. Still many vectors that point to the opposite direction are present. This increases the standard deviation of the angular error that is presented in Table 1. Since we did not develop a confidence measure it is unjustified to leave these incorrect vectors out.

6 Summary and Recommendations We have proposed an inherent multi scale optic flow estimation method that finds a flow field using the dynamic properties of so called anchor points. These anchor points can by any type of point as long as it is isolated and geometrically identifiable. We selected singular points of a Gaussian scale space image for the evaluation of our method. General reconstruction theory can be applied to optic flow retrieval from anchor points by encoding the dynamic properties of these points in a so called flux field. In order to obtain a dense flux field from the sparse set of flux vectors at multi scale anchor points, which is related to the optic flow of an image sequence, general reconstruction theory can be applied to each dimension independently. The evaluation of zeroth order flow estimation, for which up to zeroth order dynamic features are taken into account, shows reasonable results considering the sparseness of features. Problems that showed up are mainly caused by too sparse a set of features. Other causes of error are instability of the anchor point flow estimation, which is in general caused by occlusion and the image boundary, and numerical instability in the inversion step that is present in the reconstruction algorithm. One of the unique properties of the proposed algorithm is that the velocity vectors possess a scale component. This can potentially be useful for segmentation and camera

778

B.J. Janssen et al.

motion estimation. We neglected the presence of this scale component in the reconstruction process. It does give more information about the local structure of the vector field and can therefore be used as an extra constraint for the estimation of the spatial components of the optic flow field. A “zoom”, for example, will cause singular points to translate and move downwards in scale. A method to add this extra property in the estimation of the optic flow field has yet to be proposed. 6.1 Recommendations We suggest the following improvements to the algorithm which may lead to a significant performance increase: – Taking into account higher order features (accelerations etc.). – Improved reconstruction, e.g. by choosing a more appropriate space for projection. – Using more anchor points. Possible anchor point candidates are singular points of the laplacian of an image, so called blob points as proposed by Lindenberg [29], scale space saddles as proposed by Koenderink [30] or other singular or critical points. One can use all these point types simultaneously.

Acknowledgement The Netherlands Organisation for Scientific Research (NWO) is gratefully acknowledged for financial support.

References 1. Horn, B.K.P., Schunck, B.G.: Determining optical flow. Artificial Intelligence 17 (1981) 185–203 2. Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Proceedings of the European Conference on Computer Vision. Volume 3024 of Lecture Notes in Computer Science., Springer-Verlag (2004) 25–36 3. Nagel, H.H.: Displacement vectors derived from 2nd order intensity variations in image sequences. Computer Vision, Graphics and Image Processing 21 (1983) 85–117 4. Uras, S., Girosi, F., Verri, A., Torre, V.: A computational approach to motion perception. Biological Cybernetics 60 (1988) 79–87 5. Anandan, P.: A computational framework and an algorithm for the measurement of visual motion. International Journal of Computer Vision 2 (1989) 283–310 6. Singh, A.: Optic Flow Computation: a Unified Perspective. IEEE Computer Society Press, Los Alamitos, CA (1991) 7. Heeger, D.: Optical flow using spatio-temporal filters. International Journal of Computer Vision 1 (1988) 279–302 8. Fleet, D.J., Jepson, A.D.: Computation of component image velocity from local phase information. International Journal of Computer Vision 5 (1990) 77–104 9. Terzopoulos, D.: Image analysis using multigrid relaxation methods. IEEE Transactions on Pattern Analysis and Machine Intelligence 8 (1986) 129–139

Optic Flow from Multi-scale Dynamic Anchor Point Attributes

779

10. Wu, Y.T., Kanade, T., Li, C.C., Cohn, J.: Image registration using wavelet-based motion model. International Journal of Computer Vision 38 (2000) 129 – 152 11. Florack, L.M.J., Niessen, W.J., Nielsen, M.: The intrinsic structure of optic flow incorporating measurement duality. International Journal of Computer Vision 27 (1998) 263–286 12. Suinesiaputra, A., Florack, L.M.J., Westenberg, J.J.M., ter Haar Romeny, B.M., Reiber, J.H.C., Lelieveldt, B.P.F.: (Optic flow computation from cardiac MR tagging using a multiscale differential method—a comparative study with velocity-encoded mri) 483–490 13. Werkhoven, P., Toet, A., Koenderink, J.J.: Displacement estimates through adaptive affinities. IEEE Trans. Pattern Anal. Mach. Intell. 12 (1990) 658–663 14. Barron, J.L., Fleet, D.J., Beauchemin, S.S.: Performance of optical flow techniques. International Journal of Computer Vision 12 (1994) 43–77 15. Bruhn, A., Weickert, J., Feddern, C., Kohlberger, T., Schn¨orr, C.: Variational optical flow computation in real time. IEEE Transactions on Image Progressing 14 (2005) 608–615 16. Unser, M., Aldroubi, A.: A general sampling theory for nonideal acquisition devices. IEEE Transactions on Signal Processing 42 (1994) 2915–2925 17. Lillholm, M., Nielsen, M., Griffin, L.D.: Feature-based image analysis. International Journal of Computer Vision 52 (2003) 73–95 18. Nielsen, M., Lillholm, M.: What do features tell about images? In Kerckhove, M., ed.: Scale-Space and Morphology in Computer Vision: Proceedings of the Third International Conference, Scale-Space 2001, Vancouver, Canada. Volume 2106 of Lecture Notes in Computer Science. Springer-Verlag, Berlin (2001) 39–50 19. Janssen, B., Kanters, F., Duits, R., Florack, L.: A linear image reconstruction framework based on Sobolev type inner products. In: 5th International Conference on Scale Space and PDE Methods in Computer Vision. (2005) 20. Haar Romeny, B.M.t.: Front-End Vision and multiscale image analysis. Kluwer Academic Publishers (2002) 21. Lindeberg, T.: Scale-Space Theory in Computer Vision. The Kluwer International Series in Engineering and Computer Science. Kluwer Academic Publishers, Dordrecht, The Netherlands (1994) 22. Florack, L.M.J.: Image Structure. Volume 10 of Computational Imaging and Vision Series. Kluwer Academic Publishers, Dordrecht, The Netherlands (1997) 23. Damon, J.: Local Morse theory for solutions to the heat equation and Gaussian blurring. Journal of Differential Equations 115 (1995) 368–401 24. Florack, L., Kuijper, A.: The topological structure of scale-space images. Journal of Mathematical Imaging and Vision 12 (2000) 65–79 25. Gilmore, R.: Catastrophe Theory for Scientists and Engineers. Dover Publications, Inc., New York (1993) Originally published by John Wiley & Sons, New York, 1981. 26. Abramowitz, M., Stegun, I.A., eds.: Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables. Dover Publications, Inc., New York (1965) Originally published by the National Bureau of Standards in 1964. 27. Wolfram, S.: Mathematica: A System for doing Mathematics by Computer. Second edn. Addison-Wesley (1991) 28. Kanters, F.: Scalespaceviz. http://www.bmi2.bmt.tue.nl/image-analysis/people/FKanters/ Software/ScaleSpaceViz.html (2004) 29. Lindeberg, T.: Feature detection with automatic scale selection. International Journal of Computer Vision 30 (1998) 79–116 30. Koenderink, J.J.: A hitherto unnoticed singularity of scale-space. IEEE Transactions on Pattern Analysis and Machine Intelligence 11 (1989) 1222–1224