Topological Changes in Scale Space as a Function of Image

Topological Changes in Scale Space as a Function of Image Transformations Comparing Dierent Stability Criteria Márta Fidrich, Jacques Feldmar, and Jean-Philippe Thirion INRIA, B.P. 93, 2004 route des Lucioles 06902 Sophia-Antipolis FRANCE Tel: (33) (0) 4 93 65 76 22

Abstract. To provide a basis for the registration of medical images we search for reliable feature points using a scale-space approach. The features are dened by dierential invariants at increasing scales and their extraction is based on iso-surface techniques. We investigate some topological changes of orbits which can be observed as a result of image transformations. We also examine whether features, stable at multiple scales, are also stable with respect to various types of transformations. Thus we can compare the usefulness of dierent stability criteria for registration. We present statistical results showing the dependency on the type and strength of the transformation and also on the scale parameters. keywords: Scale space, Registration of medical images, Dierential geometry, Singularity theory 1 Introduction

Automatic registration of medical images acquired at dierent times or from dierent patients is of growing interest. As already pointed out [11,7], we have to nd a compromise between the data representation and the correspondence search. The simpler the representation is the more sophisticated algorithm we need, and we think that a registration algorithm based on well-chosen features should give promising results. To fully automate the matching, we must search for geometric features that are invariant both with respect to various transformations and to smooth changes of resolution. Though scale-space theory [18] provides an elegant framework to study the multiscale behavior of these characteristics, there are only few examples [18,12,13] where scalespace properties are explicitly used in matching  applications are generally limited to speed up [17]. Here we try to establish a link between theory and applications by examining topological changes and dierent stability criteria. In a previous paper [5] we have presented a natural method, based on iso-surface techniques [16], to extract and represent features in linear scale space. This representation is very convenient for observing both bifurcation and delocalization eects. To follow feature points, i.e. orbits, we have proposed [4] incorporating multiscale properties into the extraction process, which makes the algorithm faster. Moreover, it is possible to follow features across scales without ambiguity while sampling scale at a practical density. Concerning 2D images we have analyzed extrema of the isophote curvature at increasing scales and have compared some criteria of their signicance. Here

we extend the analysis with detailed statistical studies so as to answer the following questions. What kind of topological changes can we expect as a result of image transformations? Are features, stable at multiple scales, also stable with respect to various types of transformations? In the next section we describe the features to be analyzed; section 3 deals with the topological changes; and nally, in section 4 we compare dierent stability criteria. 2 Features for registration

If we restrict ourselves to medical image processing, we can exploit its specic properties very fruitfully. For example, the value of intensity f is intrinsic, so the use of iso-surface techniques is well established. In many cases the iso-surface can be extracted directly, otherwise a pre-segmentation step is required to determine which part of the image is the object and which part is the background. Features dened by curvatures are particularly meaningful for registering images: corner points (absolute maxima of the 2D isophote curvature) or crest lines (their extension to 3D) are already used to match segmented images [16], i.e. when iso-value selection is reliable. Denition 1. Corner points on iso-photes satisfy the extremality criterion eiso (x; y) = rciso (x; y)
t = 0, where t = (?fy ; fx) is the tangent to the isophote and 2rciso (2x; y) = (cx ; cy ) is the gradient of the isophote curvature ciso (x; y) = 2fx fy f(xyfx2?+ffxy2f)yy3=?2 fy fxx . The intersection of the curve eiso (x; y) = 0 with an isophote f (x; y) = I gives the points of extremal curvature on that iso-phote; corner points correspond to maxima in absolute value. However, dening organs by iso-value thresholding cannot be fully automated and is quite limiting if several organs (i.e. several iso-values) should be detected at the same time. Hence, we may need to characterize object boundaries as some sort of discontinuity of the intensity, like maxima of the gradient magnitude or zero-crossings of the Laplacian. So as to apply iso-surface techniques later on, we prefer detecting boundaries as the zero-crossings of the Laplacian, where false contours are rejected by thresholding with the gradient magnitude. Note, that the Laplacian reliably detects contours only if intensity varies linearly along an edge whose direction does not change sharply, which is usually satised for medical images. Also, we modify the measurement of the curvature by omitting the normalization with the gradient magnitude, so that it is less sensitive to small intensity variations; moreover we accept curvature maxima only above a threshold. (Remark: we do not use the formula of the curvature of the iso-Laplacian, but compute the modied iso-phote curvature on the Laplacian image proting the linearity of the Laplacian.) An MRI slice of the brain is shown with Laplacian corner points in Fig. 1. Denition 2. Corner points on Laplacian-contours satisfy the extremality criterion elap (x; y) = r cmod(x; y)
t = 0, where t = (?Ly ; Lx) is the tangent to the contour and rcmod (x; y) = (cx ; cy ) is the gradient of the cornerness cmod (x; y) = 2LxLy Lxy ? L2xLyy ? L2y Lxx computed on the Laplacian image L(x; y) = fxx + fyy . The intersection of the curve elap (x; y) = 0 with the Laplacian contour L(x; y) = 0 gives the

MRI slice of the brain with points of thresholded curvature and gradient (the highest 40%), extracted on the zero-crossings of the Laplacian ( = 1:0).

Fig. 1. points of extremal curvatures. Corner points correspond to maxima in absolute value, only points with gradient fx2 + fy2 and absolute curvature jcmod(x; y)j above a weak threshold are accepted. Returning to registration, note the trade-os between search strategy and feature space or between the number of features and precision. A relatively simple matching algorithm, like ICP [3], needs well chosen feature points to be fast and precise. The robust detection of dierential singularities, like corner points, necessitates a scale-space approach [18], since it can associate signicance to features not only via thresholding but also via multiscale properties. Witkin has proposed using scale-space life-time as a measure of stability, stating that the longer a feature survives, the more stable it is [18]. However, there can be points whose orbits survive for a long time but can hardly be considered important. In fact, life-time is signicant only if features dominate others (e.g. have higher curvature in case of corner points); otherwise it is misleading, especially in discrete images. Lindeberg has found that every feature has its own scale reecting its characteristic length [10]. It can be selected as the scale where the combination of normalized derivatives forming the feature assumes a local maximum across scale in absolute value. These criteria do not take into account that the nature of features may change over scale. They should be modied; in case of corner points, orbits should be delimited by second order nongeneric singularities [5], i.e. where the sign of curvature changes. Stability of corner points could also be measured as the ratio of life-time delimited by second order nongeneric singularities and of base [5]. The base of a corner point is dened as: the distance between the two branches of the inexion

parabola at the nest scale, if its orbit is intersected by an inexion orbit; or as the distance between the two branches of its parabola at the nest scale, if its orbit merges with another corner orbit forming a parabola; else, it is one. Fig. 2 illustrates the denition of base and delimited life-time, while Fig. 3 shows various measurements for the stability of corner points.

Fig. 2. The delimited life-time Td and base B of corner points in dierent cases. (m: min., M: max.; +: pos., -: neg.)

3 Topological changes

Theoretical results about the deep structure [9] of an image, i.e. how features can change across scales, has been obtained for the continuous case [6,8,1,13]. To match features, we must also know how the deep structure changes as a function of various transformations. By construction, linear scale space is invariant to rigid transformations in the continuous case. However, we are interested in the discrete case: now we describe what happens to the orbits of corner points. Firstly, we investigate the eects of random noise. To follow the argument, observe the deep structures of (1) the original image (2) an image perturbed by noise of uniform distribution (3) an image with added noise of Gaussian distribution in Fig. 4. (The two types of noise have equal variances.) Naturally, we can see extra orbits due to noise, however, they disappear as the noise is attenuated. The connection order of singularities changes most where the contour is straightest, since here features are not stable. However, also here there are some with high scale-space life-time, since there is no dominating feature which makes them disappear. In the case of added noise (2,3), the maximum response selection is more reliable than in the noise-free case (1): The random perturbation pronounces extrema at low scales; if extrema were

Fig. 3. From top to bottom and left to right: (1) all the isophote corner points on a CT scan

cross section of the ear, signicance based on the (2) scale-space life-time, (3) maximum response selection, (4) ratio, (5) delimited life-time, (6) delimited selection

already high, this eect is not signicant, so the scale of maximum response remains unchanged. If extrema were due only to discretization errors, random noise decreases the scale of maximum response. In fact, maximum response has a kind of thresholding property contrary to life-time. Finally, we note a surprising eect of noise with Gaussian distribution (3): it seems to produce less delocalization. Now we consider the eects of image transformation. Fig. 5 shows the deep structures of (1) a rotated image (2) an ane-transformed image (3) a spline-transformed image. Contrary to the invariance to rigid transformation that holds in the continuous case, (1) shows a slight change in the deep structure where features are not stable, i.e. if curvature extrema are due only to discretization. Depending on the strength of transformation, we can expect other switches in connection order, also at curved contour-parts. Note the eect of spline transformation (3), while applying a mild ane transformation (2) does not make considerable changes  real curvature extrema are stable, even if they are rigorously invariant only to rigid transformation. 4 Statistical evaluation of stability criteria

The previous analysis highlights some eects of various transformations. However, if we need deeper investigations, a statistical study is required instead of examining particular cases  and such is the aim of this section. Our statistical process can be summarized as follows (see Fig. 6): We extract features A = fa1; : : : an g from the reference image I at logarithmically increasing scales. Then we choose the most stable features according

Fig. 4. The orbits of isophote curvature extrema (extracted from the same CT image as in

Fig. 3) are marked on the multiscale iso-contour. Black dots correspond to the scale of maximum response, white dots show this scale eventually delimited by a second order nongeneric singularity. From top to bottom: reference image, added noise with uniform distribution, added noise with Gaussian distribution (both type of noise have the same variance)

to dierent criteria [18,10,5]. Next, we modify our primal image with some random transformation Tr which is intended to simulate the dierence between patients or caused by time evolutions [14]. Finally, we extract features B = fb1; : : : bmg from the modied image M , choose scale-stable features and measure how far these features are from the selected ones of the reference image. We repeat this procedure until we have sucient data. A random rigid transformation is generated as a random rotation, with a uniform distribution of rotation angles, around the image center. A random ane transformation is composed of a random rotation plus a random shearing and scaling whose

Fig. 5. The orbits of isophote curvature extrema (black lines) and inexion points (white lines) are marked on the multiscale iso-contour. From top to bottom: image rotated by 30o , ane-transformed with added noise, spline-transformed with added noise

magnitude follows a Gaussian distribution. The strength of anity is proportional to the variance v of this Gaussian. Naturally, stronger deformations lead to less stable feature points; we have tried values 0:02  v  0:2. To generate transformed images we use warping techniques, i.e. for each voxel x 2 M we compute Tr(x) 2 I and read out the intensity at Tr(x) (that is why Tr maps in the inverse direction M ! I ). Since features are extracted independently from the original and from the sequence of modied images, their correspondence is unknown. Thus, the dierence between the set fa1 ; : : : an g 2 I and a transformed set Tr(fb1 ; : : : bm g) 2 Tr(M ) is measured by comparing two characteristic distances : the average minimal distance dA in A and the

Fig. 6. The modied image M is obtained by warping from the primal image I applying a transformation Tr : M respectively.

!

I . Features A and B are extracted from images I and M ,

average minimal distance dA;Tr(B) between A and Tr(B ). w minfdist(a ; a )g d = 1

Xi i j kAk i j6 i X wi minfdist(ai; Tr(bk))g dA;Tr B = 1 A

(

)

=

kAk

i

k6=i

Theoretically, in the continuous domain, applying rigid transformations, dA;Tr(B) should be zero. Practically, in the discrete domain, if feature detection and selection via scale is stable, we should get dA;Tr(B)  dA for each generated transformation Tr. Either we select some points by thresholding their scale  which can be the highest (delimited) scale detected, the scale of (delimited) maximum response or the ratio of delimited life-time and base  and we x weights wi to be one. Or, we consider all the detected points, and weights wi are chosen to be their scales (again, this can be the highest...). Now we recapitulate the dierent possibilities and the notations to be used in the sequel. corner points thresholded by scales of: (weights = 1) (no threshold) life-time  lt delimited life-time  dlt  mr weighted by: maximum response delimited maximum response  dmr ratio of delimited life-time and base  rlb The reason of weighting distances with scales is to penalize large dierences between features of high scale, i.e. between features which should be stable according to scalespace assumptions. In fact, we would like to nd out which kind of selection / weight is the most suitable, i.e. which one minimizes the distance dA;Tr(B) with respect to dA .

We have carried out several tests varying the type (rigid, ane) and the strength (of anity) of the transformation and also varying the scale parameters (sampling, stability criterion). Fig. 7 shows sequences of measured distances for (1) isophote corner points and for (2) Laplacian corner points. To create discrete scale space, we have used Gaussian convolution [2] with logarithmically increased variance [15]: at resolution level i = 0; : : : l, the scale is i = " i . We have found that the results do not depend on the specic choice of the smoothing parameters, if we take the intervals 0:7  "  1:3, 1:4    2:5, 5  l  10 (ratio of the image and the largest kernel size: 3  4). However, there is a considerable dierence in stability depending what criteria we employ to select features. Generally, points of relatively high scales (either dlt or dmr) are more stable as can be seen from the scale-weighted measures in Fig. 7 (c,d). We have also observed the following relations: the usefulness of rlb is similar to the usefulness of dmr, the latter is superior to mr; also, dlt is superior to lt. In case of isophote corner points, dmr (mr) is better than dlt (lt); moreover, the latter criteria can select unstable points for ane transformation even if the deformation is slight (see (1a) in Fig. 7). Considering the computation time as well, the dmr is the best choice. In case of corner points on Laplacian contours, it is dicult to decide between dlt (lt) and dmr (mr) by examining the graphs (see (2) in Fig. 7). When we tried to apply a registration algorithm, ICP [3] to match the selected points, we found that dlt selects more stable points (i.e. nds a much closer transformation to the generated one) then dmr. So, here the dlt is the best choice. This dierent outcome can be explained by the fact that Laplacian curvature maxima in absolute value were accepted only above a threshold, while for iso-photes all the absolute maxima were accepted. Thus Laplacian corner points all have similar scale of mr, as a consequence, lt can be better used to measure their stability. 5 Conclusion

We have examined several eects of various transformations on the deep structure of images. We have found that random noise improves the performance of (delimited) maximum response; we have also observed that random noise with a Gaussian distribution seems to maintain during diusion the original form of contours as compared to noise with a uniform distribution. Applying geometric transformations, we have discovered that the connection order of singularities can change depending on the strength of deformation. Even rigid transformation  contrary to the continuous case  slightly alters the deep structure of corner points, particularly where curvature extrema are not signicant. We have also compared ve criteria to measure the stability of features. The result of numerous tests is that for isophote corner points the delimited maximum response is the best, while for Laplacian corner points, thresholded by gradient and curvature, the delimited life time is the most reliable to select stable points.

8

1b

6 7

1a

5

6

5

4

4 3 3 2 2

1

1 0

10

20

30

40

50

0

10

20

x

30

40

50

x

140 100

1c

120

1d

80 100

60 80

60

40

40 20

20

0

10

20

30

40

00

50

10

20

2.4

30

40

50

x

x

3.5

2a

2.2

2

2b

3

1.8

1.6

2.5

1.4

1.2

2

1

0.8

1.5

0.6

0.4

0

10

20

30

40

50

0

10

20

30

40

50

x

x

250

2c

2d

180

160

200

140

150

120

100 100 80

60 50 0

10

20

30

40

50

0

10

20

30

40

50

x

x

Fig. 7. The graphs presents some results of the transformation experiments for (1) isophote

corner points and for (2) Laplacian corner points. The horizontal axis is the trial number and the vertical axis shows distances dA (horizontal lines) and sequences of 50 distances dA;Tr(B) . The success of a stability measure can be gained by how far below a horizontal line dA are the data points dA;Tr(B) . We have used (a) dlt above a threshold, weight=1 (b) dmr above a threshold, weight=1 (c) all the points, weight=dlt (d) all the points, weight=dmr. Parameters for (1): i = 0:7 1:5i i = 0; : : : ; 9 v = 0:05 and for (2): i = 0:7 2:4i i = 0; : : : ; 4 v = 0:1





References 1. Haruo Asada and Michael Brady. The curvature Primal Sketch. IEEE PAMI, 8, 1986. 2. Jean Babaud, Andrew P. Witkin, Michael Baudin, and Richard O. Duda. Uniqueness of the Gaussian Kernel for Scale Space Filtering. IEEE PAMI, 8, Jan 1986. 3. Jacques Feldmar and Nicholas Ayache. Rigid, Ane and Locally Ane Registration of Free-Form Surfaces. the International Journal of Computer Vision, 18:99119, 1996. Also Research Report INRIA Sophia, no 2220. 4. Márta Fidrich and Jean-Philippe Thirion. Multiscale Extraction of Features from Medical Images. In Int. Conf. on Computer Analysis of Images and Patterns, volume 970 of LNCS, pages 637642, Prague, September 1995. 5. Márta Fidrich and Jean-Philippe Thirion. Multiscale Representation and Analysis of Features from Medical Images. In Int. Conf. on Computer Vision, Virtual Reality and Robotics in Medicine, volume 905 of LNCS, pages 358364, Nice, April 1995. Also a Technical Report no 2365. 6. John M. Gauch and Stephen M. Pizer. Multiresolution Analysis of Ridges and Valleys in Grey-Scale Images. IEEE PAMI, 15, 1993. 7. Luc Van Gool, Theo Moons, Eric Pauwels, and André Oosterlinck. Semi-Dierential Invariants. In Joseph L. Mundy and Andrew Zisserman, editors, Geometric Invariance in Computer Vision. Mit-Press, 1992. 8. Peter Johansen. On the classication of Toppoints in Scale Space. Journal of Mathematical Imaging and Vision, 4:5767, 1994. 9. Jan J. Koenderink. The structure of Images. Biological Cybernetics, 50:363370, 1984. 10. Tony Lindeberg. On scale selection for dierential operators. The 8th Scandinavian Conf. on Image Analysis, pages 857866, May 1993. 11. D. Marr. Vision. MIT Press, 1982. 12. Farzin Mokhtarian and Alain K. Mackworth. Scale-Based Description and Recognition of Planar Curves and Two-Dimensional Shapes. IEEE PAMI, 8, Jan 1986. 13. Farzin Mokhtarian and Alain K. Mackworth. A Theory of Multiscale, Curvature-Based Representation for Planar Curves. IEEE PAMI, 14, Aug 1992. 14. X. Pennec and J.P. Thirion. Validation of 3-D Registration Methods based on Points and Frames. In Proceedings of the 5th Int. Conf on Comp. Vision, 1995. 15. Bart M. ter Haar Romeny, Luc M. J. Florack, Jan J. Koenderink, and Max A. Viergever. Scale Space: Its Natural Operators and Dierential Invariants. In LNCS, volume 511, pages 239255. Springer-Verlag, July 1991. 16. Jean-Philippe Thirion and Alexis Gourdon. Computing the Dierential Characteristics of Isointensity Surfaces. CVGIP, pages 190202, March 1995. Also a Tech. Report no 1881. 17. P. A. van den Elsen, J. B. A. Maintz, E. J. D. Pol, and Viergever M. A.l. Automatic registration of ct and mr brain images using correlation of geometrical features. IEEE Transactions on medical images, 14:384398, 1995. 18. Andrew P. Witkin. Scale Space Filtering. In Int. Conf. on Articial Intelligence, volume 511, 1983. Karlsruhe.