Linear Registration paper - CiteSeerX

The Registration of MR Images using Multiscale Robust Methods M.E. Alexander and R.L. Somorjai

National Research Council Canada, Institute for Biodiagnostics, Winnipeg, CANADA Published in: Magnetic Resonance Imaging, Vol. 14, No. 5, pp.453-468 (1996) Mailing address of corresponding author: National Research Council Canada, Institute for Biodiagnostics, 435 Ellice Avenue, Winnipeg, Manitoba, CANADA R3B 1Y6

Telephone: (204) 984-6995 Fax: (204) 984-5472 email: [email protected]

November 1995

ABSTRACT Acquisition of MR images involves their registration against some pre-chosen reference image. Motion artifacts and misregistration can seriously flaw their interpretation and analysis. This paper provides a global registration method that is robust in the presence of noise and local distortions between pairs of images. It uses a two-stage approach, comprising an optional Fourier phase-matching method to carry out pre-registration, followed by an iterative procedure. The iterative stage uses a prescribed set of registration points, defined on the reference image, at which a robust nonlinear regression is computed from the squared residuals at these points. The method can readily accommodate general linear, or even nonlinear, registration transformations on the images. The algorithm was tested by recovering the registration transformation parameters when a 256 by 256 pixel T2*-weighted human brain image was scaled, rotated and translated by prescribed amounts, and to which different amounts of Gaussian noise had been added. The results show sub-pixel accuracy of recovery when no noise is present, and graceful degradation of accuracy as noise is added. When 40% noise is added to images undergoing small shifts, the recovery errors are less than 3 pixels. The same tests applied to the Woods’ algorithm gave slightly inferior accuracy for these images, but failed to converge to the correct parameters in some cases of large-shifted images with 10% added noise. Keywords: Image registration; Robust regression; Multiscale methods; MR image analysis.

2

1. Introduction Image matching is a fundamental problem in image analysis, and is used in a wide variety of applications. It is becoming a basic clinical tool in non-invasive medicine: in identifying and accurately targeting tumours during radiotherapy treatment planning, for follow-up studies of the growth or movement of tissues and organs, and correction of subject movement over the course of multiple acquisitions of data. Another important application is the integration of images taken using different modalities (e.g., MRI and CT scans). A still commonly used method for alignment of MRI slices is visual inspection and manipulation of the image slices, which is both error-prone and timeconsuming. The methods in this paper, though applicable to multimodal data, will focus on the unimodal application to MRI, where a sequence of images may represent either a time course or a spatial ordering of parallel 2D sections through the body. Typically, one needs to register a set of several hundred images with respect to some reference image, (which may be either one of the set or one derived from the set) by matching global anatomical features between the reference image and each of the images in the set. This will correct spatial distortions and misalignments caused by linear shifts (including uniform scaling differences, rotations and translations) between the images. The residue differences after this has been done will comprise both uncorrected (nonlinear) distortions and features of anatomical interest. Distinguishing between these types of differences then forms the task of a subsequent nonlinear registration algorithm. However, by first achieving an accurate global linear alignment, the remaining distortions are usually local in nature, and the amount of computational work required is thereby significantly reduced. As the speed of acquisition of images improves, the need for the rapid availability of automatically registered images for subsequent analysis by computer or clinician becomes more pressing. This paper describes a method for registering 2D images by means of translation, rotation and uniform scaling transformations. Its generalization to 3D registration (where the data may have required the registration of a set of 2D image slices anyway), or to nonuniform (including shearing) and even global nonlinear distortions, can be achieved in an obvious way. There are three broad categories of algorithms for registration1: (i) those based directly on image pixel intensities, such as the correlation2-4, Fourier5-9 and moment10-12 methods; (ii) the feature-matching methods, in which significant features must first be extracted then used for matching13-16; and (iii) elastic model-based methods17-19, in which image distortions are modelled as deformations of an elastic material. The first two categories comprise global registration methods, whereas the third is useful for correcting local distortions between images. The two chief disadvantages of the first category of methods is their sensitivity to changes between the images (such as is due to variations in illumination caused by changes in sensor characteristics, or to image acquisition by different modalities), and the limitation in the types of geometric transformations that can be applied to register the images (generally, rigid translations, rotations and scaling along

3

each coordinate axis). The feature-matching methods (ii) require that significant features in the images first be identified. The types of features sought are generally ones which are immune to changes in external conditions and convey significant information about the object being imaged; these include, for example, edges, corners, and points or lines of extremal curvature of the intensity surface. The third category of methods often assume that the images have been pre-aligned by some global method, in order to remove largescale differences between the images. The method in this paper belongs in the category of feature matching algorithms. The features chosen for matching were edges. The edges are extracted as individual points, without any assumption about their interconnectivity to form contours, and are thus treated as point-based features. This allows the more general types of features mentioned above to be easily incorporated as well, since the feature extraction and registration tasks are independent of each other. The method was devised to address two important issues in registration. First, the need for fast, numerically efficient registration of a set of images, in order to provide a tool for a clinical environment. Apart from pre-registration, the execution time of the registration algorithm scales linearly with the number of registration points used, which is under usercontrol. Furthermore, the registration parameters (presently, the scaling, rotation and xand y-translations) found at any stage can be used as a starting configuration for subsequent iterative refinements of the registration - for example, by re-defining the set of registration points, based on extracting edge points from the reference image at a finer spatial scale of resolution. Second, a global method should be able to adequately match global features without being affected by local distortions. While areas in the image affected by local distortions could in principle be avoided when choosing the registration points, these areas are not always easily identified and at best would require timeconsuming operator intervention to do so. Thus, the method should be robust: mismatches of some minority of the points (“outliers”) should not affect an otherwise good match found by the remaining majority of points. Somewhat related to this second issue is the effect of noise on registration. Noise can be significant at the feature extraction stage, and unless it is properly removed could result in a corrupted choice of registration points. Nevertheless, for the same reason as before, it is desirable that the registration method be robust to the presence of some minority of corrupted registration points. The method proposed requires the optimization of a distance function (calculated between the image to be registered and some pre-defined reference image), and is iterative in nature. It is advantageous, and sometimes (for large image shifts) essential, to start iterations as close as possible to the global minimum of this distance function which corresponds to the correct alignment of the image pair. The distance function has many local minima and saddle points in the space of parameters of the registration transform, and the algorithm may converge to a spurious local minimum, giving a false match. One way to overcome this problem is first to reduce the number of features in the images (thus increasing the distance between the local extrema of the distance function) by blurring

4

them at some scale of spatial resolution. Having obtained an approximate alignment, the registration parameters found may then be used as starting conditions for aligning a less blurred (or unblurred) version of the same pair of images. Furthermore, by blurring the images, they may be reduced in size without losing information, by resampling on a coarser lattice (this is what is done, for example, by the decimation step in the wavelet transform algorithm20, when filtering with a low-pass, high-pass quadrature filter pair), allowing faster computation. This hierarchical approach is the basis of many problems in computer vision21, in particular registration22,23,24. An alternative “pre-registration” method is to use a global method, such as the method of moments12 or the Fourier-Mellin transform based phase matching method9. The moment method, while computationally very efficient, was found to be unsatisfactory, and to give erroneous results when features are partially occluded in one or both images. As well, it is susceptible to noise. However, the much more “robust” phase-matching method was capable, in most cases, of providing adequate starting conditions for the iterative algorithm, even when using reduced sizes and coarse discretizations of the original images in order to save computation time. The present method combines this technique as an optional pre-registration stage, with subsequent application(s) of the iterative stage at chosen spatial scales of resolution of the images to be registered. This flexibility is desirable; for example, human subjects for MR imaging experiments often exhibit movements between acquisition of images sufficiently small to render pre-registration unnecessary. It should be emphasized that the pre-registration method used here is limited to rigid body transformations (uniform scaling, rotation, and translation), whereas the iterative algorithm can perform much more general transformations: for example, general linear (affine), as well as (global) nonlinear transformations, are possible, by including the appropriate type and number of registration parameters. An algorithm related to the present one is due to Woods et al.25. The method uses a different objective function for minimization. A subset of pixels is chosen from each of a pair of images to be registered, and the ratio of their values is formed. For misaligned images, this ratio will vary across the subset of pixels chosen, and the registration procedure adjusts the registration parameters until the “normalized standard deviation” (= standard deviation/mean, sometimes called the “coefficient of variation” in statistics) of these ratios of intensities is minimized. The outline of the paper is as follows. Section 2 describes the pre-registration algorithm9, and the preprocessing required on the images for the extraction of edge features and selection of registration points from them. There follows a discussion of the iterative stage of the registration method itself: its formulation as a robust nonlinear regression problem and the optimization methods used to solve it. Section 3 discusses the specific implementation of the method, in particular the convergence of the optimization methods and a hierarchical approach to successive refinement of the registration parameters. In Section 4, the performance of the method is tested on various MR images to which a known transformation (rotation, translation, scaling) has been applied. The effects of noise on the accuracy of registration is also assessed. For comparison, the same tests are

5

applied to Woods’ algorithm25. Finally, in Section 5, the generalization of the method to 3D registration and to more general global linear, as well as nonlinear, transformations is discussed. 2. Description of linear registration method The method proposed in this paper consists of two consecutive stages: a pre-registration stage and an iterative refinement stage. Pre-registration, which is optional, achieves approximate global alignment by “rigid body” (uniform scaling, rotation and translation) transformations of an image. Iterative refinement uses this approximate alignment to finetune the registration, but is not restricted to rigid transformations. Pre-registration is required because of the large number of local minima present in the distance function that can produce false matches. The Fourier phase based matching method9 provides an effective means of achieving this. Suppose that image f(x,y) is a scaled, rotated and translated version of a reference image g(x,y); we wish to apply the appropriate inverse scaling, rotation and translation transform to f to match it to g. The effects of translation can be removed by looking at the Fourier amplitudes |F(u,v)| and |G(u,v)|. Next, the cartesian frequency coordinates are mapped to log-polar co-ordinates (log(ρ),α), where u = ρ cos(α), v = ρ sin(α). The number of points used to perform this (discrete) mapping is under user-control (see below). Data windowing is applied to remove artifacts arising from the log-singularity at the origin. A second Fourier transform, this time in the log-polar plane, is computed and the Fourier phase at each pixel is extracted. These phase differences will be proportional to log(σ) and α, where σ is the scaling factor and α is the angle of rotation. By taking the inverse Fourier transform of the image formed from the phase-differences between f and g, the values of (log(σ),α) can be read off from the location of the sharp peak in the resulting image. Once σ, α are known, the original image f(x,y) may be scaled and rotated to give f’(x,y), leaving only the translation difference between this image and reference image g. By applying the same technique, this time considering the Fourier phase differences between F’(u,v) and G(u,v), the translation shift (t1,t2) may likewise be determined. The precision with which (σ,α, t1,t2) may be computed is limited by the grid size used for discretizing the (log(ρ),α)-plane, as well as the number of pixels in the original image. At best, one can achieve a one-pixel accuracy matching, if full-size images are used. The iterative refinement stage is capable of sub-pixel precision and requires starting conditions only in some vicinity of the correct position (to ensure convergence to the correct minimum of the distance function). Therefore, downsizing the images by a factor of 2 or 4, and perhaps even more coarsely discretizing the (log(ρ),α)-plane, was found to be quite adequate and resulted in significant savings in computation time (which scales as n log(n) for an n by n pixel image) compared to using the full-size images for pre-registration. The essential idea underlying the iterative stage of the method is that matching is carried out only at a prescribed number of positions in the (reference) image, defined by the registration points chosen. The distance measure to be minimized with respect to the registration parameters is some (robust) statistic based on the difference between the

6

reference image g(x,y) and a geometric transformation (using current values of the registration parameters) of the source image f(x,y) to be matched. Let ri(θ) denote the residual evaluated at the i'th registration point, when the registration parameters (defining the transformation) have the values θ = (θ1, θ2,..., θp), where p = no. of parameters: ri(θ) = f(Xi(θ),Yi(θ)) - g(xi,yi)

(1)

where (xi,yi) are the coordinates of the i'th registration point in the reference image g, and (Xi(θ),Yi(θ)) its coordinates after the transformation has been applied. For the particular case of rigid transformations (uniform scaling θ1 = σ, rotation angle θ2 = α, and translation θ3 = t1, θ4 = t2) considered here, Xi(θ) = σ[ xi cos(α) + yi sin(α)] + t1; Yi(θ) = σ[-xi sin(α) + yi cos(α)] + t2.

(2)

Robust regression26-28 minimizes a statistic based on {r1(θ),r2(θ),..,rN(θ)} (N = no. of registration points used), and has the property that the existence of "outliers" in the data does not affect the regression fit based on some majority of registration points for which the image differences are minimized. In the present context, outliers are represented by spurious values of the image intensities at some of the registration points, caused by one or more of: local distortions in the vicinity of the registration points; features not present in both images; or noise artifacts - all of which give rise to spuriously high values in the residual ri(θ) computed at those registration points. Thus, it is desirable to choose a statistic with the highest possible "breakdown point"29, defined as the maximum fraction of contamination of the sampled data set by outliers that can be tolerated without corrupting the regression estimate. For the classical Least Squares regression, this breakdown occurs at 1/n for a sample of size n, so this method breaks down in the presence of even one outlier and is therefore not suitable. Since ri(θ) is nonlinear in the registration parameters θ, any robust regression method chosen will be nonlinear. A subclass of estimators is obtained from minimizing the distance function D(θ) given by D(θ) = Σi wi [r(i)(θ)]2

(3)

where wi are fixed weights (Σi wi = 1) and [r(1)(θ)]2, [r(2)(θ)]2,..., [r(N)(θ)]2 is a sorted list of the squared residuals: ([r(1)(θ)]2 < [r(2)(θ)]2 N/2. Both LMS and LTS are robust estimators with breakdown points near 50%26, though LMS is computationally less expensive since it requires only selecting the median value, whereas LTS requires sorting the list of ri’s.

7

It should be emphasized that robustness is no guarantee against false matches occurring. A robust (or even non-robust) method becomes useful only when one is sure that the transformation parameters θ are in some vicinity of the global minimum as calculated for the majority of points (i.e., excluding outliers). Also, it frequently happens that some of the features in the image to be registered are lost off the edge of the image due to large displacements of the subject; in such cases not all the registration points may be available for matching the images. Provided a fraction of the registration points smaller than the breakdown point of the estimator suffer this fate, the robust method will still converge to the same set of θ-values as if all the registration points were used. The LMS robust estimator was chosen for the present registration method, and implemented according to Stromberg’s algorithm28 for multi-step approximation to the LMS. At each step of this algorithm, the current best estimate θ* of θ is updated whenever that step yields a smaller LMS value. The four stages in Stromberg’s algorithm are: (i) Compute the Least Squares (LS) estimate θLS for the entire N-point data set. (ii) From the N registration points, randomly choose p points and find an exact fit θLS* to these points. The random selections are made k times and the current best estimate θ* is updated whenever that particular p-point fit gives a smaller median squares residual for the entire data set. This describes the Monte Carlo subsampling technique, in which k is chosen large enough that, with high probablity P, at least one of the subsamples will contain no outliers. If a fraction ε of the data is contaminated by outliers, then26 P = 1 - [1 - (1- ε)p]k.

(4)

For example, with p = 4 and ε = 0.3, k = 10 gives P = 93.58% while k = 20 improves this to P = 99.59%. [By choosing k = C(N,p) (C is the binomial symbol), we are guaranteed that at least one of the p-point subsamples is free of outliers; however this number is much too large to be computationally feasible, so the Monte Carlo method is used]. (iii) Compute the LS fit to those data points i for which [ri(θ)]2 < median {[rj(θ)]2, 1 < j < N}. If the median resulting from this fit is smaller than that found in Step (ii), then update the current best estimate θ*. (iv) Minimize median {[rj(θ)]2, 1 < j < N} and update θ*. Modifications of the Nelder-Mead simplex-based optimizer30 was used in Step (iv), and of the Davidon-Fletcher-Powell31 quasi-Newton method for the nonlinear Least Squares computations in Steps (i)-(iii). The latter requires the computation of derivatives, with respect to the parameters θ, of the ri’s appearing in the distance function D(θ): ∂ri(θ)/∂θm = (∂f(X,Y)/∂X).(∂X(θ)/∂θm) + (∂f(X,Y)/∂Y).(∂Y(θ)/∂θm),

8

(5)

for 1 < i< N and 1 < m < p, where X,Y are evaluated at x = xi, y = yi, as in Equations (2). The spatial gradient (∂f(x,y)/∂x, ∂f(x,y)/∂y) is computed using a Sobel operator on image f and stored for later use in the iterations. The code for both the Nelder-Mead and DavidonFletcher-Powell methods was obtained from Numerical Recipes32, with some modifications to the constants used in those codes. The choice of reference image g(x,y) for the registration of a set of images {f1,f2,...,fM} (of which f above is a member) can be made in several ways, and in our case was chosen as follows. From the set {f1,f2,...,fM} derive the median image fmed by computing the median of the pixel values, taken one from each of the M images, at a given pixel location, and repeating this at each pixel location. The image g chosen for reference is defined as that image fm (1 < m < M) for which

Σ(x,y) [fm(x,y) - fmed(x,y)]2 is the least, where the sum is over all pixels (x,y) in the images. This approach was taken as an attempt to minimize the maximum registration shift over the whole set of M images. A key first step in registration is the correct preprocessing of images in order to extract relevant features from which the registration points are derived. The registration points should be located near “significant” features in the image, with a view to making the distance function sensitive to variations in the registration parameters θ. In the current implementation, a subset of edge features is used to define these points (see below); small excursions in θ will then cause large changes in f(x,y) if the point (x,y) is located near a boundary between two regions of different intensity. Typically, the edges define boundaries of anatomical objects. However, there are a number of effects which can cause erroneous interpretation. First, if the intensity levels, as recorded by the sensor, of an object and its background are not markedly different, then the location of an edge becomes ill-defined and very sensitive to the presence of noise. Another related effect is that, in low contrast images, spurious edges may appear in background regions of the image due to quantization. Two methods were adopted to counter these effects: histogram modification and multiscale edge detection. Histogram modification, if properly done, can both improve the visibility of true physical features in the image and ensure that abrupt intensity transitions do, in fact, correspond to true object boundaries, thus leading to a more accurate association of the detected edges with these boundaries. For the present, a form of histogram modification, namely histogram equalization, has sometimes been found to be adequate for enhancing low-contrast images which otherwise proved difficult or impossible to register. However, it runs the risk of enhancing background noise, leading to erroneous edge detection and placement of the registration points, so should be used with care. Multiscale detection of edges is a technique for reducing the effects of noise in the image, and is based on the observation33 that "true" features persist across a wide range of spatial scales, whereas noise is generally present at all scales. Various techniques exist34-36 for discriminating noise from true signal by its different behaviour at different spatial scales.

9

However, these were not explicitly incorporated into the present method, which uses edge points located at different scales only to provide successive refinements of the registration parameters and not to identify "persistent" structures as representing true anatomical features. In most cases of interest, blurring at a single scale was found sufficient to both partially eliminate noise and provide accurate registration. Registration points (defined on the reference image g) can be chosen in several ways either manually or as a preprocessing computational step. Furthermore, registration may be carried out on either the original images, or on pre-filtered versions of them. If the registration points are located on or near boundaries of objects in the image, then the distance function will respond sensitively to variations in the registration parameters in the vicinity of a match. Therefore, a gradient filter should be applied, to highlight these boundaries and concentrate the registration points there. In the present method, the Mallat-Zhong quadratic spline wavelet33 was used as a multiscale edge detector, in which the scale is determined by the number of levels (j) of wavelet decomposition. j can be specified by the user. If Wj1f and Wj2f denote the x- and y-component wavelet transforms at level j (spatial scale 2j), then the edge points are defined as the local maxima of the modulus of the wavelet transform [(Wj1f)2 + (Wj2f)2]1/2, in which the maxima are sought along the direction of the “gradient” vector (Wj1f, Wj2f) through the pixel in question. Having thus obtained a set of edge points, each one having a value of the maximum modulus of the wavelet transform at that point (indicating its “edge intensity”), this list is sorted in order of increasing intensity. The edge points corresponding to the nB largest intensities are then uniformly subsampled to obtain N distinct registration points (N < nB), where nB and N are chosen by the user. Computation times for the iterative part of the registration algorithm scale linearly with N. It is necessary to experiment to find a suitable value for N; for 256 by 256 images N = 1000 was generally found to provide adequate coverage of the image with registration points, and no significant improvement in accuracy was achieved by choosing larger N. 3. Description of program for registering a sequence of images A C program has been written for registering an arbitrary number of images of some specified dimensions n1 by n2 pixels (n1 and n2 are powers of two), and produces as output both a set of n1 by n2 registered images and a corresponding table of registration parameters. For the MR applications considered, the experimental field of view is always square, so that whenever the acquired images have different dimensions (n1 ≠ n2) it is assumed this is due to different data acquisition rates along the x- and y-axes, and the images are geometrically scaled by resizing to n by n pixels for the purposes of registration. The choice of n is limited only to powers of two, and at present is chosen as n = max(n1,n2). For the present, only p = 4 registration parameters are being considered: θ1 = σ (uniform scaling factor), θ2 = α (rotation), θ3 = t1, θ4 = t2 (translation components), though as mentioned in Section 1, the method admits of much more general transformations. In order to improve the convergence of the optimization routines, the parameters were normalized: θ2 ∈ [0,2π], t1,t2 ∈ [0,1] when used internally by the registration program, but converted to degrees (θ2) and pixels (θ3,4) for viewing as output.

10

All geometric mappings - such as occur when resizing the images, or when pixels are mapped under the registration transformation - in general cause pixel lattice locations to be mapped onto non-lattice sites, so that interpolation is required. In all cases, bilinear interpolation, involving the pixel values at the 4 vertices of the enclosing lattice rectangle, was used. The iterative algorithm requires evaluation of f, ∂f/∂x, and ∂f/∂y at (in general) non-lattice sites (Xi(θ),Yi(θ)) (Equations (2)), and this was done using bilinear interpolation on the pixel lattice values for f, ∂f/∂x, and ∂f/∂y, respectively (the latter two having been derived, as explained before, by applying the Sobel operator to f). The Sobel operator approximates the gradient with uniform O(h2) error (where h is the pixel lattice spacing), whereas differentiating the bilinear interpolation representation of f itself would lead to discontinuities in the gradient across lattice rectangles. The gradient-based optimization scheme (using the Davidon-Fletcher-Powell method) was found to converge more quickly than a simplex-based (modified Nelder-Mead) scheme which does not use the gradient. The tradeoff, of course, is the extra storage needed for ∂f/∂x, ∂f/∂y (which, however, is re-used as each new image fm (1 < m < M) is brought in for registering against the reference image g). Pre-registration is necessary whenever the misalignment between image pairs exceeds a certain amount determined by the size and spacing of features in the image (which in turn determine the spacing between local minima in the distance function D(θ)), in the vicinity of the registration points: anything larger runs the risk of the algorithm converging to a false match. In general, one would expect larger misalignment tolerances for blurred versions of the images than for unblurred images, but this is somewhat offset by the fact that the minima of the distance function are shallower, and therefore more difficult to localize, in the former case. For human brain, T2*-weighted 128 by 256 unblurred images (Figures 1 and 2), pre-registration was necessary whenever the expected shifts were more than 20-30 pixels, rotation angles greater than about 5°, or scaling factor outside the range [0.9,1.1]. The presence of noise affects the generation of the registration points themselves. Spurious edge elements arising from noise may place registration points in regions of the image devoid of physical features, and noise artifacts in the vicinity of such points could corrupt the regression estimate for the parameters, leading to incorrect matchings or poor convergence of the iterations. However, by initially blurring the reference image (on which the registration points are defined), the signal to noise ratio is improved and the proportion of misplaced registration points considerably reduced. For example, when Gaussian (white) noise is added with standard deviation of 40% of the peak amplitude of the original image in Figure 1, the signal-to-noise ratio of the unblurred images is 0.24 (see Figure 4). This improves to 3.1, 12.5 and 47.3 for blurring at levels 1, 2 and 3, respectively. There is the option of specifiying the registration points by other means than the “built-in” Mallat-Zhong wavelet edge detector - for example, manually defining regions of interest in the reference image so as to exclude obvious artifacts.

11

The registration points are stored in a file as integer pairs, specifying their x,y pixel coordinates. Also stored in this file are the current best estimates for the registration parameters for each of the images f1, f2,..,fM - arising from either the pre-registration or the iterative stage of the registration algorithm. This allows iterative refinement of the registration, by using the output of the previous iteration as starting conditions for the new iteration. In a separate file, the images to be registered are stored as an index list; in this way, the user is able to choose which images to register by specifying them in the list. For pre-registration, the starting values of the registration parameters for those images which do not appear in the list are set to default values corresponding to zero shift (σ = 1, α = 0, t1 = t2 = 0). Similarly, there is a list of images which the user may specify as candidates for determining the reference image g (Section 2). As a special case, if a particular image fR (1 < R < M) is desired as reference, then the list should contain the single entry R. The input images may first be blurred at some scale js before registration is carried out. The output registration parameters may then be used as starting conditions for refining them by registration at some smaller scale (say, level js-1) of blurring. The blurred images are obtained as the low-pass output of the Mallat-Zhong wavelet transform routine. This hierarchical approach was found to be particularly useful whenever large image misalignments and/or significant noise were present. In most such cases, it was found that a single level of blurring significantly improved the accuracy compared to unblurred images, and that subsequent refinement ceased to be effective beyond a certain level whenever substantial noise was present in the original images. 4. Results In order to test the performance of the algorithm, a T2*-weighted 256 by 256 MR image of the human brain (Figure 1) was artificially scaled, rotated and translated, and Gaussian noise was added to each of the resulting images. Some of the transformations were deliberately chosen so that part of the brain was occluded by the image border, in order to test the performance of the registration algorithm when not all registration points could be used - see Figure 2. The resulting registration parameters (σ, α, t1, t2) for each transformed image were then compared with the prescribed values, in order to determine their accuracy of recovery. The tests were applied in two cases: (a) when (σ, α, t1, t2) was sufficiently different from the identity transformation (1,0,0,0) that pre-registration was necessary [see Table 1]; and (b) when (σ, α, t1, t2) was close enough to the identity that only the iterative stage of the registration algorithm was needed [see Table 4]. For case (b), the starting values for the registration parameters corresponded to the identity transformation. Note that in Tables 1 and 4, the values given for the transformation parameters (σ, α, t1, t2) refer to the transformation that must be applied to image f in Equation (1), in order to match it to the reference image g. For example, in Table 1, σ > 1 for Images 3 and 4, so the corresponding f images in Figure 2 appear smaller than the reference image (Image 1 in Figure 2).

12

The set of registration points found by the Mallat-Zhong wavelet-based edge detector is shown in Figure 3. In order to extract the registration points the reference image was first blurred by a 2-level Mallat-Zhong wavelet transform, then the local maxima of the modulus of the resulting wavelet coefficients were detected, as described in Section 2. It was found that N = 1000 registration points provided adequate coverage of the 256 by 256 images. For testing the algorithm on the noisy images, 10 copies of each image were made and noise was added independently to each copy. The algorithm then registered these images against the (noisy) reference image (Image 1). The Median and Median Absolute Deviation (MAD) of the residuals between the parameters recovered from the registration and their correct values (given in Tables 1 or 4) were calculated for this sample size of 10, and are given in Tables 2, 3, 5 and 6. [The MAD is defined as the median of the set of absolute differences between the residuals and the median of these residuals]. The results of case (a) (pre-registration only) are given in Table 2. The pre-registration algorithm is able to register images from which a substantial part of the object is missing - such as Image 6 (Figure 2) - as was known9. The phase-matching algorithm assumes that the images are phase-wrapped, which is clearly not the case when an object in the image is partially occluded at the image boundary. In spite of this, the algorithm was able to correctly pre-register the image to within the discretization error of the grid (defined using the 128 by 128-pixel downsized image, with M=128, K=64). However, this algorithm also showed high sensitivity to noise. Therefore, for the noise-corrupted case, instead of registering the original (noisy) images, a jr-level Mallat-Zhong wavelet transform (jr = 2 in Table 2) was first applied to each of the pair of images to be preregistered, and either the low-pass blurred image or the images formed from the modulus of the wavelet coefficients (corresponding to a blurred edge detector), were registered. In all cases, it was found that the low-pass images did not register as accurately as the blurred-edge images, and that the more blurring that was applied (in an attempt to reduce the effects of noise) the less accurate were the results. For the image considered (Figure 1), Gaussian noise with standard deviation of up to about 10% of peak amplitude could be tolerated, before the algorithm broke down. It thus appears that the Fourier phasematching technique employed in pre-registration relies on high-frequency components in the Fourier transform to achieve accurate registration, and that these components are readily corrupted by noise. The iterative stage of the registration algorithm was subsequently applied, using the starting conditions provided by the above pre-registration steps for both the 0% and 10% Gaussian noise-corrupted image sets, and the results are given in Table 3. Both a singlelevel and a multi-level approach was used. In the latter, the pair of images to be registered were first blurred by applying js levels of the Mallat-Zhong wavelet transform, then the low-pass images extracted. The resulting registration parameters formed the starting conditions for registering the (js-1)-level blurred images. This procedure was cascaded until the original, unblurred images were registered. A value js = 2 was chosen. For the

13

case where no noise was added, the results for the single-level and multi-level approaches were comparable in accuracy. The convergence properties of the iterative stage of the registration algorithm were tested on images from Table 1 without employing pre-registration (i.e., using the identity transformation (σ, α, t1, t2) = (1,0,0,0) as starting conditions for each image). These starting conditions lay outside the domain of convergence of the algorithm for Images 4 and 6, but allowed accurate registration for Images 2,3 and 5. The results of case (b) (iterative algorithm, without pre-registration) are given in Table 5, and the 40% noise-corrupted image which was registered is shown in Figure 4a. As before, a multi-level approach was used, starting with js = 2. It is immediately apparent that the iterative algorithm (which uses Least Median of Squares regression) displays a much greater robustness to noise than does the Fourier phase-matching pre-registration method. As a further confirmation of the efficacy of the Least Median of Squares regression for noisy images, non-robust Least Squares minimization (essentially, just the first stage of Stromberg’s algorithm28 - see Section 2) was attempted for registering the 40% noise-corrupted image. It was found that, regardless of whether single- or multi-level iterations were used, the Least Median Squares approach gave an order of magnitude improvement over the Least Squares approach for accuracy of recovery of σ, and slightly more accurate recovery of the remaining parameters. The iterative algorithm was also tested on images from Table 4 without added noise and without pre-registration, to determine whether successively refining the registration parameters by repeated application of the iterative algorithm at the same level of preblurring (js) would improve the accuracy of the results. When repetitively applying the algorithm at js =0, this was indeed the case, though no substantial improvement occurred after the second pass. The convergence is initially faster when repeating at js = 2, and an accurate match was found after only one iteration. Therefore, it seems that blurring of images prior to registration can be useful in the presence of noise, and also in the absence of noise when incorporated in a multi-level approach. It was found that, for sufficiently large occlusion at the image boundary, the phasematching algorithm would still secure a reasonably close approximation to the correct registration parameters, whereas, even with these good starting conditions, the iterative algorithm converged to a false match. On the other hand, the iterative algorithm appears to be much more robust to noise, especially when the images are blurred prior to registration. The performance of the present method was also compared with that of the algorithm of Woods et al.25, which recently has become a popular method for registration of images. The registration points were chosen to be identical for the two methods. For the large misalignments given in Table 1, it was found that, without noise present, Woods’ algorithm converged successfully, whether or not pre-registration was used, in all cases except Image 6. This may indicate that the method is sensitive to occlusion effects at the

14

image boundary (see Figure 2). When 10% Gaussian noise was added (see Table 3), the algorithm failed to converge correctly for Images 4 and 6 when pre-registration was used, and failed on all images when no pre-registration was used. By comparison, the iterative algorithm was still able to converge in this latter case for all except Images 4 and 6. For small misalignments (Table 4), successful convergence was obtained in all cases tested, and the results are presented in Table 6. It can be seen that the accuracy of recovery using Woods’ algorithm is comparable to that of the present algorithm in the 10% noise case, but slightly inferior for the 40% noise case. Pre-registration and iterative algorithm timings were measured on the SGI Challenge Series (Silicon Graphics, Inc., Mountain View, CA) for the images registered above. It should be noted that, for fixed image sizes and discretization of the log-polar plane, the pre-registration algorithm timing is fixed, and - on account of the use of the FFT - scales as O(n log(n)) with image size n. For the cases studied (128 by 128 pixel downsized image, and M = 128, K = 64), the execution time was 1.50 seconds per image pair. There was a small (O(n)) overhead due to invoking the Mallat-Zhong wavelet transform when the images were blurred prior to registration. For the iterative algorithm, the timings depend on the number of registration points (N) used, and scale as O(N log(N)) on account of the sorting that needs to be done at Step (iii) of Stromberg’s algorithm (see Section 2). However, of more critical importance, for the image sizes considered, is the number of iterations (and hence function calls) that are needed to register the image pair (each of size 256 by 256 pixels), and this depends sensitively on the structure of the images to be registered, the tolerances specified for each of the optimizers, and on how close to a true match the starting configuration is. It is the latter that determines the overall performance of the algorithm. For the images registered in Table 4 (starting configuration = (1,0,0,0)), the average execution time (iterative algorithm only) was 4.3 seconds per image, for each level (js) of blurring. Again, the Mallat-Zhong wavelet transform, used for blurring the images, incurred only a small overhead: it took 2.3 seconds to generate 1000 registration points at level js = 2 of blurring. There was a small (8%) increase in timings between registering the low-noise and high-noise images of Table 5. The timing estimates for the Woods’ algorithm are comparable. Finally, it was found in general that the convergence of the iterative stage of the algorithm was sensitive to the value of σ used for starting condition. This may have been due to the near circular symmetry of the images being registered, as a slight mismatch of σ would result in a large number of α-values being able to provide several shallow and almost equal minima for the distance function D(θ) (Section 2).

5. Discussion Least Median of Squares robust regression, combined with a hierarchical multiscale approach to registration, has been shown to provide a robust method for registering images. The presence of noise causes a gradual degradation of accuracy in recovering the prescribed transformation parameters. The present algorithm employs local optimization

15

techniques, and so requires starting configurations close enough to the desired correct matching configuration that local minima of the distance function do not intervene to give a false match. In order to provide a suitable starting configuration, a Fourier-phase based matching technique9 was used for approximate pre-registration. This is required whenever there are large mismatches between the images. The accuracy of this pre-registration can be controlled by downsizing the images and by choosing the number of points of discretization in the log-polar plane of the Fourier-Mellin Invariant of the images being matched. In this way also, computational costs can be reduced. However, this method shows low tolerance to noise, and so one of the shortcomings of the proposed algorithm is that noisy images which require registration transformations differing substantially from the identity transformation, cannot be registered. In such cases, image enhancement and noise removal preprocessing are required, so that high-frequency components in the image are retained. Adaptive smoothing37-39,35 is a set of methods for smoothing signals while preserving genuine discontinuities in the underlying noise-free signal. These methods are promising and would certainly extend the range of disparities allowed between images to be registered. The present version of the program retains the same image size at all scales of blurring whenever hierarchical registration is performed. An obvious improvement (in memory requirements and speed) would be to downsample the images, by 2j along each axis, for j levels of blurring, and redefine the registration points accordingly. This strategy would become important when large amounts of data, such as in 3D image registration, need to be retained in memory at any given time. The present paper considered only rigid body transformations, involving 4 registration parameters. The general linear group of transformations, which can incorporate global shearing distortions, contains 6 independent parameters (a,b,c,d,e,f): Xi(θ) = a xi + b yi + c; (6)

Yi(θ) = d xi + e yi + f;

where θ = (θ1, θ2,..., θ6) and θk = θk(a,b,...,f), k = 1,..,6, is a nonsingular transformation. In a first approximation we expect that most of the disparities between the images would be removed by the rigid body transformations considered in this paper, so that the residual (linear) corrections would be small. Note that these corrections would apply across the entire image, and that localized distortions - which are the most important and interesting in medical images - cannot be described by these global techniques (see below). An obvious generalization of the linear case is to consider transformations involving polynomials up to a specified order1,41 which can also be implemented by a straightforward generalization of the algorithm of this paper. These higher-order corrections would again be small in general, so that one would expect rapid convergence to a correct match provided the rigid body pre-registration procedure described in this

16

paper is able to locate the starting configuration for the iterative algorithm sufficiently close to the true matching position. Convergence is greatly assisted by employing the hierarchical multiscale approach described in this paper: blurred images contain fewer features, and there is a lower bound on the distance between these features, which is determined by the level of blurring (essentially, it scales as O(2j) pixels, for a j-level wavelet transform when it is used as a low-pass filter). The detection of nonlinear, localized distortions between images is one of the most interesting applications of registration. In a real sense, the techniques discussed so far can be considered pre-processing stages which allow one to then focus attention on the localized differences. Common practical examples of local effects include: pulsatile flow in blood vessels during MR brain imaging, changes in size and shape of the heart during a heartbeat cycle, and changes in the size and shape of tumours when the time interval between image acquisition is several days or weeks. A related problem is the appearance or disappearance of local features between images, for which no matching is possible. The question then becomes: How robust is the registration method to these localized mismatches? At the least, one would want the errors caused by these mismatches not to propagate through the entire image domain. A possible approach to this problem is to use localized basis functions to represent the distortion field U(x,y), defined as the transformation required, at each point (x,y) of the image, to minimize some measure of the distance functional D[U] = f(x+U(x)) - g(x)

(7)

(where x = (x,y), and f and g have the same meanings as before). For this reason wavelet basis functions have been considered40. The method is iterative and inherently multiscale, and bears some resemblance to the iterative algorithm in this paper, though many more parameters (in this case, the wavelet coefficients) are involved. Experiments suggest that considerable savings in computation occur (through more rapid convergence of the iterations) if the images are carefully pre-registered using the global methods described in this paper. The methods of this paper can be generalized to 3D image registration. The generalization of the Fourier phase-based pre-registration method to recover uniform scaling, the 3 Eulerian angles and 3 components of translation, is an open problem9. However, the iterative stage of registration can easily be extended to 3D: there will be 7 parameters for the rigid body group of transformations (including uniform scaling), whereas the general linear transformation involves 12 parameters (effectively, the 3 Euler angles, 3 translation components, the 3 components compression/dilation along the principal axes, and 3 Euler angles describing the orientation of the principal axes). However, the number of parameters increases sharply when nonlinear transformations are included. Convergence of the method becomes correspondingly more delicate, so a hierarchical multiscale approach, possibly combined with some form of regularization41, would appear mandatory.

17

Experience with the methods of this paper has highlighted the importance of preprocessing of the images. This was found necessary, for example, to alleviate the effects of noise on the Fourier phase-based pre-registration method. Pre-filtering the images to highlight different types of features is also feasible; for example, “umbilical lines”, which are the loci of points of zero Gaussian curvature on the image intensity surface, points of local maximum or minimum Gaussian curvature, and local curvature of isophotes, are all potentially interesting features which can be highlighted using differential invariant filters42, and registration points derived for these preprocessed images in the same way as before.

18

Tables Table 1. Input images for testing pre-registration algorithm. Images 2-6 are scaled, rotated, and translated versions of Image 1 - see Figure 2. All transformations are with respect to (0,0) (i.e., corner of image) as origin. Image

Scale (σ)

Rotation (α) (°)

Translation (t1)

Translation (t2)

1 2 3 4 5 6

1.0 1.0 1.2 1.5 0.8 0.6

0.0 20.0 10.0 10.0 10.0 10.0

0.0 10.0 0.0 20.0 40.0 90.0

0.0 0.0 10.0 20.0 40.0 90.0

19

Table 2. Errors of pre-registration of the image in Figure 1, with 0% and 10% Gaussian noise added to the original images. For the case where no noise was added, the original (unblurred) images were registered. For the noise-corrupted case, the level-2 blurred edge images (obtained as the moduli of the wavelet coefficients of a level-2 Mallat-Zhong wavelet decomposition) were registered, as these gave better accuracy than registering with the original images. The results for the noise-corrupted cases are the medians and Median Absolute Deviations (MAD) calculated over 10 independently noise-corrupted copies of each image. The resized image used for pre-registration was 128 by 128, and the log-polar discrete mapping used a grid of size M = 128, K = 64. Image ∆σ 2 3 4 5 6

No noise added ∆α (°) ∆t1

0.0 0.0101 0.0213 -0.0046 0.0090

-0.3125 1.125 1.125 1.125 1.125

0.0 -1.6527 1.0344 5.2579 2.5412

∆t2 -4.0 1.5691 -4.2242 -4.7994 -10.594

Image ∆σ

10% Gaussian noise MAD MAD ∆α (°)

∆t1

MAD

∆t2

MAD

2 3 4 5 6

0.0000 0.0000 0.1099 0.0000 0.0763

0.000 0.000 -20.643 5.258 -78.376

2.000 3.306 63.223 2.514 53.901

-4.000 0.109 -14.801 -7.314 -13.123

2.000 1.460 27.020 2.514 21.229

0.0000 0.0101 -0.3997 -0.0046 0.3626

-0.313 1.125 -35.469 -0.156 42.031

0.000 1.406 28.125 1.406 12.656

20

Table 3. Errors of registration of the image in Figure 1, with 0% and 10% Gaussian noise present, using the iterative stage of the registration algorithm with starting conditions as given by the pre-registration found in Table 2. For the noise-corrupted case, the images were blurred to level js = 2 before registering. However, the starting conditions given by the pre-registration algorithm (Table 2) for Image 6 were not close enough for the iterative algorithm to converge to the correct minimum.

Image 2 3 4 5 6

∆σ 0.0001 0.0021 0.0055 -0.0003 0.0027

No noise added ∆α (°) ∆t1 ∆t2 -0.0249 0.2851 -0.1707 0.0306 -0.0594 -0.1398 0.1875 -0.4193 -0.0500 -0.0467 0.2108 -0.3008 0.3150 -2.8479 -0.9603

Image ∆σ

MAD

10% Gaussian noise MAD ∆α (°) ∆t1

MAD

∆t2

MAD

2 3 4 5 6

0.0011 0.0018 0.1165 0.0012 0.2162

-0.0266 0.0019 0.0996 -0.1490 0.5980

0.3800 0.2608 7.9823 0.3865 164.092

0.0435 -0.0274 -0.3777 -0.2538 -120.03

0.1619 0.2825 30.297 0.5317 95.333

-0.0007 0.0012 0.0073 -0.0014 0.2183

0.1229 0.0619 2.1988 0.1087 21.3162

21

0.5212 0.2146 -0.4621 0.5813 -35.838

Table 4. Input images for testing iterative registration algorithm. Images 2-6 are scaled, rotated, and translated versions of Image 1. All transformations are with respect to (0,0) (i.e., corner of image) as origin. Image

Scale (σ)

Rotation (α) (°)

Translation (t1)

Translation (t2)

1 2 3 4 5 6

1.0 1.0 1.0 1.1 1.1 0.9

0.0 5.0 0.0 0.0 5.0 5.0

0.0 0.0 10.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0 0.0 0.0

22

Table 5. Errors of recovery of registration parameters for a noise-corrupted image, subject to prescribed scaling, rotation and translation according to Table 4. Results are given for added Gaussian noise with standard deviation 10% and 40% of peak amplitude. The results for the noise-corrupted cases are the medians and Median Absolute Deviations (MAD) calculated over 10 independently noise-corrupted copies of each image. Figure 4a shows the 40% noise-corrupted image. No pre-registration was applied. Iterative registration was applied to level-2 blurred images (Figure 4c). Hierarchical registration, using the results of one level as starting values for iterative registration at the next finer level, was attempted, but did not in general produce any significant improvement over one-level (js = 2) registration. The results of Tables 2 and 4 show clearly that the iterative algorithm, which employs robust regression to find the registration parameters, can tolerate much higher corruption by noise than the preregistration algorithm. Note also that in Image 6, a few registration points are lost due to occlusion by the image boundary. No pre-registration was used. 10% Gaussian noise Image 2 3 4 5 6

Image 2 3 4 5 6

∆σ -0.0026 -0.0024 -0.0041 -0.0037 -0.0013 ∆σ 0.0009 0.0052 0.0034 0.0030 0.0013

MAD 0.0004 0.0012 0.0012 0.0005 0.0004

∆α (°) 0.1283 0.0639 0.1726 0.1403 0.1217

∆t1 -0.0914 0.1248 -0.0219 0.1412 -0.2851

MAD 0.3010 0.1626 0.4167 0.3173 0.2743

∆t2 0.6309 0.4121 0.6407 0.6892 0.3141

MAD 0.2691 0.2142 0.2421 0.1809 0.1980

MAD 0.0026 0.0038 0.0059 0.0042 0.0030

40% Gaussian noise ∆α (°) MAD ∆t1 0.0018 0.2260 -0.5010 0.0653 0.3264 -0.4583 0.4224 0.1575 -0.9629 0.4227 0.4311 -1.4927 -0.0344 0.1580 0.1052

MAD 0.5526 1.0093 0.6553 0.7345 0.2267

∆t2 -0.0173 -0.6771 0.4260 0.2593 -0.7068

MAD 0.6066 1.0705 1.2124 0.8297 1.0359

MAD 0.0703 0.0422 0.0853 0.1010 0.0808

23

Table 6. Results of Woods’ algorithm: Errors of recovery of registration parameters for noise-corrupted images, subject to prescribed scaling, rotation and translation according to Table 4. Compare with results in Table 5: the accuracy of recovery using Wood’s algorithm is comparable in the 10% added noise case, but slightly inferior for the 40% added noise case. 10% Gaussian noise Image 2 3 4 5 6

∆σ -0.0031 -0.0025 -0.0053 -0.0040 -0.0028

MAD 0.0009 0.0007 0.0008 0.0009 0.0011

∆α (°) 0.0161 0.0631 0.1611 0.0686 0.1682

MAD 0.0750 0.1418 0.0666 0.1334 0.0786

∆t1 0.1501 0.0380 0.1831 0.2987 -0.2610

MAD 0.2629 0.2556 0.3797 0.2071 0.1967

∆t2 0.5869 0.5732 0.8753 0.7214 0.6737

MAD 0.1770 0.2074 0.3082 0.3017 0.2474

MAD 0.9694 1.3813 1.5027 1.3069 1.3694

∆t2 -1.1309 -1.7342 -0.8568 -1.4365 -2.5837

MAD 1.3487 0.7743 1.3393 1.3573 2.0126

40% Gaussian noise Image 2 3 4 5 6

∆σ 0.0042 0.0089 0.0077 0.0094 0.0142

MAD 0.0041 0.0038 0.0051 0.0098 0.0058

∆α (°) 0.0097 0.3220 0.2821 0.0199 -0.2511

MAD 0.4279 0.3630 0.7183 0.6952 0.3100

24

∆t1 -0.8398 -1.7775 -1.8246 -1.0853 -0.2237

List of Figure Captions Figure 1: The T2*-weighted 256 by 256 pixel image used for testing the registration algorithm. Figure 2: The images resulting from scaling, rotating and translating the image of Figure 1 according to the transformations specified in Table 1. Image 6 shows substantial loss of information by occlusion by the image border. The pre-registration algorithm is sufficiently robust to still be able to register this image without large error9. Figure 3: The set of registration points derived from Image 1, the reference image for registration (see Figure 1). These are the edge points found by the Mallat-Zhong waveletbased edge detector at level 2 of blurring, using the method described in Section 2, and used by the iterative algorithm. For the computations of this paper, N = 1000 points were used. Figure 4: The noise-corrupted image (a) obtained by adding Gaussian noise with standard deviation equal to 40% of the peak amplitude of pixel intensity in the original uncorrupted image (Figure 1). The blurred images resulting from applying the MallatZhong wavelet blurring filter at level 1 (b), level 2 (c), and level 3 (d), are also shown. The signal-to-noise ratio for each case is (a) 0.24, (b) 3.1, (c) 12.5, and (d) 47.3. As Table 5 shows, the iterative algorithm was still able to register the scaled, rotated and translated versions of this image (Table 4) with a maximum error of less than 3 pixels.

25

References 1.

Brown, L.G. A survey of image registration methods. ACM Computing Surveys 24: 325-376; 1992.

2.

Pratt, W.K. Digital Image Processing. New York: John Wiley and Sons; 1978: 526566

3.

Goshtasby, A. Template matching in rotated images. IEEE Trans. Pattern Analysis and Machine Intelligence 7: 338-344; 1985.

4.

Segman, J. Fourier cross correlation and invariant transformations for an optimal recognition of functions deformed by affine groups. J. Opt. Soc. America A 9: 895902; 1992.

5.

Kuglin, C.D.; Hines, D.C. The phase correlation image alignment method. Proc. of the IEEE 1975 International Conference on Cybernetics and Society. IEEE: New York, 1975: pp. 163-175.

6.

Oppenheim; A.V. Lim, J.S. The importance of phase in signals. IEEE Proc: 69: 529-541; 1981.

7.

De Castro, E.; Morandi, C. Registration of translated and rotated images using finite Fourier transforms. IEEE Trans. Pattern Analysis and Machine Intelligence 9: 700703; 1987.

8.

Lee, D.J.; Krile, T.F.; Mitra, S. Digital registration techniques for sequential fundus images. IEEE Proc. SPIE 829: 293-300; 1987.

9.

Chen, Q.-s.; Defrise, M.; Deconinck, F. Symmetric phase-only matched filtering of Fourier-Mellin transforms for image registration and recognition. IEEE Trans. Pattern Analysis and Machine Intelligence 16: 1156-1168; 1994.

10.

Hu, M.K. Visual pattern recognition by moment invariants. IRE Trans. Information Theory 8: 179-187; 1962.

11.

Abu-Mostafa, Y.S.; Psaltis, D. Recognition aspects of moment invariants. IEEE Trans. Pattern Analysis and Machine Intelligence 6: 698-706; 1984.

12.

The, C.H.; Chin, R.T. On image analysis by the methods of moments. IEEE Trans. Pattern Analysis and Machine IntelligenceVol. 10: 496-513; 1988.

26

13.

Borgefors, G. Hierarchical chamfer matching: A parametric edge matching algorithm. IEEE Trans. Pattern Analysis and Machine Intelligence 10: 849-865; 1988.

14.

Besl, P.J.; McKay, N.D. A method for registration of 3-D shapes. IEEE Trans. Pattern Analysis and Machine Intelligence 14: 239-256; 1992.

15.

van den Elsen, P.A.; Maintz, J.B.A.; Pol, E.-J. D.; Viergever, M.A. Image fusion using geometrical features. IEEE Proc. SPIE: 1808, Visualization in Biomedical Computing: 172-186; 1992.

16.

Hu, X.; Ahuja, N. Feature extraction and matching as signal detection. International J. of Pattern Recognition and Artificial Intelligence 8: 1343-1379; 1994.

17.

Burr, D.J. A dynamic model for image registration. Computer Graphics and Image Processing15: 102-112; 1981.

18.

R. Bajcsy,R.; Broit, C. Matching of deformed images. 6th International Conference on Pattern recognition; 1982. pp 351-353.

19.

Bajcsy, R; Kovacic, S. Multiresolution elastic matching. Computer Vision, Graphics and Image Processing 46: 1-21; 1989.

20.

Daubechies, I. Orthonormal bases of compactly supported wavelets. Comm. Pure and Appl. Math 41: 909-996; 1988.

21.

Burt, P.; Adelson, E. The Laplacian pyramid as a compact image code. IEEE Trans. Communications 31: 482-540; 1983.

22.

Hansen, M.; Anandan, P.; Dana, K.; van der Waal, G.; Burt, P. Real-time scene stabilization and mosaic construction. Proceedings IEEE Workshop on Applications of Computer Vision; 1994; pp.54-62.

23.

Lavallee, S.; Szeliski, R.; Brunie, L. Matching 3-D smooth surfaces with their 2-D projections using 3-D distance maps. Proceedings SPIE, No. 1570 - Conference on geometric methods in computer vision; 1991, San Diego, CA; pp.322-336.

24.

Zheng, Q.; Chellappa, R. A computational vision approach to image registration. IEEE Trans. Image Processing 2:311-326; 1993.

25.

Woods, R.P.; Cherry, S.P.; Mazziotta, J.C. Rapid automated algorithm for aligning and reslicing PET images. Journal of Computer Assisted Tomography 16: 620-633; 1992.

27

26.

Rousseeuw, P.J.; Leroy, A.M. Robust Regression and Outlier Detection. New York: John Wiley and Sons; 1987.

27.

Stromberg, A.J.; Ruppert, D. Breakdown in nonlinear regression. J. American Stats. Assoc. 87: 991-997; 1992.

28.

Stromberg, A.J. Computation of high breakdown nonlinear regression parameters. J. American Stats. Assoc. 88: 237-244; 1993.

29.

Donoho, D.L.; Huber, P.J. The notion of breakdown point. In: P. Bickel, K. Doksum and J.L. Hodges (Eds.). A Festschrift for Erich Lehmann. Belmont, CA: Wadsworth; 1983.

30.

Nelder; J.A. Mead, R. A simplex method for function minimization. Computer Journal 7: 308; 1965.

31.

Polak, E. Computational Methods in Optimization. New York: Academic Press; 1971: Section 2.3

32.

Press, W.H.; Flannery, B.P.; Teukolsky, S.A.; Vetterling, W.T. Numerical Recipes in C, Second Edition. New York: Cambridge; 1992: Chapter 9.

33.

Mallat; S. Zhong, S. Characterization of signals from multiscale edges. IEEE Trans. Pattern Analysis and Machine Intelligence 14: 710-732; 1992.

34.

Mallat, S.; Hwang, W.L. Singularity detection and processing with wavelets. IEEE Trans. Information Theory 38: 617-643; 1992.

35.

Xu, Y.; Weaver, J.B.; Healy, D.M.; Lu, J. Wavelet transform domain filters: A spatially selective noise filtration technique. IEEE Trans. Image Processing 3: 747758; 1994.

36.

Bruce, A.G.; Donoho, D.L.; Gao, H.-Y.; Martin, R.D. Denoising and robust nonlinear wavelet analysis. IEEE Proc. SPIE, Vol. 2242, Wavelet Applications; 1992. , pp 325-336.

37.

Saint-Marc, P.; Chen J.-S.; Medioni, G. Adaptive smoothing: A general tool for early vision. IEEE Trans. Pattern Analysis and Machine Intelligence 13: 514-529; 1991.

38.

Perona; P. Malik, J. Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Analysis and Machine Intelligence 12: 629-639; 1990.

28

39.

Steen, E.; Olstad, B. Scale-space and boundary detection in ultrasonic imaging using nonlinear signal-adaptive anisotropic diffusion. Proc. SPIE No. 2167 - Image Processing: 116-127; 1994.

40.

Amit, Y. A nonlinear variational problem for image matching. SIAM J. Sci. Comput. 15: 207-224; 1994.

41.

Szeliski, R.; Lavallee, S. Matching 3-D anatomical surfaces with non-rigid deformations using octree splines. Proceedings IEEE Workshop on Biomedical Image Analysis; 1994, pp. 144-153.

42.

ter Haar Romeny, B.M.; Florack, L.M.J.; Salden, A.H; Viergever, M.A. Higher order differential structure of images. Image and Vision Computing 12: 317-325; 1994.

29

Figure 1

30

1

2

3

4

5

6 Figure 2

31

Figure 3

32

a

b

c

d Figure 4

33