Weighted Least Squares for Point-Based Registration

Weighted Least Squares for Point-Based Registration in Digital Subtraction Angiography (DSA) Thorsten M. Buzug*, Jürgen Weese and Cristian Lorenz Philips Research Laboratories Hamburg, Röntgenstraße 24, 22335 Hamburg, Germany ABSTRACT Four main problems have to be solved for template matching based motion compensation in digital subtraction angiography. All the problems are concerned with the similarity measure that is the objective function to be optimized within the template matching procedure: 1) Due to the injection of contrast agent, mask and contrast image are dissimilar, which degrades the quality of some similarity measures: As described recently, we can cope with this problem using robust similarity measures based on strictly convex weighted one-dimensional histograms [see, for instance, Buzug and Weese, Journal of Computerized Medical Imaging and Graphics 22, 2 (1998) 103-113 and the papers cited therein]. 2) Homogeneous areas in the fluoroscopic images lead to an insufficient quality of the similarity measure: To solve this problem an estimate of the maskimage information content is calculated via the entropy. Templates with a high contrast show a broad histogram whereas templates with a low contrast show a narrow distribution of gray-values. The histogram dispersion for each template is quantified by the entropy which is subsequently used as a measure of reliability of the matching results. Consequently, this quantification of the mask-image flatness can be used for an isotropic weighting in the linear least squares procedure. The accuracy of the motion vectors obtained from homogeneous regions can accordingly be ranked low. 3) Shift invariant structures in fluoroscopic images (e.g. straight lines or edges) lead to a ridge-like objective function that potentially gives wrong results from the optimization procedure: If a ridge-like structure of the objective function is present, movements along this direction cannot be detected. Therefore, the local accuracy of the estimated motion component parallel to these directions must be ranked low while the corresponding orthogonal direction must be ranked high. We here present a technique to obtain local, directional rankings from the shape of the objective function which especially improves the quality of DSA images obtained from peripheral areas like the shinbone. 4) Inhomogeneous movements inside a single template lead to ambiguous or even irrelevant optima of the objective function: This problem is out of the scope of the present paper and will therefore not be addressed here. The performance of the point-based registration using different weightings in the least squares procedure (equally, isotropic and anisotropic weighting) has been compared. The isotropic and anisotropic weighting turned out to be superior to the equally weighted least-squares procedure. Keywords: Digital subtraction angiography, template matching, registration, similarity measure, weighted least squares

1. INTRODUCTION For vessel diagnosis X-ray fluoroscopy is a standard of today’s clinical routine (see e.g. [1]). Visibility of the vessels is provided by injection of radio-opaque contrast agent during image acquisition. To get rid of the static, permanent structures in the X-ray image, like for instance bone shadows, and to enhance the visibility of the vessel tree, digital subtraction angiography (DSA) is a widely accepted technique (see [2] and the papers cited therein). One image, the mask image, is acquired before contrast injection, and a sequence of typically ten to twenty images is taken during contrast injection. The latter ones are called contrast images. If mask and contrast image are subtracted, all static, permanent structures, visible in both images disappear and only the contrasted vessels remain. However, due to patient motion artifacts appear in the subtraction image. While in clinical routine an interactive shifting procedure is established to compensate for patientmovement artifacts, an automatic affine correction based on motion vector fields has previously been proposed [3]. The affine correction allows for the compensation of shift, scale, rotation as well as skew artifacts and is therefore better adapted to the appearing artifacts of clinical practice than the manual shift operation. However, four main problems have to be solved for a template-matching based motion compensation in digital subtraction angiography. All the problems are concerned with the similarity measure that is the objective function to be optimized within the template matching procedure. *

Current address and correspondence: RheinAhrCampus, University of Applied Sciences Remagen, Südallee 2, 53424 Remagen, Germany; E-mail: [email protected]; Tel.: +49 (0) 26 42 / 932 – 318, Fax: +49 (0) 26 42 / 932 – 301

Part of the SPIE Conference on Image Processing • San Diego, California • Februar 1999 SPIE Vol. 3661 • 0277-786X/99

139

1. 2. 3. 4.

Due to the injection of contrast agent, mask and contrast image are dissimilar, which degrades the quality of some similarity measures. Homogeneous areas in the fluoroscopic images lead to an insufficient quality of the similarity measure. Shift invariant structures in the fluoroscopic images (e.g. straight lines or edges) lead to a ridge-like objective function (similarity measure) that potentially gives wrong results from the optimization procedure. Inhomogeneous movements inside a single template lead to ambiguous or even irrelevant optima of the objective function (similarity measure).

The latter problem addresses a dilemma of DSA. The patient motion that induces the distortions are inherently 3dimensional, whereas the images to be compared are 2-dimensional. This may violate topology of the 2-dimensional mapping between mask and contrast image in the sense that border points and inner points of a projected object need not remain border and inner points after transformation. Template-matching procedures cannot cope with such a problem. Even if two distinct homogeneous movements are detectable by clearly separable maxima of the energy similarity measure, unfortunately, the spatial information is lost, and therefore, it is not known which maximum in the measure corresponds to which pixel inside the template. Besides this ambiguity, in real images the patient movements are often not homogeneous such that no clear separation of maxima can be found. The authors do not expect that this problem can be solved on the basis of a single projection image pair to an extent that all motion artifacts disappear in subtraction images. The crucial point for the solution of the first problem is the choice of an appropriate similarity measure [4-9], because mask and contrast image are inherently dissimilar by injection of contrast agent. Recently, similarity measures have been introduced, obtained from weighted one-dimensional histograms that are optimally adapted to the dissimilarities of DSA [10,11]. Such measures are defined in a 3-step procedure that consists in: (i) subtraction of images inside the template, (ii) calculation of gray-value histogram of the difference image and normalization of the histogram according to Σpk=1, where pk is the fraction of pixels with gray-value gk. The fraction of pixels depends on the shift parameters r and s: pk=pk(r,s), and (iii) evaluation of similarity measure M(r,s)=Σh(pk), using an appropriately defined strictly convex weighting function h(p). It has been shown that the energy similarity measure - the sum of the squared histogram values, i.e. h(pk)=pk2 - turned out to be the most suitable measure for template matching and is consequently used in this paper. The histogram-based similarity measures are motivated by the observation that for optimal registration of mask and contrast image the difference image shows low contrast variation in the area of the template, whereas in the case of misregistration the contrast variation is larger. The situation corresponding to a perfect registration, i.e. a very peaky histogram, must have the maximum distance from the worst case, i.e. an equally distributed histogram. That distance is evaluated by the similarity measure. In the present report a solution is presented for the second and third problem mentioned above. In section 2 an isotropic weighting based on an entropy measure is presented to account for the reliability of a single template match. In section 3 it is shown how the parameters of local ellipses, i.e. the principal axes, are obtained from the energy similarity measure and how these parameters can be interpreted as uncertainties within the least squares approximation of an affine transformation used for patient motion correction. In section 4 experimental results are presented and the isotropic entropy-based weights are compared to the ellipses obtained form the energy similarity measure. In section 5 the conclusions are summarized.

2. ISOTROPIC WEIGHTINGS BASED ON THE ENTROPY MEASURE To solve the second problem, i.e. homogeneous areas in the fluoroscopic images lead to an insufficient quality of the similarity measure, a 2-step procedure is proposed. In the first step we exclude templates that are expected to give incorrect results: Due to the well known fact from signal-processing theory that for matched filters the so-called correlation gain depends on the product of template size and signal bandwidth [12], only those templates should be incorporated in our algorithm which have sufficient contrast variation. We have suggested an exclusion technique [13] using an estimate of the mask-image contrast variation calculated via the entropy h = -Σ pg log pg, where pg is the fraction of pixels with gray-value g and the sum runs from 0 to gmax, the maximum gray-value of the mask image. Templates with a high contrast show a broad histogram whereas templates with a low contrast show a narrow distribution of gray-values. Therefore, only those templates having an entropy that exceeds a certain threshold are taken into account. In the second step it is utilized that the entropy values for the templates are estimates for the respective bandwidth and lead to an information about the reliability of the matching results. Therefore, we have proposed to use this information about the flatness of the mask image for a entropy-based weighting in the linear least squares procedure. The accuracy of the motion vectors obtained from homogeneous regions can then accordingly be ranked low.

140

It is exemplarily shown here how this works for the approximation of an affine motion-correction mapping. A set of homologous landmarks, i.e. the template centers, resulting from the template matching is straightforwardly used to estimate the parameters of an appropriate transformation f := (fx,fy) relating the points of the mask image p := ( x , y) to the corresponding points in the contrast image p’ := ( x ', y ') inside the region-of-interest:

(

)

( x' , y' ) → ( x, y) = f x ( x' , y' ), f y ( x' , y' ) .

(1)

The affine correction can be written as

x = a11 + a12 x '+ a13 y '

(2)

y = a 21 + a 22 x '+ a 23 y '

As there are generally far more than three homologous landmarks, this is an overdetermined problem. As usual the merit function 2 N ⎛ p i − f ( p' i ) ⎞ 2 (3) ⎟ χ =∑⎜ σi ⎠ i =1 ⎝ must be minimized to find the best parameters for the transformation. N is the number of homologous points and the value σi is the standard deviation for the ith homologous data-point pair. A singular value decomposition (SVD) is employed to produce a solution that is the best result in the least-squares sense [14]. We can write eq. (3) as 2

χ 2 = p − Aa ,

(4)

where A is the weighted design matrix, i.e.

⎛ x1 ⎜ ⎜ x2 2 χ = ⎜ ⎜ ⎜ ⎜ ⎝ xN

/σ1 /σ2 . . /σ N

⎛ 1/ σ1 y1 / σ 1 ⎞ ⎜ 1/ σ 2 ⎟ y2 / σ 2 ⎟ ⎜ ⎜ . . ⎟ −⎜ ⎟ ⎜ . . ⎟ ⎜ . ⎟ yN / σ N ⎠ ⎜ ⎝1 / σ N

x ' 1 /σ 1 x ' 2 /σ 2 . . . x ' N /σ N

y ' 1 /σ 1 y ' 2 /σ 2 .

⎞ ⎟ ⎟⎛ ⎟ ⎜ a 11 ⎟ ⎜a . ⎟ ⎜ 12 ⎝a . ⎟ 13 ⎟ y ' N /σ N ⎠

2

a 21 ⎞ ⎟ a 22 ⎟ ⎟ a 23 ⎠

(5)

The values for the standard deviations are often not known and hence set to σi=1. Unfortunately, the set of homologous landmarks is always error prone. The template matching procedure yields estimates of the motion vectors that are expected to be inaccurate in areas where the contrast variation of the mask image is low. Additionally, even outliers may occur in those regions as demonstrated above. Therefore, it is of great importance to obtain estimates for the ‘measurement error’ or reliability of the motion vector field. These estimates are given by the isotropic weights obtained from entropy values for the mask image templates. Let M be the number of templates T’={tj} for j=1,...,M that fit the user-defined region-of-interest. Each template tj is associated with an entropy value hj. The set of templates is sorted with respect to the set H’={hj} for j=1,...,M and a predefined percentage (N/M*100) of templates with lowest entropy is discarded. The remaining sets of templates and associated entropies are re-indexed such that T’⊇T={ti} with H’⊇H={hi} for i=1,...,N and N≤M. The set of entropies is normalized according to

wi =

hi − min {hk } k = 1,..., N

⎛⎜ ⎞ hi − min {hk }⎟⎠ ∑ ⎝ k N = 1 ,..., i =1

,

(6)

N

and can be interpreted as a set W={wi} with i=1,...,N of reliabilities for the motion vectors. The weights must be understood as estimates for the reciprocal standard deviations. Figure 1 illustrates two examples for weights associated to the selected landmarks (or control points) inside a rectangular region-of-interest. Figure 1a shows an abdominal fluoroscopy and fig. 1b a shinbone fluoroscopy. 20 percent of templates with lowest entropy are discarded in both images.

141

a

b

Fig. 1: Mask images of the abdominal fluoroscopy (a) and shinbone fluoroscopy (b). Visualization of reliabilities of the motion vectors. The radii of the circles can be used as weighting factors in the linear least squares procedure.

3. ERROR ELLIPSES FROM THE ENERGY SIMILARITY MEASURE Due to shift invariant structures of the images to be compared, the optimization of the similarity measure is difficult, because these structures lead to a ridge-like structure of the objective function, such that movements along this direction cannot be detected. Therefore, the local accuracy of the estimated motion component parallel to these directions must be ranked low while the corresponding orthogonal direction must be ranked high. The isotropic, entropy-based weighting shows the drawback that shift invariant structures like straight lines or edges may lead to incorrectly high weightings. Hence, we here present a technique to obtain local, directional rankings from the shape of the objective function. In this section it is shown how the curvature of the similarity measure, i.e. the shape of the optimum of the objective function in the template matching procedure, leads to a local, directional measure of uncertainty.

a

b

s r

Menergy

s

d

10.00

c

5.00

0.00

-5.00

-10.00

s

r

-15.00 -15.00

r -10.00

-5.00

0.00

5.00

10.00

Fig. 2: Mask (a) and contrast image (b) of an abdominal fluoroscopy. (c) Energy similarity measure for the template indicated in the contrast image. r and s denote the shift coordinates in the interval r,s∈[-15,14]. The energy similarity measure is normalized to values in the interval [0,255]. (d) Contour plot of the similarity measure. The iso-energy lines are plotted in steps of 20.

142

In fig. 2 and 3 two examples of fluoroscopies and corresponding objective functions are given. Fig. 2 shows an abdominal scene. Mask (a) and contrast image (b) are of size (512x512) pixel. In this first example a (128x128)-pixel template is located at the vertebra. The shift coordinates indicated in the contrast image are denoted with r and s. The vertebra is well structured within the shift interval of ±15 pixels around the template center in vertical as well as in horizontal direction. Hence, the quality of the energy similarity measure Menergy is very good in any direction. Fig. 2c shows the measure in the respective interval. Fig. 2d shows the corresponding contour plot of Menergy. Near the optimum of the objective function at (r,s)=(0,0) it can be seen that the iso-energy lines, i.e. lines of constant value of Menergy, are almost circular. Therefore, we expect principal axes of equal length, which means equal accuracy of motion estimation in any direction. The second example shown in fig. 3 is a shinbone fluoroscopy of size (1024x1024) pixel. A (128x128)-pixel template is located such that a vertical, shift invariant edge of the shinbone is included. As expected, the energy similarity measure shows a ridge.

a

b

s r

s

Menergy

d

10.00

c 5.00

0.00

-5.00

s

-10.00

r

-15.00 -15.00

-10.00

-5.00

0.00

5.00

10.00

r

Fig. 3: Mask (a) and contrast image (b) of a shinbone scene. (c) Energy similarity measure for the template indicated in the contrast image. r and s denote the shift coordinates in the interval r,s∈[-15,14]. The energy similarity measure is normalized to values in the interval [0,255]. (d) Contour plot of the similarity measure. The iso-energy lines are plotted in steps of 20.

A slight optimum can be recognized near the expected optimal shift at (r,s)=(0,0). That means that either some structure is present also in the almost homogeneous areas representing soft tissue or/and a slight global bending of the bone edge breaks the shift invariance of the template. However, usually smaller templates are used for the template matching procedure of the DSA method. Consequently, the significance of that maximum will decrease due to less global bending of the bone structure inside the template. Fig. 3d shows the contour plot of the energy similarity measure Menergy. Near the optimum at (r,s)=(0,0) it can be seen that the iso-energy lines are elongated. Therefore, we expect a large principal axis in direction of the shift

143

invariant edge, which means low accuracy of motion estimation in that direction, and a small principal axis perpendicular to the edge, reflecting high accuracy of that motion component. In subsection 3.1 it is described how these anisotropic error estimates, represented by ellipses, are obtained from the objective function via principal components analysis, and in subsection 3.2 the anisotropically weighted least squares procedure is explained for parameter estimation of the affine transformation. 3.1 Estimation of ellipse parameters by principal component analysis The calculation of ellipses that can be used as local, directional error estimates, is a by-product of the sub-pixel estimation procedure. We have shown in [15] how sub-pixel shift value estimates are obtained by a quadratic approximation of the similarity measure in the vicinity of the extremum (given in pixel accuracy). The 8-pixel neighbourhood is used of the similarity measure optimum, i.e. the maximum of the energy similarity measure, to fit a quadratic function. The quadric parameters of the form

M ( x , y ) = b0 + b1 x + b2 y + b3 x 2 + b4 xy + b5 y 2

(7)

are computed using singular value decomposition. The standard form of eq. (7) is

a11 x12 + 2a12 x1 x 2 + a 22 x 22 + 2a 01 x1 + 2a 02 x 2 + a 00 = 0

(8)

which leads to the equivalent matrix notation

x T Ax + 2a T x + a 00 = 0 ,

(9)

where

⎛ a11 A=⎜ ⎝ a12

a12 ⎞ ⎛ a 01 ⎞ ⎛ x1 ⎞ ⎟ , a = ⎜ ⎟ and x = ⎜ ⎟ . a 22 ⎠ a ⎝ 02 ⎠ ⎝ x2 ⎠

(10)

To eliminate the second term in eq. (8) the coordinate system (reference system) must be aligned with the local coordinate system of the ellipse, given by its principal axes, using principal components analysis. Eigenvalues and eigenvectors of A are obtained by diagonalization (principal components analysis):

⎛ λ1 Σ=⎜ ⎝0

0⎞ T T ⎟ = B AB ⇔ A = BΣB . λ2 ⎠

(11)

The columns of B are the normalized eigenvectors of A. The eigenvectors span the local coordinate system at the ellipse, i.e. the principal axes. Inserting eq. (11) into eq. (9) yields

x T B Σ B T x + 2a T B B T x + a 00 = 0 ,

(12)

The new vector x* is obtained by multiplication with the rotation matrix BT: x*=BTx. The transformed quadric is hence given by (13) x *T ( B T AB ) x * + 2( a T B ) x * + a 00 = 0

or * * * * a11* x1*2 + a 22 x2*2 + a01 x1 + a 02 x2* + a 00 = 0 ,

(14)

respectively, which is called the principal components form. a*11 and a*22 are the eigenvalues λ1 and λ2, respectively. To eliminate the linear terms of eq. (13/14) a displacement x* = x − c is applied, i.e. a shift of the ellipse center to the origin of the reference system, where

⎛ a* a* ⎞ c T = 0.5 ⋅ ⎜ 01* , 02* ⎟ , ⎝ a11 a 22 ⎠

(15)

* 2 * a11 x1 + a 22 x 22 + a 00 = 0 ,

(16)

* 2 * 2 * * * . a 00 = a11 c1 + a 22 c2 − a 01 c1 − a 02 c2 + a 00

(17)

such that

where

144

This leads straightforwardly to the ellipse equation x12 x 22 + = 1. a a 00 − 00 − * * a11 a 22

(18)

The square roots of the denominators in eq. (18) yield the lengths of the principal axes. If A is positive definite, which holds in the present case, the nominators are always positive. Fig. 4 shows mask (a) and contrast image (b) of the shinbone fluoroscopy, introduced in fig. 3. A set of non-overlapping, quadratic (80x80)-pixel templates are indicated in both images (set of dashed squares). A number of shift invariant structures, i.e. bone edges, are present in this image scene and it can be seen that the large principal axis of the ellipses fall into the direction of the edge structures, if parts of the edges are included in the respective template.

a

b

Fig. 4: Mask (a) and contrast image (b) of a shinbone fluoroscopy. The location and size of the (80x80)-pixel templates are indicated in both images by the set of non-overlapping quadratic squares drawn with dashed lines. The dark crosses in the contrast image mark the respective template centers. As a result of the template matching procedure the estimated error ellipses are drawn in the contrast image for each template pair.

145

3.2 Estimation of affine transformation parameters considering anisotropic error estimates The covariance matrix is a standard element to take into account anisotropic error estimates in a least squares fit. The coefficients of the quadric are not representing the covariance matrix K, but its inverse K-1. The eigenvalues are therefore reciprocally proportional to the lengths of the principal axes. Let us denote the inverse covariance matrix with:

K

−1

⎛ a11 ⎜ = ⎜ a12 ⎜ ⎝ a 01

a 01 ⎞ ⎟ a 02 ⎟ . ⎟ a 00 ⎠

a12 a 22 a 02

(19)

The functional to be minimized is given by n ~ χ 2 = ∑ (Tp i − qi ) T K i−1 (Tp i − qi ) → min ,

(20)

i =1

where n is the number of landmarks, pi and qi are the (homogeneous) coordinates of mask and contrast image, respectively, and T is the affine transformation ⎛ t11 t12 ⎜ T = ⎜ t21 t22 ⎜ ⎝0 0

t13 ⎞ ⎟. t23 ⎟ ⎟ 1⎠

(21)

~ −1 To obtain K , we have to shift again the center of the quadric such that we obtain after normalization

⎛ a11 / a 00 ~ −1 ⎜ K = ⎜ a12 / a 00 ⎜ ⎝ 0

a12 / a 00 a 22 / a 00 0

0⎞ ⎟. 0⎟ ⎟ 1⎠

(22)

To solve the minimization problem given above, as usually, we have to set to zero the derivation of χ2 with respect to the elements of T, which leads to six linear equations with six unknown parameters.

4. COMPARISON BETWEEN ENTROPY WEIGHTS AND ERROR ELLIPSES The description of the motion vector uncertainty in terms of isotropic weights and anisotropic error ellipses show an interesting difference. An example set of weights (or reliabilities) is visualized in fig. 5a for the mask image of the abdominal scene introduced above. No rejection of templates is used, i.e. N=M holds for this example. The weights wi are visualized as radii of circles overlaid to the relevant template centers. It can be seen that those templates located in regions with high contrast correspond with large circles. Whereas, templates located in homogeneous regions correspond with small circles. The weights can be understood as an estimate for the reciprocal standard deviations in the least squares approximation of the affine motion parameters. In fig. 5b the corresponding error ellipses are marked in the contrast image. One would expect that the sizes of the ellipses are correlated with the reciprocal radii of the respective circles drawn onto the mask image in the way that an overall small ellipse, reflecting a small uncertainty, corresponds to a large circle, reflecting a large weighting or reliability. This is generally the case if the images to be compared are not dissimilar due to contrast injection. However, the comparison between fig. 5a and 5b reveals that in regions with large gray-value distortions the ellipses become very large which is in contradiction to the corresponding large isotropic weights that indicate a good reliability. This can be explained by the fact that the entropy-based isotropic weighting is calculated from the mask image alone, and consequently, no information about gray-value distortions influences the weight estimate. Therefore, the weight is high due to the well structured vertebra. On the other hand, the strong gray-value distortion introduced by the contrast agent reduces the overall sharpness of the difference image histogram, and hence, the sharpness of the energy similarity measure. This is correctly visualized by the relatively large ellipses.

146

a

b

Fig. 5: Mask (a) and contrast image of the abdominal scene. Mask and contrast have overlaid the isotropic weights and the anisotropic error ellipses, respectively.

5. EXPERIMENTAL RESULT: SHINBONE FLUOROSCOPY To show the differences between the alternative weightings we used the fluoroscopic image pair of the shinbone introduced in fig. 3. For demonstration purposes we artificially shifted the entire contrast image 5 pixel vertically and 8 pixel horizontally. Inside the region-of-interest a 5x5 grid of squared, non-overlapping (80x80)-pixel templates is used. The regions-of-interest indicated in the mask (fig 6a) and the contrast image (fig. 6b) are blown-up for better visualization. The isotropic weights are reflected by the different radii of the circles located at the center of each template in the mask image. The anisotropic or local, directional error estimates are represented by the ellipses located at the center of each template (dark crosses) in the contrast image. Additionally, the motion vector field is visualized by white lines attached to these template centers. The ellipses correspond to the shape of each objective function near the optimum. Therefore, it is not very easy to understand the shape by looking at the contrast image alone. As mentioned in the previous section, the size and direction is also influenced by the dissimilarities between mask and contrast image. Due to the proposed template rejection scheme [13] 40 % of the less contrasted templates are not included within the optimization procedure. Consequently, only those templates including the well contrasted bone edge steps are selected. But unfortunately, these templates, which are indicated to be optimal by the entropy-based rejection scheme exhibit the undesired shift invariance. In fact the remaining 60 % are indeed better than those which would obviously be located in homogeneous soft tissue areas, but here, only a local, directional weighting is capable to prevent for the destructive influence of shift invariance based outliers. Such outliers can be found for the three middle templates of the first column. While the motion component perpendicular to the step edge bone structure is correctly obtained by the optimization of the energy similarity measure, the component parallel to this shift invariance shows a deviation from the expected shift value. Fig. 7a shows the uncorrected subtraction result. Due to the introduced shift of (r,s)=(5 pixel, 8 pixel) strong artifacts appear near the step edges of the bones. Fig. 7b, c and d show the subtraction results using affine motion compensation based on equally weighted motion vectors, isotopic, entropy-based weighted motion vectors, and anisotropic weighted motion vectors, respectively. The analysis is focused on the indicated region-of-interest only. Thanks to the fact that an overdetermined system of equations must be solved for the affine correction transformation if more than 3 homologous point pairs are used, the three outliers in the motion vector field do not play a dominant role, even if all motion vectors are weighted equally. The image quality of fig. 7b is significantly better than that of the uncorrected subtraction result of fig. 7a. Nevertheless, the entropy-based isotropical weighting of the motion vectors leads to a further

147

slight improvement of the image quality (fig. 7c), because relative small weightings are attributed to the outlier vectors. However, the anisotropic weighting leads to the best result, which can be found in fig. 7d. Since the large principal axis of the ellipses, indicating the direction and largest amount of uncertainty, fall into the direction of the shift invariant bone edge structure, the respective components of the motion vectors are accordingly ranked low.

a

b

Fig. 6: Mask (a) and contrast image (b) of a shinbone scene. 40 % of the templates are discarded (ranked with respect to the mask image entropy). The regions-of-interest indicated in the mask and contrast image by the bright rectangles are blown up for better visualization. The distribution of reliabilities (the weights are proportional to the radius of the circles located at the template centers) is given in the mask image. The respective distribution of the error ellipses (the uncertainties are proportional to the principal axes of the ellipses located at the template centers) is given in the contrast image.

6. SUMMARY AND CONCLUSIONS Anisotropically weighted least squares has been applied to the registration problem of digital subtraction angiography (DSA) that is based on a template matching procedure for patient motion correction. A novel method has been presented for the extraction of local, directional weightings from the objective function, i.e. the energy similarity measure to be optimized. These weightings are obtained by principal components analysis and are represented by ellipses associated to each template. Especially for image pairs including shift invariant structures like straight lines or edges it has been shown that anisotropical error estimates improve the motion correction, and hence, the subtraction image quality. The anisotropic weighting has been compared to the recently proposed isotropic weighting. Both approaches have their merits and drawbacks. The entropy-based isotropic weighting is a by-product of the template ranking procedure and is therefore practically obtained for free, which is an attractive argument for the DSA application where the time consumption of the entire procedure determines the clinical acceptance.

148

a

b

c

d

Fig. 7: Subtraction images of the shinbone fluoroscopies given in fig. 6. Uncorrected subtraction image (a), equally weighted affine corrected (b), isotropically entropy-weighted affine (c), and anisotropically weighted affine correction (d) of patient motion prior to subtraction.

It has been shown that this method is capable to identify homogeneous regions of the fluoroscopies. Templates located in these areas are consequently associated with a low weighting which enables a sound suppression of outliers. Besides, the entropy weighting allows a forecast whether an outlier has to be expected or not. However, the limitations of this methodology are twofold. Firstly, the entropy-based weighting is obtained by a mask image analysis alone, and therefore, the degradation of the objective function due to dissimilarities between mask and contrast image are ignored. Secondly, shift invariant structures may lead to an relatively high entropy that is correct only for the motion component perpendicular to the straight structure. The anisotropic approach deals with the two problems that are left by the isotropic approach. Firstly, the anisotropic weighting is based on the combination of both images to be compared. Therefore, the overall sizes of ellipses, that are obtained from areas that are dissimilar due to injected contrast agent, are larger than ellipses obtained from similar areas.

149

This correctly reflects a loss of sharpness of the objective function. And secondly, the principal components analysis allows the assignment of different weight to different directions, i.e. the larger principal axis of an error ellipse falls into the direction of the straight line- or edge-like structure. On the other hand some drawbacks of this approach must be mentioned as well. Firstly, the time account of the overall DSA enhancement procedure is charged with the principal components analysis of each quadric that is obtained from the sub-pixel shift estimation. Secondly, ideal straight, shift invariant structures are usually not available in the human body, except for artificial objects, e.g. metal implants. Even the peripheral fluoroscopies show almost always a global curvature of the bone structures and/or a residual texture that can be detected by the very sharp energy similarity measure, if the template is large enough. For examples where no outliers are obtained from the template matching procedure the image quality is comparable to that of the isotropic approach, therefore, it is the impression of the authors, that the higher time consumption of the principal components analysis cannot by justified by the overall quality enhancement of anisotropically weighted least squares.

ACKNOWLEDGMENTS The authors would like to thank Dr. L. J. Schultze Kool, University Hospital Leiden, for providing us with the data sets. The algorithm was implemented on an experimental version of the EasyVision workstation from Philips Medical Systems and we would like to thank EVM (Easy Vision Modules, EasyVision/EasyGuide™) Advanced Development, Philips Medical Systems, Best, for helpful discussions.

REFERENCES 1. 2.

3.

4.

5. 6. 7. 8.

9. 10. 11. 12. 13. 14. 15.

150

W. A. Chilcote, M. T. Modic, W. A. Pavlicek et. al., “Digital subtraction angiography of the carotid arteries: A comparative study in 100 patients”, Radiology 139, p. 287, 1981. J. M. Fitzpatrick, J. J. Grefenstette, D. R. Pickens, M. Mazer and J. M. Perry, “A system for image registration in digital subtraction angiography”, in: Information processing in: medical imaging, C. N. de Graaf and M. A. Viergever (eds.), p. 414, Plenum Press, New York, 1988. Th. M. Buzug and J. Weese, “Improving DSA images with an automatic algorithm based on template matching and an entropy measure”, CAR’96, H. U. Lemke, M. W. Vannier, K. Inamura and A. G. Farman (Eds.), p. 145, Elsevier, Amsterdam, 1996. Th. M. Buzug, J. Weese, C. Fassnacht and C. Lorenz, “Using an entropy similarity measure to enhance the quality of DSA images with an algorithm based on template matching”, VBC’96, K. H. Höhne and R. Kikinis (Eds.), Lecture Notes in Computer Science 1131, p. 235, Springer, Berlin, 1996. Th. M. Buzug and J. Weese, “Voxel-based similarity measures for medical image registration in radiological diagnosis and image guided surgery”, Journal of Computing and Information Technology 6, p. 165, 1998. L. V. Tran and J. Sklansky, “Flexible mask subtraction for digital angiography”, IEEE Trans. on Medical Imaging 11 (1992) 407 and SPIE 939, p. 203, 1988. J. M. Fitzpatrick, D. R. Pickens, H. Chang, Y. Ge and M. Özkan, “Geometrical transformations of density images”, SPIE 1137, p. 12, 1989. A. Venot, J. Y. Devaux, M. Herbin, J. F. Lebruchec, L. Dubertret, Y. Raulo and J. C. Roucayrol, “An automated system for the registration and comparison of photographic images in medicine”, IEEE Trans. Medical Imaging 7, p. 298, 1988. J. M. Fitzpatrick, D. R. Pickens, J. J. Grefenstette, R. R. Price and A. E. James, “Technique for automatic motion correction in digital subtraction angiography”, Optical Engineering 26, p. 1085, 1987. Th. M. Buzug, J. Weese, C. Fassnacht and C. Lorenz, “Image Registration: Convex functions for histogram-based similarity measures”, CVRMed’97, Lecture Notes in Computer Science 1205, p. 203, Springer, Berlin, 1997. Th. M. Buzug and J. Weese, Journal of Computerized Medical Imaging and Graphics 22, p. 103, 1998. R. Gagliardi: Introduction in Communications Engineering. John Wiley & Sons, New York, 1978. Th. M. Buzug, J. Weese and K. C. Strasters, “Motion detection and motion compensation for digital subtraction angiography image enhancement”, Philips Journal of Research 51, p. 203, 1998. W. H. Press, B. P. Flannery, S. A. Teukolsky and W. T. Vetterling, Numerical Recipes in C, pp. 534-539, Cambridge University Press, Cambridge, 1990. Th. M. Buzug, J. Weese, C. Fassnacht and C. Lorenz, “Elastic matching based on motion vector fields”, CAR’97, H. U. Lemke et al. (Eds.), p. 139, Elsevier, Amsterdam, 1997.