Automatic registration and alignment on a template ... - Semantic Scholar

IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 16, NO. 6, DECEMBER 1997

1

Automatic registration and alignment on a template of cardiac stress & rest reoriented SPECT images Jer^ome Declerck, Jacques Feldmar, Michael L. Goris and Fabienne Betting

Abstract | Single photon emission computed tomography (SPECT) imaging with 201Tl or 99m Tc agent is used to assess the location or the extent of myocardial infarction or ischemia. A method is proposed to decrease the eect of operator variability in the visual or quantitative interpretation of scintigraphic myocardial perfusion studies. To eect this, the patient's myocardial images (target cases) are registered automatically over a template image, utilizing a non-rigid transformation. The intermediate steps are: 1. Extraction of feature points in both stress and rest 3D images. The images are resampled in a polar geometry to detect edge points, which in turn are ltered by the use of a priori constraints. The remaining feature points are assumed to be points on the edges of the left ventricular myocardium. 2. Registration of stress and rest images with a global ane transformation. The matching method is an adaptation of the Iterative Closest Point algorithm. 3. Registration and morphological matching of both stress and rest images on a template using a non-rigid local spline transformation following a global ane transformation. 4. Resampling of both stress and rest images in the geometry of the template. Optimization of the method was performed on a database of 40 pairs of stress and rest images selected to obtain a wide variation of images and abnormalities. Further testing was performed on 250 cases selected from the same data base on the basis of the availability of angiographic results and patient strati cation. Keywords | non-rigid matching, B-splines, SPECT, myocardial perfusion imaging.

N

I. Introduction

UCLEAR medicine imaging provides 3D density maps of blood perfusion in a non-invasive way. In the stressrest study, two perfusion maps of the heart muscle, and more particularly the left ventricle (LV) are taken: one obtained after an injection of the tracer at rest (rest image) and the other after the injection of the tracer during maximal exercise (stress image). Comparing the two images provides a classi cation of areas of the myocardium in 3 main classes (Fig. 1):  the intensity distribution is normal in both rest and stress images, the area is normal,  there are one or more regions with abnormally low count rate densities in both rest and stress images. The abnormality is said to be xed, and this has been considered to connote a myocardial infarction, or in some cases a very narrow stenosis, with resting hypoperfusion and a hibernating or stunned myocardium, J. Declerck is with the Epidaure Group, INRIA, B.P. 93, 06902 Sophia-Antipolis Cedex, France (e-mail: [email protected]). J. Feldmar and F. Betting are working in the Epidaure Group, INRIA, B.P. 93, 06902 Sophia-Antipolis Cedex, France. M.L. Goris is professor of radiology in the Division of Nuclear Medicine, Stanford University Hospital, Stanford, CA 93405 USA.

Rest

Stress

Normal

Infarcted

Ischemic

Fig. 1. Classi cation of the perfusion of the myocardium from the nuclear medicine myocardial perfusion study. A lled square means a high intensity in the image.

 there are one or more regions of low count rate densities

in the stress image, but the densities are normal in the rest image. The abnormality is said to be transient, and connotes stress ischemia. This encoding subsumes two comparisons: the patient's image is compared to a virtual image of what the normal distribution of densities should be (external comparison). This virtual image accommodates normal variations. The stress image is also compared to the rest image (internal comparison). In both visual and quantitative analysis the images need to be realigned in some way, at least suciently to allow the comparison of equivalent myocardial segments. Classically, this analysis splits into three steps: 1. centering and reorientation, in order to resample the thoracic image in the geometry of the LV, 2. registering the pair of images by comparing the shapes in dierent slices, 3. comparison of the images to detect intensity dierences. Various steps have been automated in a variety of ways:  for the centering and reorientation, the authors of [1] and of those of [2], [3] de ne automatic methods to segment the myocardiumdirectly from the transaxial original images, or to de ne a center and a long axis as in [4]. Once the long axis is identi ed, the other walls are fairly easy to segment in the thoracic geometry and the reorientation is complete.  For the registration, the authors of [5], inspired by [6], show an automatic method to compute the rigid registration between the stress and rest images using correlationbased techniques. In [7], the authors also use a correlationbased technique with a stochastic sign change criterion to match the images. In [8], the authors also use correlation techniques to nd a non-uniform scaling transformation to match dierent hearts. A template is built by normalizing and averaging the resampled images. Some more complicated (free-form) transformations may be found with the

2

IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 16, NO. 6, DECEMBER 1997 reconstructed original volume patient heart

reoriented region of interest

liver

Fig. 2. The heart is laying on the diaphragm. The small cube which de nes the volume of the heart is extracted from the whole thorax transaxial image reconstructed from the projections. A part of the liver is often included in the extracted cube.

Frontal (short axis)

intensity-based method described in [9]. All those methods use the intensity information, which may be problematic since the aim of the myocardial perfusion study is to detect intensity dierences that those methods tend to minimize. Sagittal Transverse The aim of the method proposed in this paper is to pro(vertical long axis) vide a reproducible, robust and highly automatic way of (horizontal long axis) comparing the stress and rest perfusion images of the same patient, or images of dierent patients. To this end, we Fig. 3. The reoriented heart: the slices show dierent views of the heart as it appears on conventional nuclear medicine interfaces. introduce a robust 3D matching procedure by which stress Bottom left on the frontal view, part of the liver appears beside and rest images are matched to each other and morphologthe heart wall (infero-septal wall). ically aligned to a \template" image of the heart. II. Data acquisition, reconstruction and pre-processing

The acquisition is in tomographic mode, step and shoot, with 32 projections over 180 degrees or 64 over 360. The image format is 64 x 64 with size of 6.25 mm x 6.25 mm, and an acquisition time of 15-25 seconds/step. The learning set, consisting of 32 paired stress and rest images were selected from a clinical data base to obtain a mixed group of image abnormalities and variations, with a more important proportion of \dicult" cases than an average data base should contain in order to test the robustness of the registration algorithm. The method and the parameters were set with this data base. An additional set of 8 cases (chosen for their disparity in size and shape) were added to the data base to make a crude validation of the parameters. The \optimized" algorithm was then tested on 250 cases, selected from the same data base on the basis of the availability of angiographic information, or a clinical validation for low probability of coronary artery disease. The images were acquired with both 201Tl and 99mTc tracers. A three-dimensional isometric image was reconstructed from the projection data by ltered backprojection and prealigned manually by an expert and the image is zoomed by a factor of 2-2.5 (linear). This alignment ensures that the center of the image is inside the cavity of the left ventricle and that the direction in the image is parallel to the apico-basal vector (Fig. 2 and 3). These constraints are crucial for our edge extraction step, but

they are not hard to satisfy. Where necessary and to avoid including other structures or organs such as the liver or the kidneys (which are very close to the heart and which readily x the radioactive tracer), the image is masked. The mask is an ellipsoid whose three axes can be enlarged or reduced individually, and which can be displaced in the image volume. This mask is centered and sized to cover the myocardium in its entirety and exclude high density regions adjacent but not belonging to it. Sizing and location is performed under visual control, while the three orthogonal slices intersecting in the center of the ellipsoid are visualized. All voxel values outside the mask are set to zero. III. Description of the morphological matching method

The method that we propose splits into two major steps:  The stress/rest pair of images is matched using a featurepoints based method, yielding an ane transformation which de nes a correspondence between a point in the stress image and a point in the rest image.  A \template" heart is matched to the stress image using local spline transformations (de ned in section III-B.4) and both stress and rest images are resampled in its geometry. In the method, this template is only de ned by its shape. In our experiments, a morphological normal case is chosen as the template (see section III-A.3).

DECLERCK ET AL.: AUTOMATIC REGISTRATION AND ALIGNMENT ON A TEMPLATE OF MYOCARDIAL PERFUSION STUDIES

After this transformation of the images (the output is a new stress/rest pair in a new geometry), the coordinates (x,y,z) of a 3D voxel in any image (stress, rest, template) correspond to the same part of the myocardium and allows quantitative intrapatient and interpatient comparison.

3

w w φ M

A. Extraction of feature points

v

A v

C θ

θ

A.1 Edge detection in a 3D polar map φ A The feature points we want to extract from the image are u supposed to feature edges of the heart wall. It would be useless to try to nd the precise location of the true limit of Fig. 4. The (x,y,z) cartesian location of point M is converted into the heart wall, since the existence of discrete boundaries of spherical coordinates in the 3D polar image, the depth Z is the distance from the center of the cartesian image, the position the myocardium in the images is compromised by blurring, (X ,Y ) in the plane is de ned with the two angles  and , like due to cardiac and respiratory motion and by the relatively in the 2D planispheric mapping : parallels ( constant) become poor spatial resolution of the imaging system. Therefore, concentric circles and meridians ( constant) become segments. the points we are looking for should be a rough, but stable, approximation to the myocardium edges. To detect such feature points, we use an algorithm based on the rst derivative of the image by detecting the edges with a Canny-Deriche recursive lter [10] in a particular 3D polar geometry. The 3D polar map is de ned as the original image resampled in spherical coordinates (r, , ): in the resampled image X,Y and Z are de ned as: 8 < X =  cos() Y =  sin() (1) : Z=r M

Since the angular sampling is perfomed in 128 steps for X and Y , and over 32 steps for r, from a 64 x 64 x 64 image, a 128 x 128 x 32 image is constructed. In this image, the plane (X,Y ,0) contains the central voxel value 128 x 128 times. Oversampling decreases with increasing values of Z. Since the heart is roughly spherical around the apex and roughly cylindrical around the base, in the 3D polar map, the myocardium has the shape of an irregular pancake. (Fig. 4; 5). The advantage of this transformation lies in the fact that radial sampling can now occur in (X,Y ,Z) coordinates: a radius is a segment along the Z axis and is supposed to cross the myocardial wall orthogonally. Extracting edges in this map with a 3D edge detector provides a smooth de nition of edge points. Fig. 6 shows intensity pro les along two dierent radii. Because the mask is sometimes used to hide non-cardiac areas, some arti cial edges are created, with a very high gradient value (the background noise intensity decreases sharply from roughly 0.1 (normalized value) down to 0 in a one pixel shift). In order to eliminate such arti cial extrema, the intensity pro le along the radius is modi ed after the point before the last non-zero intensity point: intensity is set constant after this point. The last non-zero intensity point has a biased value due to partial volume eect and resampling, this is why its value is not used (Fig. 7). A.2 Filtering the edge points Some detected edges do not correspond to the heart wall, as shown on the dotted curve in Fig. 6: an edge of another organ has been detected. Therefore, a nal step is required,

Fig. 5. Left, lhe LV in the original geometry, right, the LV in the polar geometry. myocardium

forbidden cone

Intensity 2 OUTLIER ENDOC. EPIC.

2 1 1

other organ

radius

Fig. 6. Intensity pro les along two dierent radii starting from the center of the image. The black dots show on both curves the edges detected with the Canny-Deriche lter. Curve 1 shows an ideal situation, the radius crosses a healthy and well contrasted myocardial area. For curve 2, the radius crosses a hypoperfused myocardial area and another \hot" organ, yielding a non cardiac edge. The rays are sampled in any direction but a 15 degrees cone around the apico-basal direction.

using a priori constraints: along a radius, on the intensity pro le, the rst signi cant increasing gradient extremum is likely to belong to the endocardium and the rst following signi cant decreasing gradient extremum is likely to belong to the epicardium. Based on this empirical description, we de ne some constraints to lter incorrect edge points:

4

IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 16, NO. 6, DECEMBER 1997 I

intensity

endocardium

the expected configuration both edges are kept

epicardium

r

mask

I

the last edge is eliminated the two first are kept

radius

Fig. 7. The original pro le is shown of a solid line, the black dots show the detected edges. The third dot is an arti cial edge due to a mask which sharply sets the intensity to zero. The dashed line shows the corrected pro le used to remove this erroneous edge.

1. Any edge point in a 15-degree cone around the apicobasal vector is eliminated. This angle value is very low and it is dicult to set correctly for a large data base. Such points almost certainly do not belong to the myocardial edge. 2. The lling procedure removes gradients along the radii, but it yields gradients orthogonal to the radii. Such extrema are detected on the deepest plane (32) of the polar map (a high radius value, for which we are sure that no cardiac structure is represented). If there is an edge point for r = 32, all the neighboring edge points which have the same gradient norm as this one (with a 10 % margin due to smoothing during edge detection) are eliminated. Experiments on our data base have shown that around 80 percent of those arti cial edge points are removed. 3. The remaining points are sorted with respect to the absolute value of the Z coordinate of the gradient. Thresholding is performed on the contour image in order to eliminate the lowest 20 percent, and only large connected components are retained (minimum requested size of the component is 500 points). 4. An ordering constraint is applied: we insist on con gurations tting our empirical description and reject edges for which we are sure are non-cardiac: we reject every edge point after the rst descending gradient extremum which is not too close to the center (a minimum radius of 5 is xed as a threshold). In Fig. 8, dierent con gurations of intensity pro les are listed with the corresponding algorithmic decision. Fig. 9 illustrates dierents steps of the ltering procedure on a real image. Finally, the points are converted back into cartesian coordinates. Fig. 10 shows the results of feature extraction for dierent hearts. The proportion of good points is satisfying, as the robust matching algorithm is able to cope with remaining outliers.

r I

the first descending edge is too close from the center: it is removed

r I

the epicardial edge is not detected we keep what we have !

r I

the endocardial edge is not detected the last (non-cardiac) edge is removed

r

Fig. 8. Left, some con gurations for the intensity pro le along a radius. Right, the consequent decision for the ltering.

between edge points and no spurious edges. Those edges de ne the shape we use for the registration. B. Registration

Edges are detected in both images (stress and rest, or stress and template). We assume that a rst derivative extremum in the rst image can be matched with a rst derivative extremum in the second image. Because in abnormal cases there are likely to be fewer edge points in the stress image than in the rest image, we de ned the transformation which deforms the stress edge points to the rest edge points (matching stress on rest). The matching method is an enhancement [11] of the iterative closest point [12], [13], adapted to our problem. B.1 The matching criterion We seek a matching function f: given a 3D point M in Image 1, f(M) should be the equivalent 3D point in Image 2. To calculate f, we assume that the image of a feature point in Image 1 is a feature point in Image 2. For the other points, an interpolation is made depending of the class of f. We de ne thus a criterion C:

A.3 Special edges for the template X As said in introduction, the template is only de ned as C(f) = kf(M ) , CP2(f(M ))k2 its shape. The template image is a single selected normal 21 image, characterized by good contrast and low densities on all non-cardiac features. The parameters described in Given a feature point in Image 1, the function f must give the previous paragraph are adapted in an ad-hoc manner a point as close as possible to a feature point in Image 2 to obtain an almost perfect set of points, with no gaps (this is what the function CP2 calculates). The criterion is i

Mi

S

i

DECLERCK ET AL.: AUTOMATIC REGISTRATION AND ALIGNMENT ON A TEMPLATE OF MYOCARDIAL PERFUSION STUDIES Original image

5

Anterior

Polar image Septum

Lateral

1 2 3

Inferior 4

Z

Original edges filling procedure artificial edges

endocardial edges

Sep. - Lat.

Inf. - Ant.

Lat. - Sep.

epicardial edges noise edges

1

After artificial edges removal and gradient threshoding noise edges

endocardial edges

2

epicardial edges

Image with edges

3

After ordering constraint

endocardial edges

4

epicardial edges

Fig. 10. Edges (in white) automatically extracted and ltered from 4 dierent stress images (one patient per row). On each row, we see central slices resampled from the 3D image by rotation Fig. 9. The edge extraction, the ltering is processed in the polar imaround the apico-basal axis (see the representation on top drawage from which a slice is displayed from top to bottom. Spurious ing). This makes easier the display of the myocardial structure edges are progressively removed. and avoids the problems of conventional displays. We thus see how the method behaves with hypoperfused areas (rows 2, 3 and 4).

the sum of all residual distances extended to S1 , a subset of the feature points in Image 1 for which the matching is 3. With the ltered list S1 of pairs of points, we calcuconsidered as being reliable. late F which is the best least squares t for the pairs of points. This new transformation is calculated within a B.2 Minimizing the criterion class of acceptable functions (rigid, then ane and nally The minimization process is iterative, given an initial local spline). transformation f0 . This initial transformation is chosen The iterative process stops when a maximum number of in our experiments to be a simple uniform scaling without iterations is reached, or when S1 = S1 ,1. [11] gives furrotation: assuming that the images to be registered are ther details about this adaptation of the iterative closest approximately aligned, it only adjusts the sizes of the ob- point algorithm, for instance about the convergence propjects to be registered (see section III-B.5). Each iteration erties. n splits into three steps: 1. For each point M , we calculate f ,1 (M ) and we iden- B.3 De nition of the closest point tify its closest feature point in Image 2. We therefore end For each point, the matching function CP2 takes its geup with a list of possible matched pairs of points. ometric position and the local direction of the intensity 2. We calculate the residual distance for each pair, and gradient into account. Considering 2 points M and N and we decide whether a pair is reliable or not: we rst elim- their intensity gradient vectors ,! n and ,! n respectively, inate pairs for which the residual distance exceeds a xed the distance between them is calculated as follows: threshold. Second, we compute the mean  and the standard deviation  attached to the remaining pairs. We then d(M; N)2 = k,,! MN k2 + :k,! n , ,! n k2 eliminate the points for which the distance is greater than another threshold depending on the distance distribution where  is a weighting coecient. (+c:, where c can be easily set using a 2 table [11]). We This double de nition of a point (location + direction) get for this iteration a list S1 of reliable pairs of matched re nes the matching criterion and makes it more robust for points. Notice that if a point is not matched in this itera- instance in the presence of a crude initialization (Fig. 11). tion, it may be matched in one that follows. ;n

n

;n

i

n

;n

i

M

N

M

;n

N

6

IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 16, NO. 6, DECEMBER 1997

This energy is expressed as a second order Tikhonov stabilizer:

M Q

P

S(fx) =

Z

IR3

2

2 fx @x2

4@

2

@ 2 fx

2

2

2

2 @ 2fx + + @ fx + @y 2 @z 2

@ 2 fx

2

@ 2 fx

2 @x@y + 2 @x@z + 2 @y@z Fig. 11. In dotted lines, the feature points of Image 2, in full lines, part of the feature points of Image 1. P is the closest point to M considering only geometric location, leading to a mismatch (endocardial point matched to an epicardial point). Q is the closest point to M taking the direction into account.

and the same for the other coordinates y and z. This smoothing energy is weighted in the global de nition of the least square criterion. A more complete description of the local spline transformation can be found in [16]. The transformation is de ned given a number of n :n :n parameters (= 512 in our experiments). As the rigid transformation is a particular ane transformation, the ane transformation itself is a particular local spline transformation, with a special distribution of control points. To match the stress to the rest image, the transformation is an ane, the rigid transformation is used as an initialization to the ane, applied serially with 10 iterations for each. In the case of the template to stress match, the transformation is a local spline, applied serially with 6 iterations after the rigid (10 iterations) and ane (10 iterations) transformations. B.5 The initial transformation The images are supposed to be reoriented in an approximately right position. The initial transformation for the ICP is a uniform scaling without rotation, the goal is to crudely adjust the positions and sizes of the lists of points. For that purpose, we de ne a \size" of the list of points as the medial distance of the points to the centroid of the list of points. Lists of points L1 and L2 are extracted from Image 1 and Image 2 from polar centers C1 and C2. The sizes t1 ou t2 are computed for L1 and L2 respectively. We de ne a transformation K which puts L2 in the geometry of L1 (Fig. 12) so that the registration is processed with ICP on L1 and L02 = K:L2 (the deformed L2 ): as L1 and L02 are comparable in sizes and position, they are more likely to match than L1 and L2 . Writing the scaling factor as k = tt1 , K is de ned as: 2 x

B.4 The class of transformations The least square t is calculated separately for three volume transformations: a rigid, de ned by 6 parameters, an ane de ned by 12 parameters and a local spline. For the local spline, the transformation f is a 3D tensor product of cubic B-splines. If (fx; fy; fz) are the coordinates of f: ,1 nX y ,1 nz ,1 X X

nx

fx(x; y; z) =

=0 j =0 k=0 nx ,1 ny ,1 nz ,1 X X X

fx B (x) B (y) B

z k;K

(z)

fy B (x) B (y) B

z k;K

(z)

fz B (x) B (y) B

z k;K

(z)

ijk

x i;K

y

j;K

i

fy(x; y; z) =

=0 j =0 k=0 nx ,1 ny ,1 nz ,1 X X X

ijk

x i;K

y

j;K

i

fz(x; y; z) =

=0

i

j

=0

k

=0

ijk

x i;K

y

j;K

where we use the following notation (for the x coordinate, for instance):  n : the number of control points in the x direction. This controls the accuracy of the approximation (8 in our experiments).  (fx ) : the 3D matrix of the control points abscissae. These are the parameters which de ne the transformation.  B : the ith B-spline basis function. Its degree is K. The B generate the vector space of piecewise K th degree polynomials (see [14], [15]). fx is then a piecewise K th degree polynomial in each variable x, y and z. We choose cubic B-splines in our experiments (K = 3), because of their regularity properties. For the knots, we used the classic regular mesh: t0 = ::: = t = min t = min + (max , min ) ni ,,KK for K < i < n t = ::: = t + K = max where min and max are the boundaries of the de nition domain (bounded by the border of the image or the organ). In the de nition of the criterion, a smoothing energy S is added in order to control the regularity of the solution. x

ij k

x i;K x i;K

x

x i

x K

x

x

x

x

x

x

x nx

nx

x

x

x

x

23 5





y

z

K = k:I0 3 C1 ,1k:C2 so that K:C2 = C1 and the size of L02 be t1. The ICP algorithm is processed on L1 and L02 , calculating a transformation f 0 . The matching transformation between L1 and L2 is therefore de ned as f = K ,1of 0 , with: 2

K ,1 = 64

1 1 k :I3 C2 , k :I3:C1 0 1

3 7 5

DECLERCK ET AL.: AUTOMATIC REGISTRATION AND ALIGNMENT ON A TEMPLATE OF MYOCARDIAL PERFUSION STUDIES t 1->2

Image 2’

7

Image 2

L2

L1

-1

f=K o f’

C1

C2 Q

P

t2

t1

K

f’

-1

L’2 C1 Q’ t1

A

D

B

C

D’

A’ B’

C’

Fig. 13. The resampling procedure is shown in 2D on this gure: the intensity of a pixel in Image 2' (the vertices of which are drawn as the square (ABCD)) is the integral of the intensity of Image 2 inside the area (A0 B 0 C 0 D0 ), which is (ABCD) deformed by t1!2 .

geometry to the rest, it is possible to resample the stress image in the geometry of the template, and by composition (t ! = t ! o t ! ), we resample the rest image in geometry of the template after registration on the stress. The resampling is a purely geometric transformation, the integral count density information is not preserved. T

R

S

R

T

S

Fig. 12. Lists L1 and L2 have \sizes" t1 et t2 et the centers used D. First steps to automatic 3D quanti cation of pathologies for the extraction are C1 and C2 respectively. K transforms L2 in L02 in order to put it in the geometry of L1 (size and center). ICP will run on L1 and L02 . D.1 2D normalisation and its limitations

Depending of the type of f 0 , f is calculated as follows:  f 0 is rigid or ane:   f 0 = A0 T1 so we have for f: 2

f = 64  f0

3

1 1 k :A C2 , k :(T , C1) 0 1

7 5

is spline : f0 =

X

PC 0 :B (x):B (y):B ijk

x i;K

y

z k;K

j;K

(z)

i;j;k

so f is also a tensor product of B-splines, the control points PC are de ned as: ijk

PC = k1 :PC 0 + C2 , k1 :C1 ij k

ijk

C. Resampling the rest and stress images

After the matching step, we get a transformation t1!2 from Image 1 to Image 2. With this transformation, it is possible to build an image Image 2' which is Image 2 resampled in the geometry of Image 1 (Fig. 13). Having computed the transformations t ! from the template geometry to the stress, and t ! from the stress T

S

R

S

The quanti cation of scintigraphic myocardial perfusion [17], [18] images is generally based on some form of a polar transformation. In this approach the myocardial densities are sampled from an origin in the center of the cavity, along (evenly spaced) radii. In one particular application [19] the three-dimensional images are sampled along rays or radii (radial sampling), originating in the center of the left ventricular cavity. The maximal intensity along the radii are reported on the 2D polar image (just as if the 3D polar image used for the segmentation was projected on the X,Y with maximal intensity projection. In this image, central points represent apical locations, peripheral points represent the basal locations. The anterior wall is mapped on top, the inferior wall at the bottom, the septum in the left-hand side of the image. This image is comparable to the bull's eye maps described in the literature [20], [21], [22], [23], [24], [25], [26], [27], except for the fact that the distance from the center represents an elevation in three dimensions, rather than a short axis plane position: the bull's eye map approach interrogates the volume as if it were a set of planes [28]. The eect of a polar transform amounts to a normalization of size and shape [29], since all morphological attributes of the image are reduced to angular coordinates [30]. This is also true for the polar component in bull's eye mapping, in which the third axis is divided in a set number of parallel "thick slices" between the apex and the base of the heart. This normalization allows the comparison between target cases and a population of normal cases, and the intercomparison of target cases.

8

IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 16, NO. 6, DECEMBER 1997

Fig. 14. Dierent VLA slices of patient Ridompa study, from the Fig. 15. Dierent transverse slices of Ryalmpa study, the display is anterior wall on top down to the inferior wall at the bottom. The the same as in Fig. 14. two left columns show slices of the rest and stress images before registration. The detected edge points are overlaid in white. The next two columns on the right show the rest and stress IV. Results and discussion images after registration and alignment. On the right column, the corresponding slices of the template. The edge points of the A. Matching and alignment template are overlaid on the images of the right three columns.

D.2 3D normalisation Having normalised the shape of the left ventricles, we de ne 3D images of mean and standard deviation of the intensity over a normal population. We also de ne 3D images of mean and standard deviation of the \rest { stress" dierence over the same population. As in [19], [31] for the 2D polar mapping, we de ne two normal inferior limit images as the mean image minus two standard deviations (this factor two depends on the number of images used to de ne the average population). In [19], [31], the comparison of the stress image to the inferior limit image shows abnormally hypoperfused areas at stress, the comparison of the rest { stress dierence image to the inferior limit dierence image shows abnormally redistributing areas.

Fig. 14 shows VLA slices of the stress and rest images of a normal study (Ridompa). From top to bottom, the anterior wall down to the inferior wall. The septum is on the left, the lateral wall is on the right. From left to right, columns 1 and 2 show the original images, the edges are overlaid in white. Columns 3 and 4 show the stress and rest images after matching and alignment on the template, displayed column 5. In the interpretation of the images in Fig. 14-16 one should remember that the edges do not necessarily correspond to iso-density lines. Fig. 15 is organised as Fig. 14, showing a pathological study (Ryalmpa): the left ventricle is aected by a large antero-lateral stress ischemia. In this gure, we can appreciate the robustness of the edge extraction procedure in the stress image with respect to hypoperfusion (curve #2 in Fig. 6). Despite the fact that the intensity in the ischemic area is low (but not as low as the background noise),

DECLERCK ET AL.: AUTOMATIC REGISTRATION AND ALIGNMENT ON A TEMPLATE OF MYOCARDIAL PERFUSION STUDIES

9

As far as an expert can appreciate the result of the matching, 80 % of the treated cases give satisfying results: in those cases, the feature points of the template deformed with the matching transformation de ne a plausible boundary for the left ventricle to an expert eye. For the remaining 20 % (which are dicult cases, according to an expert), some parts of the ventricle are not included in the volume or, on the contrary, some non-cardiac areas are included in the volume. It appears that these errors are due, rst, to some defects in the segmentation (the basis is not well de ned, as for patient Ledompa, Fig. 16, line 5) and second, because the local spline function does not deform suciently. However on the testing set of 250 cases (a routine data base) the success rate was 243/250 (97 %). The discrepancy is partially due to a larger prevalence of normal cases, and the fact that aberrant cases were not over represented. The average time of the whole treatment (edge extractions + matchings + resamplings) is roughly 1 minute for one study on a DEC Alpha 500 400 MHz workstation. We are currently working on some optimizations of the source code which could reduce the computational time by at least 50 %. B. A 3D display

As a corollary result, Fig. 17, top, shows a surface of the template shape. This surface has been obtained by deforming an isosurface of the template image to the edge points with our method. This closed surface can be a de nition of the volume of the left ventricle. Fig. 17 shows 3 dierent LVs at stress (Lojumpa, Ridompa and Ryalmpa) from two dierent views. These surfaces were obtained by deforming the surface of the template with the deformation t ! . Some slices of those images are displayed in Fig. 1416. With this method, it is possible to de ne a volume of the left ventricle for any SPECT heart image, which is a key-point in cardiac SPECT image quanti cation and analysis. On Fig. 17, the right column shows a regular 3D grid deformed by the alignment transformations of the 3 same LVs. The size of the grid is 10x10x10 and covers the whole image. The non-rigid deformation looks regular over the volume of the image. T

Fig. 16. Experiments conducted on the data base. Each row shows a dierent study (whose coded name is written on the left), the display is the same as in Fig. 14. The morphological transformation leaves density dierences unaected.

it is possible to detect the cardiac edges. Fig. 15 also illustrates the point that the morphological transformation leaves density dierences unaected. Fig. 16 is organised as Fig. 14 and 15, each line show a dierent study (one slice per image). On the slices, we can appreciate in the left two columns (the original rest and stress images) the dierences in sizes and shapes of the patient LVs. In the next two columns, the morphological dierences are removed whereas the perfusion information (the intensity level) is conserved. Of course, there is no point to point correspondence between slices of same number in the original images and in the realigned images: therefore, a pathological area may appear or disappear in this display because, after matching and resampling, it is located in other slices vicinous to the displayed slice. This artefact is illustrated by the Lojumpa study (Fig. 16, line 7), an apical hypoperfused area looks normally perfused in the aligned images.

S

C. 3D quanti cation of pathologies

For the `stress' images and `stress { rest' images populations, we compute both mean and standard deviation images. The size of the population is set to 10, the patients are chosen among the training data set. The target cases can thus be quantitatively compared to these references. Fig. 18 shows a 3D display of the largest ischemic area of the Ryalmpa study using this de nition. The 3D display shows a better representation than the 2D polar mapping [19], [31], [32], avoiding limitations inherent to the approach [30]. The quanti cation of pathologies using this procedure is currently under evaluation with the routine data base (250 cases).

10

IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 16, NO. 6, DECEMBER 1997 I

Template

S

L

Lojumpa

A basal view

Mcwampa

I

A

lateral view

L

S

inferior view

Fig. 18. Dierent views of ischemic area (clearer shape in transparency) in Ryalmpa study, shown in the normalised geometry.

Ridompa

Ryalmpa

(a)

(b)

(c)

Fig. 17. The closed surface of the template deformed to de ne the boundary of the left ventricle of 4 dierent hearts, (a) on a coronal projection and (b) on a frontal projection. The surfaces are represented at the same scale, which allow to appreciate the differences in sizes and shapes. (c), a regular grid is deformed by the alignment transformation.

V. Conclusion

We presented an automated method to register and align images from myocardial perfusion SPECT studies. To reconstruct \better" images, feature points are extracted in stress and rest images, the stress and rest points are matched together using an ane transformation (this gives the stress-rest registration) and the stress points are matched with a template using a local spline transforma-

tion (which gives the alignment). New images are resampled in the geometry of the template. The method has been tested and implemented on a data base of 32 stress-rest studies speci cally selected to obtain a sample of image variability and abnormalities. An additional set of 8 cases was used to check the parameters, giving a rst appreciation of the validation of the method. In as much as the testing population of 250 can be accepted as representative, and if there is any validity to the visual interpretation of the \goodness" of the match, the method seems indeed to be a reliable morphological normalization method. In the future, we will de ne a clinical validation protocol of the process which will include an experimentation of the method on a large cohort of patients. The method will be validated by demonstrating that after polar sampling, the standard deviation around the mean of the normal case stress polar maps is smaller when this registration method is used, than if the alignment is performed by an experienced operator or the alignment which was performed in routine clinical practice prior to the design of this study. In addition, the method will be validated if, while the normal values are clustered closer around the mean, the abnormal cases remain distinct from the normal cases (that is if the reduction of normal variation does not result in the reduction of the dierence between normal and abnormal). After having validated the method, in a second step, we will work on a more sophisticated statistical analysis of the quanti cation of SPECT myocardial images, inspired by the works of Houston et al. [33] or Strother et al. [34]

DECLERCK ET AL.: AUTOMATIC REGISTRATION AND ALIGNMENT ON A TEMPLATE OF MYOCARDIAL PERFUSION STUDIES

using Principal Components Analysis. VI. Acknowledgements

We thank Focus Imaging for their help in making possible the collaboration between the INRIA and the Stanford teams. The Focus team in Sophia is currently working on the industrialization of the method. We give also special thanks to Gregoire Malandain, to the teams of the Stanford University Hospital (California, USA) and of the Centre Antoine Lacassagne (Nice, France) for the rich and constructive discussions and comments about the project. This work was partially supported by regional grant of the Region Provence Alpes C^ote d'Azur (doctoral research contract). References [1] G. Germano, P.B. Kavanagh, and H.T. Su, \Automatic reorientation of three-dimensional transaxial myocardial perfusion SPECT images," Journal of Nucl. Med., vol. 36, pp. 1107{1114, 1995. [2] R. Mullick and N.F. Ezquerra, \3D visualization of pose determination: application to SPECT imaging," in Visualization in Biomedical Computing. Oct. 1992, vol. 1808, pp. 445{452, Chapel Hill, NC. [3] R. Mullick and N.F. Ezquerra, \Automatic determination of LV orientation from SPECT data," IEEE Transactions on Medical Imaging, vol. 14, no. 1, pp. 88{99, Mar. 1995. [4] J.C. Cauvin, J.Y. Boire, J.C. Maublant, J.M. Bonny, M. Zanca, and A. Veyre, \Automated detection of the left ventricular myocardium long axis and center in Thallium-201 single photon emission computed tomography," European Journal of Nucl. Med., vol. 19, pp. 1032{1037, 1992. [5] C. Perault, H. Wampach, and J.C. Liehn, \Three-dimensional SPECT myocardial rest-stress substraction images after automated registration and normalization," in Proceedings of Information Processing and Medical Imaging. June 1995, vol. 3, pp. 391{392, Kluwer Academic. [6] R.P. Woods, J.C. Mazziotta, and S.R. Cherry, \MRI-PET registration with automated algorithm," Journal of Computer Assisted Tomography, vol. 17, pp. 536{546, 1993. [7] A. Venot, J.C. Liehn, J.F. Lebruchec, and J.C. Roucayrol, \Automated comparison of scintigraphic images," Journal of Nucl. Med., vol. 27, pp. 1337{1342, 1987. [8] P.J. Slomka, G.A. Hurwitz, J. Stephenson, and T. Cradduck, \Automated alignment and sizing of myocardial stress and rest scans to three-dimensional normal templates using an image registration algorithm," Journal of Nucl. Med., vol. 36, pp. 1115{ 1122, 1995. [9] J.-P. Thirion, \Fast non-rigid matching of 3D medical images," Tech. Rep. RR 2547, INRIA, May 1995. [10] O. Monga, R. Deriche, and J.M. Rocchisani, \3D edge detection using recursive ltering: application to scanner images," Computer Vision, Graphics and Image Processing, vol. 53, no. 1, pp. 76{87, Jan. 1991. [11] J. Feldmar and N. Ayache, \Rigid, ane and locally ane registration of free-form surfaces," International Journal of Computer Vision, vol. 18, no. 2, pp. 99{119, 1996, (Also INRIA Research Report # 2220). [12] P. Besl and N. McKay, \A method for registrationof 3D shapes," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, pp. 239{256, Feb. 1992. [13] Z. Zhang, \Iterative point matching for registration of free-form curves and surfaces," International Journal of Computer Vision, vol. 13, no. 2, pp. 119{152, Dec. 1994, Also INRIA Research Report #1658. [14] J.-J. Risler, Methodes mathematiques pour la CAO, Masson, 1991. [15] G. Farin, Curves and Surfaces for Computer Aided Geometric Design, Academic Press, Inc., 1989. [16] J. Declerck, G. Subsol, J.-P. Thirion, and N. Ayache, \Automatic retrieval of anatomical structures in 3D medical im-

[17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29]

[30] [31] [32] [33] [34]

11

ages," in Computer Vision, Virtual Reality and Robotics in Medicine. Apr. 1995, vol. 905 of Lecture Notes in Computer Science, pp. 153{162, Springer-Verlag, (Also INRIA Research Report # 2485). J. Caldwell, D. Williams, and G. Harp, \Quantitation of size of relative myocardial perfusion defect by single-photon emission computed tomography," Circulation, vol. 70, pp. 1048{1056, 1985. E.E. Di Pasquale, A.C. Nody, and E.G. DePuey, \Quantitative rotational Thallium-201 tomography for identifying and localizing coronary artery disease," Circulation, vol. 77, no. 2, pp. 316{327, 1988. M.L. Goris, S. Boudier, and P.A. Briandet, \Two-dimensional mapping of three-dimensional SPECT data: a preliminary step to the quantitation of Thallium myocardial perfusion single photon emission tomography," Amer. Journal of Physiol. Imag., vol. 2, pp. 176{180, 1986. E. Garcia, J. Maddahi, D. Berman, and A. Waxman, \Space/time quantitation of Thallium-201 myocardial scintigraphy," Journal of Nucl. Med., vol. 22, pp. 309{317, 1981. E. Garcia, K.V. Van Train, and J. Maddahi, \Quanti cation of rotational Thallium-201 myocardial tomography," Journal of Nucl. Med., vol. 26, no. 1, pp. 17{26, 1985. J. Maddahi, K. Van Train, and F. Pringent, \Quantitative single photon emission computed Thallium-201 tomography for detection and localization of coronary artery disease: optimization and prospective validation of new technique," Journal of Am. Coll. Cardiol., vol. 14, pp. 1689{1699, 1989. J.J. Mahmarian, T.M. Boyce, and R.K. Goldberg, \Quantitative exercise Thallium-201 single photon emission computed tomography for the enhanced diagnosis of ischemic heart disease," Journal of Am. Coll. Cardiol., vol. 15, pp. 318{329, 1990. K.F. Van Train, E. Garcia, and J. Maddahi, \Improved quantitation of stress / redistribution Tl-201 scintigrams and evaluation of normal limits," in IEEE Transactions on Computers in Cardiology, 1982. K.F. Van Train, D.A. Berman, and E. Garcia, \Quantitative analysis of stress Thallium-201 myocardial scintigrams: a multicenter trial," Journal of Nucl. Med., vol. 27, pp. 17{25, 1986. K.F. Van Train, J. Meddahi, and D.A. Berman, \Quantitative analysis of tomographic stress Thallium-201 myocardial scintigrams: a multicenter trial," Journal of Nucl. Med., vol. 31, pp. 1168{1179, 1990. R.A. Vogel, \Quantitativeaspects of myocardial perfusion imaging," Seminars in Nuclear Medicine, vol. 10, pp. 146{155, 1980. M.L. Goris, S. Boudier, and P.A. Briandet, \Interrogation and display of single photon emission tomography data as inherently volume data," Amer. Journal of Physiol. Imag., vol. 1, pp. 168{ 180, 1986. M.L. Goris, J. Sue, and M.A. Johnson, \A principled approach to the "circumferential" method for Thallium myocardial perfusion scintigraphy quantitation," in Noninvasive assessment of the cardiovascular system: diagnostic principle and techniques. Edward P. Diethrich & John Wright, Eds. Littleton, MA: PSG, 1982, pp. 273-276. J. Declerck, J. Feldmar, M.L. Goris, and F. Betting, \Automatic registration and alignment on a template of cardiac stress & rest SPECT images," in Mathematical Methods in Biomedical Image Analysis, San Francisco, CA, June 1996, pp. 212{221. Also INRIA RR 2770, Feb. 1996. M.L. Goris, J. Bretille, S. Askienazy, G.P. Purcell, and V. Savelli, \The validation of diagnostic procedures on strati ed populations: application on thequanti cation of Thallium myocardial perfusion scintigraphy," Amer. Journal of Physiol. Imag., vol. 4, pp. 11{15, 1989. M.L. Goris and J. Bretille, Atlas en couleurs de cardiologie nucleaire, Medecine Sciences Flammarion, 1993. A.S. Houston, P.M. Kemp, and M.A. MacLeod, \A method for assessing the signi cance of abnormalities in HMPAO brain SPECT images," Journal of Nucl. Med., vol. 35, pp. 239{244, 1994. S.C. Strother, J.R. Anderson, K.A. Schaper, J.J. Sidtis, J.S. Liow, R.P. Woods, and D.A. Rottenberg, \Principal component analysis and the scaled subpro le model compared to intersubject averaging and statistical parametric mapping: I. "Functional connectivity" of the human motor system studied with O15 water PET," Journal of Cerebral Blood Flow and Metabolism, vol. 15, pp. 738{753, 1995.