Frequency Domain Registration of Computer Tomography Data Marco Andreetto, Guido M. Cortelazzo Department of Information Engineering University of Padua, Padua, Italy Phone: + 39-049-827-7642, Fax: +39-049-827-7826 E-mail: {marco.adreetto,corte}@dei.unipd.it
Luca Lucchese School of Electrical Engineering and Computer Science Oregon State University, Corvallis, OR 97331 Phone: 541-737-2980, Fax: 541-737-1300 E-mail:
[email protected]
Abstract This paper presents a new method for registering computer tomography (CT) volumetric data of human bone structures relative to observations made at different times. The system we advance was tested with different kinds of CT data sets. In this paper we report on some representative experimental results obtained with the CT data of the hip bones of a patient prior to and after prosthetic surgery aimed at the reconstruction of the hip articulation. The method works with rigid data having arbitrary relative position and orientation and proves to be robust with respect to CT acquisition noise and with respect to the segmentation technique adopted to select the region of interest for registration. The method is capable of registering correctly data sets taken from the same articulation and whose components have undergone small relative displacements. The method is also amenable to registration of heteregeneous kinds of volumetric data, e.g., CT scans and Magnetic Resonance Imagery (MRI) scans, which show different characteristics in correspondence to the same organic structure.
Keywords – 3-D Registration, Motion Compensation, 3-D Fourier Transform, Computer Tomography (CT), Iterative Closest Point (ICP) Algorithm.
1 Introduction Diagnostic tests based on volumetric data such as CT and MRI have become commonplace in hospitals and health care centers owing to their wealth of information, supplant-
ing, for instance, X-ray based tests where only 2-D information is available. The examination of a single series of CT slices provides the surgeon or doctor with important information on a given internal structure of the human body. Another important application consists in comparing CT measurements of a patient made at different times in order to monitor the temporal evolution of a physiological process or to check the success of a surgery operation. Sometimes, it may be necessary to integrate measurements obtained with different acquisition modalities, e.g., CT, MRI, or PET, in order to gain a better diagnostic picture [1]. In all these scenarios, two or more data sets describing the same anatomical part have to be spatially registered; in the case of measurements of a given object taken at different times, this operation is referred to as motion compensation. In the field of medical imaging, various methods have been advanced for registering volumetric data which are generally extensions of 2-D techniques. Duncan and Ayache [2] present a general survey on the algorithms proposed for medical imaging applications in the last few years whereas Maintz et al. [3] perform a thorough analysis of the registration methods for a variety of medical data including 2-D imagery, volumetric and surface data. Among the solutions representing extensions of 2-D techniques, Fei et al. [4] advance a multi-resolution algorithm for registering 2-D images obtained through interventional magnetic resonance imaging (iMRI) with respect to a volume acquired through MRI; their method estimates the correct registration by comparing two similarity figures usually adopted for images, mutual information and correlation, which are chosen based on the image resolution. Also in [5], the authors resort to the same figures applied to different types of images in order to obtain the most robust
Proceedings of the 2nd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT’04) 0-7695-2223-8/04 $ 20.00 IEEE
CT1
Isosurface Extraction
CT2
Segmentation
Segmentation
Point Cloud 1
Point Cloud 2 Frequency Registration RTi
Surface 1
Isosurface Extraction Surface 2
ICP
RTf Figure 1. Registration Scheme. registration. Hemmendorf et al. [6] suggest a motion compensation algorithm for data having arbitrary dimensionality (2-D, 3-D, and N-D); their method decomposes the input data in terms of a subband representation and finds the motion relating the two data sets through the phase information from the outputs of a filter bank. Other registration methods exploit the identification of convenient anatomical features in the data to register; e.g., Porter et al. [7] segment the data in order to determine the main blood vessels and align the two vessel networks via a generalized correlation coefficient obtained from the Schwartz inequality. This method, however, requires an initial registration which is provided by human supervision. Betke et. al [8] resort to different kinds of anatomical elements, such as vertebrae, trachea, and sternum, to determine a first coarse estimate of the registration of thorax CT scans taken at different times; this estimate is then refined by means of a variation of the ICP algorithm operating on the segmented surfaces of the lungs [9, 10, 11]. To determine the position of the anatomical elements, template matching techniques are utilized; therefore, this method cannot be used in applications where templates cannot be exploited. The two-level architecture of [8] is very similar to the method we propose with one important difference: our method does not rely on the determination of specific elements, which makes it more general. The registration technique advanced in this paper exploits the algorithmic framework proposed in [13]. The original contribution of this paper consists in adding additional stages and pre-processing
steps to make the algorithm of [13], which is tailored to surfaces, work with volumetric data as well and in assessing its overall performance. This paper has five sections. Section 2 describes the format of the CT data employed in our experiments and their preliminary processing before registration. The requirements for the correct operation of our method are discussed as well. Section 3 details the registration algorithm which consists of two steps: in the first, a preliminary alignment is sought after by exploiting the relationships in the frequency domain between the two data sets; in the second, the coarse estimate is refined through the ICP algorithm. Section 4 reports on some experimental results relative to both synthetic and actual medical data. Section 5 draws the conclusions and discusses possible extensions of our technique.
2 Preliminary Data Processing Our algorithm performs the registration of two 3-D data sets which may have been recorded at different times. The alignment is carried out by using all the information encoded in the CT scans without any external fiducial markers. The only assumption made is that the anatomical structures to register are related by a global 3-D rigid transformation. This constraint can also be relaxed as will be shown in Section 3. In all our experiments, only CT scans relative to the same patient (intrasubject registration) have been registered (monomodal registration). However, the system can
Proceedings of the 2nd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT’04) 0-7695-2223-8/04 $ 20.00 IEEE
5
8
x 10
7
6
5
4
3
2
1
0 −500
0
500
1000
1500
2000
2500
3000
Figure 3. Intensity histogram of a CT scan. Figure 2. An example of CT scan. easily be modified to register different kinds of data, e.g., CT and MRI data (multimodal registration). Furthermore, even though the application scope of the experiments presented in this contribution is hip articulation, our method has also been successfully used to register other anatomical structures where different data sets are reasonably related by rigid transformations. In order to be aligned with our frequency domain algorithm, the CT data sets have to be segmented first. Figure 1 shows the block diagram of our registration system, including the preliminary segmentation stage and the surface computation stage for the final application of the ICP algorithm. The data we have used in our experiments are the time series of two CT scans relative to two patients who have undergone prosthetic surgery in the hip-femur socket. For both patients, CT scans have been acquired before and after surgery in order to assess the correct position of the prosthesis. Every CT scan consists of a series of rectangular slices spatially aligned along the X and Y coordinates and taken at different values of the Z coordinate with an adaptive strategy: the spacing along Z increases where the volume has more articulation. In turn, every slice consists of a regular sampling lattice with even spacing along the X and Y coordinates (the sampling interval is the same for both dimensions) and the scalar value at each pixel represents the response to radiation of the tissue under investigation (this value will be referred to as the CT intensity). An example of CT slice is shown in Figure 2. For an in-depth description of this data structure, the reader is referred to the RECTILINEAR GRID data structure described in [12]. It is important to observe that CT data sets relative to the
same patient have different geometric characteristics (position, lattice spacing, and dimensions) whereas the intensity histograms are very similar, since they stem from the same acquisition system and protocol. This property is important for the correct operation of our registration system. In fact, if the intensity distributions of the CT data sets are different, they have to be appropriately equalized before processing. As an alternative, only geometric information can be used for registration provided that it is possible to segment the same anatomical structure in different data sets. From all the information associated with a CT scan, only spatial location within the lattice and intensity are considered, whereas topological information on the lattice is disregarded. The points obtained in this way are segmented through thresholding which filters out all the values outside a preselected interval. The goal of this filtering is to keep only the data relative to bone structures that are characterized by high intensity and that can be assumed to be rather constant during acquisition. The threshold values can be chosen interactively by the user or can be computed without supervision by analyzing the CT intensity histograms. Figure 3 displays a typical histogram which shows three modes: one, at low intensities and very close to zero, can be ascribed to acquisition noise. The other two peaks are far away from the first and rather close to each other; they correspond to low-density tissues such as muscles and to high-density tissues such as calcified bones. Once the CT data sets have been segmented, the baricenter of each one is computed and a 3-D orthogonal Cartesian reference system is attached to it. It will be shown in Section 3 that this provision is beneficial to improve the overall performance of the system.
Proceedings of the 2nd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT’04) 0-7695-2223-8/04 $ 20.00 IEEE
(a)
(b)
Figure 4. (a) A point set; (b) Same data resampled on a regular domain.
3 Registration Algorithm The registration algorithm is comprised of two steps (see diagram in Figure 1). The first step computes a coarse estimate of the rigid transformation relating the pre-processed data sets. The second step performs the fine alignment of the iso-surfaces describing the bone structures.
3.1 Coarse Estimation of the 3-D Rigid Transformation The 3-D rigid motion relating the two data sets is computed with a variation of the frequency domain algorithm advanced by Lucchese et al. in [13] which operates on surfaces rather than on volumetric data. The data sets are resampled on a regular 3-D lattice in order to apply 3-D Fast Fourier Transform (3-D FFF) algorithms. To this end, the bounding box enclosing both data sets is computed. This is then divided into 128 × 128 × 128 voxels, each of which is assigned an intensity value through a nearest neighbour strategy: each voxel is given the intensity value of the closest point to its center. The result of this operation is shown in Figure 4. Let d1 (x) and d2 (x), x ∈ R3 , be two data sets after this pre-processing stage. We assume that the two sets are
related by a 3-D rigid transformation, i.e., d2 (x) = d1 (R−1 x − t),
(1)
where R ∈ S O(3) is a 3-D rotation matrix and t ∈ R3 is a 3-D displacement. Matrix R is expressed as a single rotation ψ about an axis having the direc. tion defined by the unit vector ω = [ωx ωy ωz ]T = 3 [cos ϕ sin ϑ sin ϕ sin ϑ cos ϑ] ∈ R , ω = 1, i.e., R = eΩψ , where 0 −ωz ωy 0 −ωx (2) Ω = Ω(ω) = ωz −ωy ωx 0 1
is a skew-symmetric matrix associated with ω. This representation is not unique since choosing ω = −ω and ψ = 2π − ψ gives the same rotation as ω and ψ [19]. The 3-D rotation matrix is thus described by the three angles ψ, ϑ, and ϕ. By applying the 3-D Fourier transform to Eq. (1), it is D2 (k) = D1 (R−1 k)e−j2πk
T
Rt
,
(3)
where D1 (k) and D2 (k), k ∈ R3 , are the Fourier transforms of d1 (x) and d2 (x), respectively. By keeping only 1 S O(3)
denotes the group of the 3 × 3 special orthogonal matrices.
Proceedings of the 2nd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT’04) 0-7695-2223-8/04 $ 20.00 IEEE
(a)
(b)
(c)
(d)
Figure 5. (a)-(b) Centered point sets. (c)-(d) Results of the frequency domain registration algorithm of [13].
Proceedings of the 2nd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT’04) 0-7695-2223-8/04 $ 20.00 IEEE
(a)
and (b), respectively, show a regular series of CT slices as acquired in a practical scenario and its perturbed counterpart. Since our method works with volumes, the perturbation of the z coordinate transforms a series of 2-D planes containing the data into a 3-D volume. One may then compute the 3-D FFT of these volumes and apply the algorithm of [13]. This yields a first coarse estimate of the 3-D rotation matrix which allows the alignment of the CT data sets. Figures 5 (a) and 5 (b) show the two CT data sets centered at their baricenters whereas Figures 5 (c) and 5 (d) show the results of their registration with the frequency domain algorithm.
3.2 Fine Estimation of the 3-D Rigid Transformation
(b)
The estimation of the rigid transformation obtained in the frequency domain usually is not accurate enough to be used for registration. It is therefore necessary to resort to a refinement phase. Unlike the first, which operates on volumes, the second works with surfaces (see [14]) through the ICP algorithm [9, 10]. The two surfaces are obtained with the isosurface extraction algorithms implemented in the scientific visualization library VTK [12, 15]. Rather than using a single algorithm, the ICP method determines a class of algorithms which differ in the strategy used to find the solution to various subproblems composing the ICP (computation of the closest point, value to estimate, caching, etc.). Among the various implementations available, we have used the version advanced by Pulli [16] for registering multiple 3-D views. Other implementative choices can be made as well [17, 18].
Figure 6. (a) Regular CT slices. (b) Perturbation of the z coordinate of the slices.
the magnitude information of the spectra, Eq. (3) becomes |D2 (k)| = |D1 (R−1 k)|.
(4)
From Eq. (4), the algorithm of [13] builds an auxiliary function −1 . |D1 (k)| |D2 (k)| |D1 (k)| |D1 (R k)| ∆(k) = − = − , D1 (0) D2 (0) D1 (0) D1 (0) (5) which allows the estimation of Ω in Eq. (2) and ψ in two steps. From these values the rotation matrix can easily be estimated and, in a third step, the translational displacement t is eventually computed through phase correlation. For the algorithmic details, the reader is referred to [13]. The algorithm of [13] requires a 3-D lowpass filtering operation for removing the high frequency components which are typical of the Fourier transforms of surfaces. In our case, however, this kind of filtering is not necessary since the data are already volumes. The only provision taken consists in perturbing the z coordinate of the slices making up a given CT scan. As an example, Figures 6 (a)
4 Experimental Results In order to assess the performance of our algorithm, we have carried out several series of simulations where various data sets have been used. In each experiment, a known rigid motion has been applied to each point of the selected data set whose geometric position has been corrupted by white Gaussian noise to simulate a real acquisition scenario. In each experiment a segmented cloud of points is considered. A 3-D rotation is applied to it using as a center of rotation the center of mass of the point set. The three rotation angles ψ, ϕ, and ϑ take on values between 0◦ and 90◦ , for an overall number of 343 rotations. The set of points obtained with these imposed rotations is then perturbed by adding zero-mean white Gaussian noise to their 3-D Cartesian coordinates. The relative rotation between the two data sets is then estimated with our algorithm. For each rotation, the estimation error has been assessed in two ways: 1) by providing the average Euclidean distance between the original position of each point and the corresponding point after
Proceedings of the 2nd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT’04) 0-7695-2223-8/04 $ 20.00 IEEE
File fem post1.vtk fem post2.vtk fem pre1.vtk fem pre2.vtk hip post1.vtk hip post2.vtk hip pre1.vtk hip pre2.vtk
mean 6.016 7.691 4.565 5.562 0.572 0.635 0.547 0.491
ed std 7.047 9.398 5.367 10.355 0.400 0.445 0.397 0.317
median 3.297 4.337 2.822 3.949 0.588 0.641 0.606 0.518
mean 3.916 5.194 3.119 6.373 0.957 1.071 0.918 0.851
ψε std 4.858 6.385 3.795 9.53 0.670 0.740 0.662 0.544
median 2.007 2.430 1.677 2.125 1.042 1.088 1.034 0.905
Table 1. Simulation results. ed is measured in millimeters, ψ in degrees. compensation for rotation. Let p1 denote the location of a point of the original set and let p2 denote the corresponding location after applying a 3-D rotation matrix Ro . The two data sets then relate as p2 = Ro p1 + ε,
(6)
where ε the additive noise. From our algorithm we obtain ˆ o which leads to an estimated location an estimate R ˆ o p1 p3 = R
(7)
in lieu of p2 . If the set is comprised of N points, a first error measure can be obtained as ed =
1 N Σ p2i − p3i . N i=1
(8)
Another error measure we have considered is ψ = arccos (0.5(trace(R ) − 1))
(9)
. ˆ T from [13]. For all 343 simulations where R = Ro R o both error measures have been computed; Table 1 shows the mean value, the standard deviation, and the median for both figures relative to eight data sets. The labels of the files containing these data are displayed in the first column of the table. The method has also been tested in two real cases (see Figure 5). In both instances, the rigidity assumption is not strictly met. In fact, even though the shape of the bones analyzed is the same (to the exception of the prosthesis), their relative position is different since the patient cannot be placed in the same position during the two examinations.
5 Conclusions We have presented a new method for registering CT volumetric data of bone structures acquired at different times. The system we propose was tested with several CT data sets. In this paper, we have discussed a representative example relative to the CT data of the hip bones of a patient prior to
and after prosthetic surgery aimed at the reconstruction of the hip articulation. The method works with rigid data having arbitrary relative position and orientation and proves to be robust with respect to CT acquisition noise and to the segmentation technique employed to select the region of interest for registration. The method also provides a valuable support in the registration of data sets relative to the same articulation and whose components have undergone small displacements. The method is also amenable to registration of heteregeneous kinds of volumetric data, such as CT scans and Magnetic Resonance Imagery (MRI) scans, which show different characteristics in correspondence to the same organic structure.
6 Acknowledgments We would like to acknowledge Cinzia Zanoni and Marco Petrone of CINECA (Bologna, Italy) and Marco Viceconti and Riccardo Lattanzi of “Laboratorio Tecnologia dei Materiali” from Istituto Ortopedico Rizzoli (Bologna, Italy) for kindly supplying the CT data. The research was supported in part by the University of Padova under Progetto Ateneo CPDA 024157/2002 and in part by FIRB-PRIMO.
References [1] D. Reuckert, L. I. Sonoda, C. Hayes, D. L. G. Hill, M. O. Leach, and D. J. Hawkes, “Nonrigid Registration Using Free-Form Deformations: Application to Breast MR Images,” IEEE Transactions on Medical Imaging, Vol. 18, No. 8, Aug. 1999, pp. 712-721. [2] J. S. Duncan and N. Ayache, “Medical Image Analysis: Progress Over Two Decades and the Challenges Ahead,” IEEE Transactions on Medical Imaging, Vol. 33, No. 1, Jan. 2000, pp. 85-106. [3] J. B. A. Maintz and M. A. Viergerver, “A Survey of Medical Image Registration,” Medical Image Analysis, Vol. 2, No. 1, Mar. 1998, pp. 1-36.
Proceedings of the 2nd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT’04) 0-7695-2223-8/04 $ 20.00 IEEE
[4] B. Fei, J. L. Duerk, D. T. Boll, J. S. Lewis, and D. L. Wilson, “Slice-to-Volume Registration and Its Potential Application to Intervation MRI-Guided RadioFrequency Thermal Ablation of Prostate Cancer,” IEEE Transactions on Medical Imaging, Vol. 22, No. 4, Apr. 2003, pp. 515-525. [5] L. G. Nyul and J. K. Udupa and P. K. Saha, “Incorporating a Measure of Local Scale in Voxel-Based 3-D Image Registration,” IEEE Transactions on Medical Imaging, Vol. 22, No. 2, Feb. 2003, pp. 24-26. [6] M. Hemmendorf, M. T. Andersson, T. Kronander, and H. Knutssuon, “Phase-Based Multimensional Volume Registration,” IEEE Transactions on Medical Imaging, Vol. 21, No. 12, Dec. 2002, pp. 1536-1543. [7] B. C. Porter, D. J. Rubens, J. G. Strang, J. Smith, S. Totterman, and K. J. Parker, “Three-Dimensional Registration and Fusion of Ultrasound and MRI Using Major Vessels as Fiducial Markers,” IEEE Transactions on Medical Imaging, Vol. 20, No. 4, Apr. 2001, pp. 354-359.
[15] W. J. Schroeder, K. Martin, and B. Lorensen, The Visualization Toolkit: An Object Oriented Approach to 3D Graphics 3rd Edition, Kitware, New York, 2003. [16] K. Pulli, “Multiview Registration for Large Data Sets,” Proc. of the Second Intl. Conf. on 3-D Digital Imaging and Modeling, Ottawa, Canada, Oct. 1999, pp. 113-120. [17] T. Jost and H. H¨ugli, “A Multi-Resolution Scheme ICP Algorithm for Fast Shape Registration,” Proc. of 1st International Symposium on 3D Data Processing Visualization and Transmission (3DPVT2002), Padova, Italy, 2002, pp. 540-543. [18] C. Kapoutsis, C. Vavoulidis, and I. Pitas, “Morphological Iterative Closest Point Algorithm,” IEEE Transactions on Image Processing, Vol. 8, No. 11, Nov. 1999, pp. 1644-1646. [19] R. Murray, Z. Li, and S. Sastry, “A Mathematical Introduction to Robotic Manipulation,” CRC Press, Boca Raton, FL, 1994.
[8] M. Betke, H. Hong, and J. P. Ko, “Automatic 3D Registration of Lung Surfaces in Computed Tomography Scans,” Computer Science Department, Boston University, Apr. 2001, http://www.cs.bu.edu/techreports /pdf/2001-004-betke-hong-ko.pdf. [9] P. J. Besl and N. D. McKay, “A Method for Registration of 3D Shapes,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 14, No. 2, Nov. 1992, pp. 239-259. [10] Z. Zhang, “Iterative Point Matching for Registration of Free-Form Curves and Surfaces,” International Journal of Computer Vision, Vol. 13, No. 2, 1994, pp. 119-152. [11] S. Rusinkiewicz and M. Levoy, “Efficient Variants of the ICP Algorithm,” Proceedings of the Third International Conference on 3D Digital Imaging and Modeling, Quebec City, Canada, May 2001, pp. 145-152. [12] W. J. Schroeder, The VTK User’s Guide: Updated for Version 4.0, Kitware, New York, 1998. [13] L. Lucchese, G. Doretto, and G. M. Cortelazzo, “A Frequency Domain Technique for 3-D View Registration,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 24, No. 11, Nov. 2002, pp. 1468-1484. [14] M. A. Audette, F. P. Ferrie, and T. M. Peters, “An Algorithmic Overview of Surface Registration Techniques for Medical Imaging,” Medical Image Analysis, Vol. 4, No. 3, Mar. 2000, pp. 201-217.
Proceedings of the 2nd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT’04) 0-7695-2223-8/04 $ 20.00 IEEE
