152
IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 54, NO. 1, FEBRUARY 2007
Multiresolution Elastic Registration of X-Ray Angiography Images Using Thin-Plate Spline Jian Yang, Yongtian Wang, Songyuan Tang, Shoujun Zhou, Yue Liu, and Wufan Chen
Abstract—X-ray angiography, a powerful technique for the visualization of blood vessels, has been widely used in clinical practice. However, due to unavoidable motion of patient, the subtraction images often suffer from misregistration artifacts. In order to improve the quality of subtraction images, registration algorithms are often employed before direct subtraction of mask and live images. A novel multiresolution elastic registration algorithm is proposed for the registration of the digital angiographic images using thin-plate spline (TPS). Our main contribution is a multiresolution search strategy specifically designed for the template matching method. In this strategy, the mask image is decomposed to coarse and fine sub-image blocks iteratively using the pyramid approach. Experimental results show that the multiresolution refinement strategy is well adapted to the template matching method, and can achieve better performance than comparable single step algorithms, because local minima can be overcome by the gradual coarse-to-fine approach that also ensures convergence. Registration results of four typical similarity measures, namely energy of histogram of differences (EHD), mutual information (MI), correlation and sum of squared differences (SSD), are compared. Three different interpolation methods, including nearest-neighbor, bilinear and bicubic, are also tested and compared. The overall conclusion is that the multiresolution refinement algorithm based on EHD combined with the bicubic interpolation method is very robust and effective for the registration of X-ray angiography images, which can obtain sub-pixel registration accuracy and is fully automatic. In addition, the objective measurement method developed in this paper on simulated data makes it possible to quantitatively evaluate the quality of the elastic registration results. Index Terms—Energy of histogram of differences, multiresolution, thin-plate spline, X-ray angiography.
I. INTRODUCTION
X
-RAY angiography is a powerful technique for the visualization of blood vessels. Clear images of abdominal, renal, cerebral and peripheral blood vessels can be obtained by the highly sophisticated equipments. It represents the most common image modality applied in clinical practice to assist vessel diag-
Manuscript received November 15, 2005; revised August 21, 2006. This work was supported by the National Basic Research Program of China (Project No. 2003CB716105). J. Yang, Y. Wang, S. Tang, S. Zhou, and Y. Liu are with the School of Information Science and Technology, Beijing Institute of Technology, Beijing 100081, China (e-mail:
[email protected];
[email protected];
[email protected];
[email protected];
[email protected]). W. Chen is with the Key Lab for Medical Image Processing, Southern Medical University, Guangzhou 510515, China (e-mail:
[email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TNS.2006.889161
nosis and therapy [1]. With this technique, a bolus of radiopaque contrast material is injected into the blood vessels of interest through a plastic catheter, and a sequence of X-ray projection images is acquired during (live image) and prior (mask image) to the passage of the material in the blood vessels. By subtracting the mask image from the live image, background structures in the live images are largely removed. However, mask and live images are acquired at different time, and the positions of the background tissues around the vessels often change during the elapsed time due to the unavoidable respiration and organ movement of the patient. If these images were subtracted directly, motion artifacts would occur. Such artifacts may hamper proper interpretation of the images and even lead to misdiagnosis. In order to reduce artifacts, several techniques were developed, such as Time Interval Difference (TID) subtraction [2], energy subtraction [3], automatic remasking [4], hybrid subtraction [5] and so on. However, because artifacts are the combination of several factors, more complicated techniques are required in order to eliminate the artifacts completely. Image registration is a useful approach to correct artifacts. In this technique, correspondences between pixels in mask and live images are calculated. Then a certain warping method is applied to the mask image. After subtracting the two images, artifacts can be greatly eliminated [6], [7]. During the past twenty years, enormous efforts have been spent on this subject and great progresses have been achieved. An overview of the registration techniques for X-ray angiography images was made by Meijering [8] in 1999. However the non-rigid motion of the tissue inside human body is complicated and often there are global and local disparities between the mean gray-levels of the live and mask images. Registration of such kind of angiographic images is very difficult. A simple shift or rotation of the misregistered image cannot eliminate those artifacts. Until now none of techniques developed can deal with all kinds of artifacts. The only widely used method for motion correction on commercial X-ray angiography devices is manual pixel shifting [9], which is very time-consuming and tedious. A novel registration algorithm is proposed for coronary angiographic images, which can be outlined as follows. Firstly, the mask image is decomposed of many sub-image blocks. With a template matching method based on typical similarity measures, the rigid matching between mask sub-image blocks and the live image can be achieved. Secondly, the control point pairs are selected from the center points and four valid corner points, which are extracted in the original and transformed mask images. By using the thin-plate spline as the mapping function, a global elastic registration is accomplished. With the five selected control points of each sub-image blocks, the information of transla-
0018-9499/$25.00 © 2007 IEEE
YANG et al.: MULTIRESOLUTION ELASTIC REGISTRATION OF X-RAY ANGIOGRAPHY IMAGES USING THIN-PLATE SPLINE
Fig. 1. Outline of registration algorithm (Using EHD as example).
tion and rotation of the affine transformation can be utilized effectively. Thirdly, in order to obtain the best matching results, a multiresolution approach is proposed. In this strategy, the mask image is decomposed to coarse and fine sub-image blocks iteratively using the pyramid approach. Moreover, results of different registration algorithms (single step, regular grid, irregular triangular mesh and multiresolution), different similarity measures (EHD, MI, correlation and SSD), different interpolation methods (nearest, bilinear and bicubic interpolation) are experimented and compared. Experimental results show that the multiresolution refinement algorithm based on energy of histogram of differences combined with the bicubic interpolation method can be successfully applied to the registration of X-ray angiography images.
153
Fig. 2. Evaluation datasets. (a) Original live image. (b) Simulated contrast image. It is warped from the original live image with 20 random control point pairs using thin-plate spline. (c) Image difference of (a) and (b). (d) Example of control points selection. a ; b ; c and d are four random control points generated from a ; b ; c and d respectively. The motion boundaries of (a ; a ) and (b ; b ) overlap to each other which can easily change the topology of the image. On the contrary, (c ; c ) and (d ; d ) are two valid control point pairs for their motion boundaries are separated from each other.
The rigid registration algorithm of mask sub-image blocks and and live image can be defined as follows. Let represent the mask sub-image blocks and live image respectively with coordinates , where is the index number of mask image block. The task of the registration of the images is to find a best geometric transformation , which of and reach make the similarity measure its maximum or minimum, where is a set of transform parameters. Thus the search algorithm can be summarized as (taking EHD as an example): (1)
II. REGISTRATION ALGORITHM A. Rigid Registration Method of Sub-Image Blocks Generally two classes of registration algorithms are involved: intensity-based algorithms and feature-based algorithms [10]. If salient features in the images can be extracted easily, and the number of common elements of the detected sets of features is sufficient, feature-based algorithm is usually adopted. However, for cardiac X-ray angiography images, pixel warping between the mask and live images is especially large, in some cases it can be as large as 80 pixels in a 1024 1024 image. It is very hard to detect pixel sets with the same properties in the two images. Therefore, the intensity-based algorithm is applied for the registration of sub-image blocks in this paper.
As a preliminary study, the search space is restricted to 2-D , and this will be extended to rotation and translations affine transformations. Then the transformation model can be defined as:
(2) In order to find out the optimum transformation, the registration process is composed of the following three important aspects: (1) similarity measures, (2) interpolation problem, (3) optimization method.
154
IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 54, NO. 1, FEBRUARY 2007
Fig. 3. Comparison of multiresolution algorithm and single step algorithm over varying numbers of calculation steps using bicubic interpolation method (EHD as the similarity measure). (a) RMS pixel error curves. (b) EHD curves of original live image and simulated contrast image.
Fig. 4. Registration results of single step and multiresolution algorithms using EHD as similarity metrics ( = 5:538). The first and second rows correspond to single step and multiresolution algorithms respectively. Columns from left to right correspond to decomposition level numbers 4, 8, 12, and 16 separately.
Similarity measures for image registration have been studied extensively, and many objective criterions have been developed during the past two decades [11]. The most commonly used similarity measures based on intensity include: EHD, MI, correlation and SSD. In this paper, all the four measures will be introduced generally, and how EHD can be successfully applied to the registration of X-ray angiography images will be shown. When the original mask sub-image blocks are mapped to the other modality based on the geometrical transformation parameters, the interpolation issue occurs because the calculated coare not integers. According to the amount of ordinates image data used as the support, the transformation interpolation can be nearest-neighbor, bilinear and bicubic etc. [12]. The ultimate registration accuracy is directly decided by the interpolation models. In the latter experiments the registration results using different interpolation methods are also compared.
Optimization method is the key point in the registration of images, which is a kind of search strategy to find the minimum of dissimilarity measurements (penalty) or the maximum of similarity measurements. There are many optimization methods in the registration of medical images, and Maintz gave an elaborate summarization in 1998 [13]. In this paper, Powell method is used as the optimization method because it does not require the calculation of the differential of parameters and it converges rapidly [14]. Although Powell method is sensitive to the initial values of the iteration and likely to be trapped into local minimum, these shortcomings can be compensated by the proposed multiresolution search strategy. B. Brief Introduction to Similarity Measures If you want to submit your file with one column electroniand denote cally, please do the following: In what follows,
YANG et al.: MULTIRESOLUTION ELASTIC REGISTRATION OF X-RAY ANGIOGRAPHY IMAGES USING THIN-PLATE SPLINE
155
Fig. 5. Multiresolution registration results for different similarity measures (EHD, MI, correlation and SSD) over varying numbers of decomposition levels, where bicubic interpolation method is used. (a) RMS pixel error curves. (b) Normalized similarity measure curves of original live image and simulated contrast image.
Fig. 6. Registration results using different similarity measures ( = 5:538). Columns from left to right correspond to multiresolution algorithm based on EHD, MI, Correlation and SSD. Rows from top to bottom correspond to decomposition level number 4, 8, 12, and 16 respectively.
156
IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 54, NO. 1, FEBRUARY 2007
Fig. 7. Multiresolution registration results for different interpolation methods (nearest, bilinear and bicubic interpolation methods) over varying numbers of decomposition levels. (a) RMS pixel error curves. (b) EHD curves of original live image and simulated contrast image.
the mask and live images to be registered. and are the pixel values of the mask and live images at the point respectively. Energy of the histogram of differences, proposed by Buzug et al. [15], [16], was regarded as the most adequate measure for registration in DSA [8]. It is defined as
[17] et al. A detailed description of the technique was given by Pluim et al. in 2003 [18]. It is a kind of statistical dependence between two random variables, i.e., the information that one variable carries about the other. Under Kullback-Leibler distance can be defined as [10], mutual information
(6) (3) where is the normalized histogram of the differences between the mask and live images, which can be written as
(4) where
is any gray value difference and is the Kronecher delta function
(5) Mutual information, a basic concept of information theory, was first used to register medical images in 1997 by F. Maes
where
and denote the marginal distributions, and indicates joint distributions between the two images. These parameters can be written as:
(7) (8) (9) where and are the intensity values of a pair of corresponding and can be obblocks of the two images. tained by the normalization of the marginal and joint intensity histograms of both images. Correlation is a classical representative of the intensity-based methods in image processing [19], which can be defined as: and are the mathematical expectation of where mask and live image values.
(10)
YANG et al.: MULTIRESOLUTION ELASTIC REGISTRATION OF X-RAY ANGIOGRAPHY IMAGES USING THIN-PLATE SPLINE
157
TABLE I REGISTRATION RESULTS USING DIFFERENT INTERPOLATION METHODS AT DIFFERENT ITERATIONS
Sum of squared differences [20] is the simplest measurement of the difference of two images, which can be defined as:
respectively. Then the energy function of thin-plate spline can be defined as:
(11)
where represents the total pixel number of the calculating block of the images. When the two images are geometrically aligned, the SSD value reaches its minimum while EHD, MI and correlation reach their maxima. So in the registration strategy the maximizing of EHD, MI and correlation measures corresponds to the minimizing of the SSD measure. C. Thin-Plate Spline Algorithm The use of thin-plate spline interpolation as a point-based elastic registration algorithm for medical images was first proposed by Bookstein [21] in 1989. One of the most important attributes of thin-plate spline is its ability to decompose a space transformation into a global affine transformation and a local non-affine warping component [22]. Under the restriction of corresponding points, matching matrixes and mapping parameters can all be achieved. Assuming that there are two sets of corresponding points and , which are represented as and
(12) where is the mapping function between the point sets and . and are the homogeneous coordinates of and respectively. The first term in the above equation is the approaching probability between point sets and . The second term, on the other hand, is a smoothness constraint. Different means different degree of warping. When is close to zero, corresponding points are matched exactly. According to this energy function, there exists a minimizing for any fixed , which can be represented function as: (13) where is the calculated point sets, is a 3 3 affine transfornon-affine warping coefficient mamation matrix, is a is a vector decided by thin-plate spline kernel. trix, , which can be defined For each point of , there exists a
158
IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 54, NO. 1, FEBRUARY 2007
Fig. 8. Multiresolution registration results using different interpolation methods based on EHD ( = 5:538). Columns from the left the right correspond to nearest, bilinear and bicubic interpolation method. Rows from top to bottom correspond to decomposition level 4, 8, 12, and 16 respectively.
as: , where is a constant. When the solution of (13) is substituted into (12), we have: (14) where and are concatenated point sets of and is an . Then, QR decomposition n n matrix formed from the is used to separate the affine and non-affine warping space. (15) where and are ortho-normal matrices, is upper triangular. The final solution for and can be written as (16) (17)
With the rigid registration of all the sub-image blocks, a set of corresponding points is obtained, which is extracted from the original and transformed mask images. Then the mask image is warped to the other modality by thin-plate spline based on these point pairs, which has the maximization of similarity or minimization of non-similarity with the live image. Thus the global elastic registration is achieved. The results of elastic registration are mostly decided by the numbers of control points and the positions of these points in the image. The transformation parameters and computational time will increase with the number of control points. Also, if the control points are not selected properly, iterative search results will be invalid. In this paper, the center and four valid corner points extracted in the original and transformed mask images are selected as control points. When sub-image blocks, the mask image is decomposed into control points are employed. Using this control point selection method, the information of translation and rotation of affine transformation can be utilized effectively.
YANG et al.: MULTIRESOLUTION ELASTIC REGISTRATION OF X-RAY ANGIOGRAPHY IMAGES USING THIN-PLATE SPLINE
159
Fig. 9. Simulated subtraction images and the corresponding subtraction images after registration by the developed multiresolution algorithm based on EHD, where the simulated contrast images are generated from 20 random control point pairs with different RMS errors. The first and the third rows correspond to the simulated subtraction images. The second and the fourth rows correspond to subtraction images after registration.
D. Multiresolution Search Technique
Generally, there are global and local disparities between the mean gray-levels of the live and mask images, especially for coronary angiographic images. Artifacts cannot be eliminated effectively when a simple single step algorithm (common template matching algorithm) is used. It is almost impossible to determine the best decomposition block size suitable for all angiographic images with different warping scales. In order to improve the registration efficiency, a coarse-to-fine multiresolution search strategy is developed. The proposed algorithm is different from previous multiresolution techniques focusing on wavelets or wavelet-like features [23]–[25] and it is specially designed for the template matching algorithm. In our algorithm, the decomposition of the mask image is conducted from coarse to fine iteratively. The calculation starts from the coarsest level of decomposition, where the size of sub-
image block equals to the whole mask image. Working iteratively from the coarsest to the finest level, for each level of decomposition, the search focuses in on an interval around the “best” warping found at the previous level and is refined at the next level up. In the coarser levels, the structures of large scales are matched to yield a coarse registration. While in the finer levels, the structures at smaller scales are matched to yield a finer registration. With the iteration going on, the best registration result is obtained ultimately. As can be seen in Fig. 1, the algorithm can be described as follows (similarity measure) of the mask image 1) and the live images is calculated. is decomposed as sub2) The mask image , where represents the level of decomimage blocks position, and represents the index of . Thus for a the sub-image block, and is 512 512 mask image, the size of sub-image block at the level of decomposition.
160
IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 54, NO. 1, FEBRUARY 2007
TABLE II COMPARISON OF REGISTRATION RESULTS FOR DIFFERENT ALGORITHMS FOR EIGHT SIMULATED IMAGE PAIRS WITH DIFFERENT SCALES OF MOTION ARTIFACTS
3) Every sub-image block is registered in the live image by the rigid registration algorithm explained above. With the extracted control point pairs, the mask image is warped by thin-plate spline to form the new mask . Then the of the new mask image image and the live image is recalculated. 4) If the metric is improved (for EHD, MI and correlation: ; as for SSD: ), the current mask image is replaced by the warped ; otherwise the current mask new mask image is retained. image 5) The newly obtained warped mask image is used as the starting point for the next level. Repeat step 2, 3, 4 till the registration result is good enough. The use of multiresolution decomposition search strategy has the following advantages. (1) Large-scale warping of anatomic
structures can be eliminated at coarse levels without being affected by the local deformation of small structures. (2) The process is less likely to be trapped into a locally optimal solution because the initial matching is performed in a very coarse resolution. (3) The computational cost can be reduced significantly, for most corrections are performed at the coarser levels. (4) The whole image is used in every iteration for the search of global optimum. (5) Registration processes tend to converge rapidly and the “best” warping can be obtained ultimately. III. EXPERIMENT RESULTS In order to investigate the performance and accuracy of the proposed registration algorithm, it is applied to several sets of simulated and clinical coronary angiographic images. All the images used are 512 512 pixels with 256 gray levels. The algorithm is implemented in the C++ programming language and
YANG et al.: MULTIRESOLUTION ELASTIC REGISTRATION OF X-RAY ANGIOGRAPHY IMAGES USING THIN-PLATE SPLINE
161
Fig. 10. An example of using irregular triangular mesh algorithm on a simulated image pair ( = 5:538). (a) Careful selection of the control points, with the thresholded gradient-magnitude image. (b) The Delaunay triangulation of the control point sets superimposed onto the gradient-magnitude image. (c) Final registration result.
the experiments are carried out on a relatively low cost PC with a Pentium IV 2.8 MHz processor and 1G memory. A. Evaluation of Algorithm on Simulated Data The algorithm is first applied to register simulated images: a live image and its elastically warped versions as the contrast images. The live image Fig. 2(a) is taken from an angiographic image sequence. A set of control points are randomly selected from this image, which can be represented as , where , and is the number of control points. For each control point, an uncorrelated random shift to both x and y dimensions ranging from - to pixels is added. A new set of random control points are thus formed, which can be written as . Note that the motions of the control points should not change the topology of the image. Therefore, the motion boundaries of each control point are strictly restricted not to overlap to each others, as seen in Fig. 2(d). With the generated control point pairs, a new simulated contrast image Fig. 2(b) is produced by thin-plate spline. Fig. 2(c) shows the results of direct subtraction of the two images, where a great deal of artifacts exist. In this way, the true transformation parameters between the two images are known. By calculating the root-mean-squares (RMS) error of the control point pairs in each iteration step, the registration effect can be obtained quantitatively and the algorithm can be evaluated objectively. The RMS error is computed as (18) where , where is the computed transforis the Euclidean distance and is mation, the number of control point pairs. Obviously, the larger the RMS error is, the larger the warping scale of the simulated contrast image will be, and the more artifacts will be found in the subtraction image. In is used to this paper, the normalized distribution control the shifting distance both to x and y dimensions of each control point, so that the RMS error can be set to a certain value. Three experiments (1–3) are conducted to compare the multiresolution search technique versus the single step algorithm,
different similarity measures (EHD, MI, Correlation and SSD) and different interpolation methods (nearest-neighbor, bilinear and bicubic). In these experiments, typically 20 control point to 10 pixels are used, with an RMS pairs ranging from . Experiments in part 4 are carried out to validate error the robustness of the multiresolution algorithm using different warping scales for the simulated contrast images. 1) Comparison of Multiresolution and Single Step Algorithms (Using Bicubic Interpolation Method): In this test, results of our multiresolution algorithm and the single step algorithm over varying numbers of calculation steps are compared, where the bicubic interpolation method and the similarity measure of EHD are used. For the single step algorithm, the live image is simply decomposed to a level of fixed block sizes and the calculation results of each level are independent to one , another. If the decomposition level is , where . The the size of the sub-image blocks is difference between the multiresolution and single step algorithms is that the multiresolution algorithm uses the successful warping result of a coarser level as the initial value for a finer level, whereas the single step algorithm does not. Fig. 3(a) shows the RMS error curves, in which the errors of the multiresolution algorithm decrease rapidly and finally reach a stabilized value of 0.482 pixels. On the other hand, the single step algorithm gives a concave curve. The minimum is obtained at level 8 and the RMS error is 1.488 pixels. Fig. 3(b) gives the EHD curves. The single step algorithm shows a convex curve, at level 8 it reaches a maximum of 0.229, which indicates that level is the best decomposition size 8 of the simulated contrast image for the single step algorithm. Correspondingly, disparities of the image pairs are eliminated to the maximum extent at this level. As for the multiresolution algorithm, EHD increases reposefully to a steady state, which reaches as high as 0.255. Registration results of the single step algorithm can be found in the first row of Fig. 4, where extensive artifacts still exist. It is seen that the subtraction image of level 8 has less motion artifacts than the other three images, which is exactly in accord with the above EHD curve and RMS analysis of the single step algorithm. Accordingly, the second row in Fig. 4 provides the registration result of the multiresolution algorithm, where artifacts are gradually removed along with the performing of the iterations.
162
IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 54, NO. 1, FEBRUARY 2007
TABLE III COMPARISON OF FINAL REGISTRATION RESULTS OF THE IRREGULAR TRIANGULAR MESH ALGORITHM, THE REGULAR GRID ALGORITHM AND THE PROPOSED MULTIRESOLUTION ALGORITHM
Fig. 11. Multiresolution registration results of different methods at varying numbers of decomposition levels for data set 1. (a) Normalized curves for different similarity measures. (b) EHD curves for different interpolation methods.
The experiment results show that the multiresolution algorithm outperforms the single step algorithm considerably, and the estimated transformation parameters of the simulate image are very close to the true values. 2) Comparisons of Different Similarity Measures (Using Bicubic Interpolation Method): As the previous experiment verifies that our multiresolution search strategy is an efficient registration algorithm for X-ray angiography images, this part is devised to compare different similarity measures used in the multiresolution algorithm, when the bicubic interpolation method is used. Fig. 5(a) shows the RMS pixel error curves, in which the RMS errors for EHD and MI decrease gradually and finally reach a stabilized point, while the errors for correlation and SSD first decrease to a minimum, then increase. The
calculated RMS errors for EHD and MI, which reach as low as 0.482 and 0.702 respectively, are evidently lower than the other two measures. Clearly, EHD is the best of the four similarity measures from the RMS error criterion. Fig. 5(b) gives the normalized measures of EHD, MI, correlation and SSD over different decomposition levels. As mentioned earlier, the optimization runs are designed to search for the maximum of EHD, MI or correlation value and for the minimum of SSD value, which is why in Fig. 5(b) EHD, MI and correlation values increase gradually, while SSD decreases on the contrary. Different measure has different optimization range. In our example, EHD has the largest optimization scale ranging from 0.22 to 1.0, while correlation has the smallest optimization scale ranging from 0.918 to 0.986. A larger optimization range scale
YANG et al.: MULTIRESOLUTION ELASTIC REGISTRATION OF X-RAY ANGIOGRAPHY IMAGES USING THIN-PLATE SPLINE
163
Fig. 12. Multiresolution registration results for data set 1. (a) Original subtraction of live images from mask image showing significant motion artifacts. (b) Registration result based on SSD. (c) Registration result based on Correlation. (d) Registration result based on MI using bicubic interpolation method. (e) Registration result based on EHD using nearest interpolation method. (f) Registration result based on EHD using bilinear interpolation method. (g) Registration result based on EHD using bicubic interpolation method.
does not necessarily correspond to a better registration result. Each similarity measure has a different criterion, and it is a relative measurement of the two images. Fig. 6 gives the registration results for the four similarity measures. The columns from left to right correspond to EHD, MI, Correlation and SSD respectively. The rows from top to bottom correspond to decomposition level 4, 8, 12, and 16 respectively. It can be seen that for the four measures, artifacts are removed gradually along with the iterations, but using EHD and MI can achieve better registration results than the other two. At the final decomposition level, artifacts are almost completely removed when EHD and MI are used as the similarity measures. 3) Comparisons of Different Interpolation Method (Using EHD as the Similarity Measure): The following set of tests is designed to compare different interpolation methods used in the multiresolution search strategy, where EHD is selected as the similarity measures. Fig. 7(a) shows the RMS pixel error curves over varying numbers of decomposition levels. It can be seen that the bicubic interpolation is the best among the three interpolation methods, for its errors are smaller than the other two methods. Table I gives the details of pixel errors, including maximum, minimum, mean and RMS errors of different iteration levels of the three interpolation methods, in which the initial mean error of the calculating point pairs is 5.082 pixels, while the RMS error is 5.538 pixels. After the registration process, the mean error and the RMS errors of the bicubic interpolation method reach 0.433 and 0.482 pixels respectively. Obviously sub-pixel registration accuracy is obtained. Fig. 7(b) shows the EHD curves of the original live image and simulated contrast image over varying numbers of decomposition levels, in which the initial EHD is 0.056. The best registration result for the nearest method reaches 0.194, and that for the bilinear method reaches a higher degree of 0.231, whereas the bicubic method achieves a value of 0.255. Fig. 8
gives the multiresolution registration results using the three interpolation methods. The columns from left to right correspond to the nearest, the bilinear and the bicubic interpolation method respectively. The rows from top to bottom correspond to decomposition level 4, 8, 12, and 16 respectively. Clearly, the bicubic interpolation method is the best among the three. In the final iteration step, the two graphics are perfectly aligned, and almost no artifacts can be found in the subtraction image. It can be seen from the experiments that although the EHD measure is used, the nearest and bilinear interpolation method cannot achieve satisfactory registration accuracy. Only with the more precise bicubic interpolation method, the desirable registration results can be obtained. 4) Validity and Robustness of the Multiresolution Algorithm (Using Bicubic Interpolation Method): This part is devised to further verify the validity and robustness of the multiresolution algorithm using EHD as the similarity measure. Eight simulated contrast images with different scales of artifacts are generated by adjusting the RMS errors of the control point pairs. For the simulated images, the control points are randomly generated and the motions of corresponding points range from 0.417 to 32.58 pixels, which is sufficient for the simulation of common clinical angiograms with a resolution of 512 512. Fig. 9 gives eight simulated subtraction images and the corresponding subtraction images after registration by the EHD based multiresolution algorithm. The first and the third rows correspond to the simulated subtraction images. The RMS errors of the first row are 3.337, 4.250, 5.538, 6.862 pixels and those for the third row 7.947, 8.867, 9.714, 10.634 pixels respectively. The second and the fourth rows illustrate the subtraction images after registration. It can be seen that the proposed algorithm is always effective for the registration of the simulated images. Almost no artifacts can be found in the subtraction images after registration.
164
IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 54, NO. 1, FEBRUARY 2007
Fig. 13. Experiment results of other six coronary angiographic images. The first and the third row: the original (before registration) coronary angiographic images. The Second and the fourth row: the corresponding subtraction images after registration resulting from the automatic image registration technique described in this paper (multiresolution algorithm based on EHD using bicubic interpolation method).
Table II lists the final registration results of different algorithms for the eight simulated image pairs. For all the image pairs tested, the multiresolution algorithm considerably outperforms the single step algorithm. When using EHD as the similarity measure, the RMS error reduction ratio for the eight image pairs is 89.5%, and the corresponding value for the single step algorithm is only 54.5%. The second part of the table shows the registration results of the multiresolution algorithm with MI, correlation and SSD as the similarity measures. It is clear that EHD is consistently the best choice of similarity measure for all the test cases. The EHD based multiresolution algorithm is also compared to the commonly used regular grid algorithm (using a grid size
of 51 51) and the irregular triangular mesh algorithm developed by Meijering [6]. Fig. 10 gives an example of using the irregular triangular mesh algorithm on a simulated image pair . It can be observed from Table III that this algorithm is effective only when the initial RMS error is comparably small (less than 4.250). For the eight tested image pairs, the reduction ratios for RMS error stand at 59.7% and 44.1% respectively for the irregular triangular mesh algorithm and the regular grid algorithm. The proposed multiresolutin algorithm is obviously much more effective. The improved registration accuracy is achieved at a cost: the proposed algorithm needs more computing. When the irregular triangular mesh registration algorithm can complete the regis-
YANG et al.: MULTIRESOLUTION ELASTIC REGISTRATION OF X-RAY ANGIOGRAPHY IMAGES USING THIN-PLATE SPLINE
tration calculation in 4.42 minutes on our computer, the proposed algorithm needs 17.87 minutes. However, with the rapid advance in computing power, the extra computation required should not be a major problem. B. Experiments on Clinical Cardiac Angiographic Images In this section, the multiresolution registration algorithm is applied to different clinical cardiac angiographic image data sets. Figs. 11 and 12 provide the detailed results on one of the data sets, where the registration results of the multiresolution algorithm using different similarity measures and different interpolation methods are compared. Fig. 13 gives the experiments on six other data sets, where the multiresolution algorithm with EHD and bicubic interpolation is used. Fig. 11(a) gives the normalized similarity measure curves (using the bicubic interpolation method) of different similarity measures. It can be seen that EHD and MI are more effective than correlation and SSD for their comparatively smooth curves. Fig. 11(b) shows the EHD curves for different interpolation methods. Although all the three curves are rising with the finer decomposition of the mask image, the bicubic interpolation method achieves better registration results than the others. The EHD value of the original live and mask images is 0.019. After the registration process, the EHD value obtained by the nearest, bilinear and bicubic interpolation methods are 0.029, 0.0305, and 0.0327 respectively. Fig. 12 provides the final registration results of data set 1 using different algorithms. It can be seen that although the bicubic interpolation method is used, artifacts cannot be removed completely if the similarity measure is SSD or correlation. Also, the desirable registration result cannot be achieved if the measure of EHD is used in combination of the nearest and bilinear interpolation methods because of their low interpolation accuracy. Only the bicubic interpolation method based on EHD and MI yield satisfactory registration results (see Fig. 12(f)), where the artifacts are almost completely removed. Moreover, the registration result of EHD is better than MI with a smoother subtraction image. Fig. 13 shows the experimental results for the other six data sets. The first and the third rows give the original subtraction of the live images from the mask images. The second and the fourth rows give the corresponding multiresolution registration results based on EHD and the bicubic interpolation. As in the previous experiments, motion artifacts are largely removed and crystal clear images of coronary arteries are obtained.
IV. CONCLUSION AND DISCUSSION In this paper, an effective and robust algorithm for the registration of digital angiography images is proposed. A multiresolution search strategy is developed in which the mask image is decomposed to coarse and fine sub-image blocks iteratively. At each level the calculation is based on the “best” warping result found at previous levels and tries to further refine it. In the experiments, the multiresolution refinement algorithm based on EHD using the bicubic interpolation method is demonstrated to
165
be very effective and robust for the registration of X-ray angiographic images. So far, a key problem of the elastic registration for multimodality images is that there is no objective evaluation of the algorithm. In order to evaluate the effectiveness of a developed algorithm, it is compared to the manual pixel shifting method, which is subjective and time consuming. In this paper an objective evaluation method is developed on simulated data, in which simulated contrast images with different scales of artifacts are generated by controlling the RMS error of the randomly generated control point pairs. By calculating the RMS errors before and after registration, it is possible to quantitatively evaluate the quality of the elastic registration results. Experimental results show that the proposed approach is capable of sub-pixel registration of X-ray angiography images. About 90% of the introduced errors are removed. Future research will focus on the better understanding of the influence of implementation issues, such as leaking of angiographic substances, low luminance contrast of micro-vessels. Also, the calculation speed needs to be improved. ACKNOWLEDGMENT The authors would like to thank Dr. Y. Li of Beijing Chaoyang Red Cross Hospital for providing us with the data sets used in the experiments. REFERENCES [1] W. R. Brody, “Digital subtraction angiography,” IEEE Trans. Nucl. Sci., vol. NS-29, no. 3, pp. 1176–1180, Jun. 1982. [2] B. Desprechins, R. Luypaert, M. Delrée, and M. Freson, “Evaluation of time interval difference digital subtraction fluoroscopy in patients with cystic fibrosis,” Scand. J. Gastroenterol Suppl., vol. 143, pp. 86–92, 1988. [3] R. A. Kruger, F. J. Miller, J. A. Nelson, P. Y. Liu, and W. Bateman, “Digital subtraction angiography using a temporal bandpass filter: Associated patient motion properties,” Radiology, vol. 145, no. 2, pp. 315–320, 1982. [4] H. Oung and A. M. Smith, “Real time motion detection in digital subtraction angiography,” in Proc. Int. Symp. Medical Images and Icons, A. Deurinckx, M. H. Loew, and J. M. S. Prewitt, Eds., Silver Spring, RI, 1984, pp. 336–339. [5] W. R. Brody, “Hybrid subtraction for improved arteriography,” Radiology, vol. 141, no. 3, pp. 828–831, 1981. [6] E. H. W. Meijering, K. J. Zuiderveld, and M. A. Viergever, “Image registration for digital subtraction angiography,” Int. J. Comput. Vis., vol. 31, no. 2/3, pp. 227–246, 1999. [7] Y. Bentoutou and N. Taleb, “Automatic extraction of control points for digital subtraction angiography image enhancement,” IEEE Trans. Nucl. Sci., vol. 52, no. 1, pp. 238–246, Feb. 2005. [8] E. H. W. Meijering, W. J. Niessen, and M. A. Viergever, “Retrospective motion correction in digital subtraction angiography: A review,” IEEE Trans. Med. Imag., vol. 18, no. 1, pp. 2–21, 1999. [9] A. S. Talukdar and D. L. Wilson, “Image quality optimization for automatic warping registration in X-ray DSA,” in Proc. 19th Annu. Int. Conf. IEEE/EMBS, 1997, vol. 2, pp. 549–552. [10] T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991. [11] B. Zitova and J. Flusser, “Image registration methods: A survey,” Image Vis. Comput., vol. 24, pp. 977–1000, 2003. [12] W. Cheney and D. Kincaid, Numerical Mathematics and Computing, 4th ed. Pacific Grove, CA: Brooks/Cole, 1999. [13] J. B. A. Maintz and M. A. Viergever, “A survey of medical image registration,” Med. Image Anal., vol. 2, no. 1, pp. 1–36, 1998. [14] M. J. D. Powell, “An efficient method for finding the minimum of a function of several variables without calculating derivatives,” Comput. J., vol. 7, pp. 155–162, 1964.
166
IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 54, NO. 1, FEBRUARY 2007
[15] T. M. Buzug, J. Weese, C. Fassnacht, and C. Lorenz, “Using an entropy similarity measure to enhance the quality of DSA images with an algorithm based on template matching,” in Visualization in Biomedical Computing (VBC’96) Transl.:Lecture Notes in Computer Science, K.-H. Hohne and R. Kikinis, Eds. Berlin, Germany: Springer-Verlag, 1996, vol. 1131, pp. 235–240. [16] T. M. Buzug and J. Weese, “Improving DSA images with an automatic algorithm based on template matching and an entropy measure,” in Computer Assisted Radiology (CAR’96), ser. International Congress Series, H. U. Lemke, M. W. Vannier, K. Inamura, and A. G. Farman, Eds. Amsterdam, The Netherlands: Elsevier, 1996, vol. 1124, pp. 145–150. [17] F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens, “Multimodality image registration by maximization of mutual information,” IEEE Trans. Med. Imag., vol. 16, no. 2, pp. 187–198, 1997. [18] J. P. W. Pluim, J. B. A. Maintz, and M. A. Viergever, “Mutual-information-based registration of medical images: A survey,” IEEE Trans. Med. Imag., vol. 22, no. 8, pp. 986–1004, 2003. [19] W. K. Pratt, “Correlation techniques of image registration,” IEEE Trans. Aerosp. Electron. Syst, vol. 10, no. 3, pp. 353–358, 1974.
[20] J. B. A. Maintz, V. D. Elsen, and M. A. Viergever, “Evaluation of ridge seeking operators for multimodality medical image matching,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 18, pp. 353–365, 1996. [21] F. L. Bookstein, “Principal warps: Thin-plate splines and the decomposition of deformations,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 11, no. 6, pp. 567–585, 1989. [22] H. Chui and A. Rangarajan, “A new point matching algorithm for non-rigid registration,” Comput. Vis. Image Understanding, vol. 89, pp. 114–141, 2003. [23] P. Thevenaz and M. Unser, “Optimization of mutual information for multiresolution image registration,” IEEE Trans. Image Process., vol. 9, no. 12, pp. 2083–2099, 2000. [24] G. E. Christensen and H. J. Johnson, “Consistent image registration,” IEEE Trans. Med. Imag., vol. 20, no. 7, 2001. [25] P. Thevenaz, U. E. Ruttimann, and M. Unser, “A pyramid approach to subpixel registration based on intersity,” IEEE Trans. Image Process., vol. 7, no. 1, 1998.
