Abstract-The mean divergence measures are used as the similarity measure of medical image registration. The square root arithmetic mean divergence (SAM), ...
Mean Divergence Measures for Multimodality Medical Image Registration Changchun Liu, Jinbao Yang, Peng Shao, Jason J. Gu* , and Mengsun Yu School of Control Science and Engineering, Shandong University, Jinan, 250061, China; * Department of Electrical and Computer Engineering, Dalhousie University, Canada Abstract-The mean divergence measures are used as the similarity measure of medical image registration. The square root arithmetic mean divergence (SAM), square root geometric mean divergence (SGM), square root harmonic mean divergence (SHM), arithmetic geometric mean divergence (AGM), and arithmetic harmonic mean divergence (AHM) are applied to rigid registration of computed tomography (CT)/positron emission tomography (PET) and magnetic resonance (MR)/ PET images. The performance of these measures is studied in comparison with mutual information. The proposed registration measures have similar function curves with mutual information. Experimental results show that the AHM measure has significant improvement in registration speed.
I.
INTRODUCTION
Intensity based image registration is one of the most popularly used methods for automatic multimodality image registration. Recently, various improvements have been suggested, ranging from variation in the similarity metrics to improvement in the interpolation techniques. As a similarity measure, mutual information was pioneered both by Collignon et al. [1], and by Viola and Wells [2]. Applied to rigid registration of multimodality images, mutual information showed great promise and within a few years it became the most investigated measure for medical image registration. The mutual information (MI) of two images A and B can be defined as p ( a ,b ) I ( A, B ) = ∑ p ( a ,b )ln , (1) ( p a ) p(b) a ,b where a and b are image intensity values, p ( a ) , p ( b ) , and p ( a ,b ) are the marginal distribution and joint distribution of gray pairs ( a ,b ) , p ( a ) p (b ) is the joint distribution in the case
of complete independence. MI measures the degree of dependence of A and B by measuring the distance between p ( a ,b ) and p ( a ) p (b ) , by means of the Kullback-Leibler measure. Maximal dependence is assumed to have occurred when the images are aligned. Misregistration will result in a decrease in the measure. Research into the similarity measure currently takes up a substantial part of medical image registration research. Pluim et al. [3] have studied the performance of f -information and Rényi relative information of order α . Wachowiak et al. [4] compare information measures based on Rényi and TsallisHavrda-Charvát entropies and Iα -information. Apart from mutual information, there are many divergence metrics can
measure the distance between two probability distributions. In this paper we proposed mean divergence as the similarity measure of medical image registration. And then we validated these measures for registering computed tomography (CT), magnetic resonance (MR), and positron emission tomography (PET) images. II. METHODS A. Mean of Order t Let us consider the following well-known mean of order t which was proposed by Beckenbach and Bellman [5], 1t ( at + bt ) 2 M (a, b) = ab t max{a ,b} min{a ,b} for all a , b , t ∈ ∩ a , b > 0 .
(
)
t ≠0 t =0 , t =∞ t =−∞
(2)
The harmonic mean, geometric mean, arithmetic mean, and square root mean are derived from the mean of order t , taking t =-1, 0, 1, and 2, respectively. 1) harmonic mean 2ab (3) M −1 ( a , b ) = = H (a , b ) a +b 2) geometric mean M 0 (a ,b)= ab =G ( a,b) (4) 3) arithmetic mean a +b (5) M1(a,b)= = A( a,b) 2 4) square root mean a 2 + b2 M ( a, b) = = S ( a, b) (6) 2 2 Since the M t ( a ,b ) is monotonically non-decreasing function in relation to t , we have the following inequality H ( a, b) ≤ G (a, b) ≤ A( a, b) ≤ S ( a, b). (7) Then we have the following non-negative differences M SA ( a ,b ) = S ( a ,b ) − A( a ,b ) =
a 2 +b 2 a +b − 2 2
(8)
M SG ( a ,b ) = S ( a ,b )−G ( a ,b )=
a 2 +b2 − ab 2
(9)
M SH ( a ,b ) = S ( a ,b ) − H ( a ,b ) =
a 2 +b 2 2 ab − 2 a +b
(10)
1171 0840-7789/07/$25.00 ©2007 IEEE
a +b − ab 2 a +b 2 ab − M AH ( a ,b ) = A( a ,b ) − H ( a ,b )= 2 a +b
(11)
M AG ( a ,b ) = A( a ,b )−G ( a ,b )=
(12)
Obviously, some of them are related in the following manners: M SG ( a ,b ) = M SA ( a ,b ) + M AG ( a ,b ) (13) M SH ( a ,b ) = M SA ( a ,b ) + M AH ( a ,b ) (14) B. Mean Divergence Measure Let ∆ = P = ( p , p ,… , p ) p > 0, ∑ n p = 1 , n ≥ 2 i =1 n
{
1 2
n
i
i
}
be
given above and sum over all i =1,2,…,n , then we have the following mean divergence measures: 1) Square root-arithmetic mean divergence (SAM) n p2 + q2 i i −1 M SA ( P Q )= ∑ 2 i =1
(15)
n M SG ( P Q ) = ∑ i =1
(16)
2) Square root-geometric mean divergence (SGM)
pi2 + qi2 2 pi qi − pi + qi 2
(17)
(18)
5) Arithmetic-harmonic mean divergence (AHM) n 2p q i i M AH ( P Q ) =1− ∑ p + i =1 i qi
Moreover, ′ ( x )= f SA
1 − , 2 2 2 x +1 x
(22)
1
f ′′ ( x ) = SA
(23) . 2 ( x 2 +1) x 2 +1 ′′ ( x ) >0 for all x∈(0,∞ ) . Also, we have Thus we have f SA f SA (1)= 0 . In view of this we can say that the SAM is nonnegative and convex in the pair of probability distributions ( P , Q) ∈ ∆ n × ∆ n .
In the same way, we can conclude that the SGM, SHM, AGM, and AHM are nonnegative and convex in the pair of probability distributions ( P , Q) ∈ ∆ n × ∆ n .
For two images A and B , let pab and pa pb denote p ( a ,b ) and p ( a ) p (b) for simplicity. By substituting pab , pa pb for pi , qi in (15), (18), and (19), we obtain the following form of SAM, AGM, and AHM measures. M SA ( A, B ) = ∑ a ,b
4) Arithmetic-geometric mean divergence (AGM) n M AG ( P Q )=1− ∑ pi qi i =1
(21)
D. Measure for Registration
3) Square root-harmonic mean divergence (SHM) n M SH ( P Q )= ∑ i =1
x 2 +1 x +1 , x∈(0,∞ ) − 2 2 According to (20), then we have I f ( P Q )= M SA ( P Q ) . f SA ( x ) =
and
the set of complete finite discrete probability distributions. For P , Q ∈ ∆ n , let us take a = pi and b = qi in the differences
pi2 + qi2 − pi qi 2
Given a function
(19)
C. Convexity of Mean Divergence We can check the convexity of the mean divergence measures by their relation to f -divergence. In fact, each of them can be considered a special type of f -divergence which expresses the difference extent of two discrete probability distributions. Given two distributions P, Q∈∆ n , the f -divergence measure introduced by Csiszár [6] is given by:
p (20) I f ( P Q ) = ∑ qi f i , i qi where f (u ) is a continuous and convex function on [0,∞ ) . The f -divergence has the well-known property [7], [8]: Let be differentiable convex and the function f : [0, ∞) → normalized, i.e., f (1) =0 , then the f -divergence, I f ( P Q ) is nonnegative and convex in the pair of probability distribution ( P , Q) ∈ ∆ n × ∆ n .
1172
2 +( p p )2 pab a b −1 2
M AG ( A, B ) =1− ∑ pab ⋅ pa pb a ,b 2⋅ pab ⋅ pa pb M AH ( A, B ) =1− ∑ a ,b pab + pa pb
(24) (25) (26)
According to (13), (14), we can obtain the SGM and SHM measures for two images: M SG ( A, B )= M SA ( A, B ) + M AG ( A, B ) (27) M SH ( A, B ) = M SA ( A, B )+ M AH ( A, B ) (28) These measures can determine the distance of p ( a ,b ) and p ( a ) p (b ) because of their convexity. The more high value denotes more dependence between the two images. When the two images are aligned, maximal dependence of the two images is assumed to occur, then the value p ( a ,b ) / p ( a ) p (b ) is far from 1, the measures SAM, AGM, AHM, SGM, and SHM get to maximum, respectively. Registration is achieved by finding the geometrical transformation, such as translation or rotation, which results in the highest measure value. III. EXPERIMENTS We validated these registration measures for registering CT/PET images and MR/PET images of brain. Each slice of them is shown in Fig. 1. The real 15 pairs of CT/PET images are obtained from our cooperative hospital and the resolution of PET image is 128×128. The emulational MR/PET images
Fig. 1. Images used for registration. (a), left top: CT, (b), right top: PET, (c), left bottom: MR, and (d), right bottom: PET.
are acquired from the Whole Brain Atlas and the resolution of images is 256×256. The probability distributions of gray values are estimated from gray value histograms, using 256 bins [9]. The curves of functions, computational demand, and convergence of these five measures are studied in comparison with mutual information measure. A. Curves of Functions As a first illustration of the performance of the different measures, registration curves as a function of one of the transformation parameters are studied. Taking Fig. 1(a) as the reference image and Fig. 1(b) as the float image, the behaviors of MI, SAM, AGM, AHM, SGM, and SHM as functions of xshifts and rotation are given in Fig. 2. and 3. Partial volume distribution (PV) interpolation is applied to approximate gray values at nongrid positions.
0.6
0.04
0.4
0.02 0 MI
20
0.12 0.1 0.08 0.06 0.04
-20
0 SAM
20
0.15
-20
0 AHM
0.1
0.05
0.05
20
-20
0 AGM
20
-20
0 SHM
20
0.15
0.1
-20
0 SGM
20
Fig.2. Registration measures as functions of x-shifts (in pixels).
0.86
0.0587
0.126 0.1255 0.125 0.1245
0.0586
0.855
0.0585 -2
0 MI
2
0.14
-2
0 SAM
2
0.185
0.1395 0.183 -2
0 AHM
2
-2
0 AGM
2
-2
0 SHM
2
0.1986 0.1984 0.1982 0.198 0.1978
0.184
-2
0 SGM
2
The comparison of computational time was made in three tests: (1) Take Fig. 1(a) as the reference image and Fig. 1(b) as the float image, calculating the value of the measures once. (2) Let translation range along x axis be (-10 10) pixels, translation range along y axis be (-10 10) pixels, calculate the time when the images are aligned. (3) For MR/PET images, taking Fig. 1(c) as the reference image and Fig. 1(d) as the float image, repeat step (2). The results are shown in Table I. The time unit is second. The SGM is the slowest one. The SAM and SHM measures have little difference in speed. The AHM measure is faster than others for registering real CT/PET data because of its simpler expression. The robustness of a measure with respect to the initial misregistration depends primarily on the smoothness of the registration function. A smooth function is easy to optimize and large initial misregistration can be corrected. In [10] and [11], the quality of the measures in the vicinity of the global extreme is characterized by a parameter that Capek named AFA (Area of Function Attraction). AFA evaluates the range of convergence of a registration measure to its global maximum, counting the number of pixels from which the global maximum is reached by applying a maximum gradient method. The higher the AFA, the wider the attraction basin of the measure is. But the computation of AFA is complicated and time-consuming. We propose another parameter named XCW (Convergence Width along the x axis). XCW counts the number of pixels from which the function curves over x shifts reach global maximum by line search. Similar to AFA, the higher the XCW, the wider the attraction basin of the measure is. Take Fig. 1(a) as the reference image and Fig. 1(b) as the float image. In the translation range of X (-30 30), Y (-30 30), we calculate the XCW, the number of local maxima (nLoc) and the nearest local maximum (NLM) of various measures given above, the results shown in Table II are obtained. The SAM measure has the highest nLoc, that is to say, the smoothness is the worst. The nLoc of AGM is lower than that
0.2 -20
B. Computational Demand
C. Robustness
0.12 0.1 0.08 0.06 0.04
0.8
We can see that the proposed registration measures have similar function curves with mutual information. In Fig. 2, all measures get to maximum at the position of zero translation and have a good smoothness in the translation range. For image rotation, all the measures yield lots of local maxima (Fig. 3). For other slices of CT/PET and MR/PET images, the results are similar. But for CT/MR images registration, all the measures have a smooth curve as a function of translation or rotation. It is the most difficult case for MR/PET images registration. One reason is the low resolution and large noise of PET images and the lack of similarities between anatomical images and PET images. And another reason is the interpolation artifacts introduced by interpolation algorithm for estimating the voxel intensities at nongrid positions.
Fig.3. Registration measures as functions of rotation (in degrees)
1173
ACKNOWLEDGMENT
TABLE I COMPUTATIONAL TIME OF MEASURES
MI
AHM
SAM
AGM
SGM
SHM
T1
0.032
0.016
0.046
0. 062
0.063
0. 063
T2
15.969
11.453
22.187
20.141
24.922
22.437
T3
27.250
33.234
43.485
30.047
44.780
37.906
This work was supported by Shandong Natural Science Foundation (Grant No. Z2006C05). REFERENCES [1]
TABLE II CONVERGENCE OF VARIOUS MEASURES
MI
AHM
SAM
AGM
SGM
SHM
nLoc
64
188
207
110
139
194
NLM
(-14, 1)
(-12, 2)
(-12, 2)
(6, 9)
(-12, 2)
(-12, 2)
XCW
60
51
50
51
50
51
of AHM, SAM, SGM, and SHM measures, but higher than MI. In sum, the MI measure has larger convergence range and is more robust than others. IV. CONCLUSION In this paper we proposed five mean divergence measures as the similarity metric of rigid medical image registration. We have evaluated the performance of our measures in rigid intrasubject registration of brain images. The proposed registration measures have similar function curves with mutual information measure. They can also be used for multimodality registration. Experimental results show that the AHM measure has significant improvement in registration speed without compromising registration performance. For the sake of registration speed, the AHM measure should be a good choice.
1174
A. Collignon, F. Maes, D. Delaere, D. Vandermeulen, P. Suetens, and G. Marchal, “Automated multimodality medical image registration using information theory,” in Proc. Int. Conf. Information Processing Medical Imaging: Computational Imaging and Vision, 1995, pp. 263-274. [2] P. Viola and W. M. Wells III, "Alignment by maximization of mutual information," in Proc. Int. Conf. on Computer Vision, Cambridge, CA, 1995,16-23. [3] J. P. W. Pluim, J. B. A. Maintz, and M. A. Viergever, “f-Information measures in medical image registration,” IEEE Trans. Med. Imag., vol. 23, no. 12, pp. 1508-1516, 2004. [4] M. P. Wachowiak, R. Smolkov, G. D. Tourassi, and A. S. Elmaghraby, “Similarity metrics based on nonadditive entropies for 2D-3D multimodal biomedical image registration,” Proc. SPIE, pp. 1090-1100, 2003. [5] E. F. Beckenbach and R. Bellman, Inequalities, Springer-Verlag, New York, 1971. [6] I. Csiszár, “Information-type measures of difference of probability distributions and indirect observations,” Studia Scientiarum Mathematicarum Hungarica, 2, pp. 299-318, 1967. [7] I. Csiszár, “On Topological Properties of F-Divergences,” Studia Scientiarum Mathematicarum Hungarica, 2, pp. 329-339, 1967. [8] I. J. Taneja, “Refinement Inequalities Among Means,” Available on-line at: arXiv:math.GM/0505192 v1, 10 May 2005. [9] F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens, “Multimodality image registration by maximization of mutual information,” IEEE Transactions on Medical Imaging, vol. 16, pp. 187198, Apr. 1997. [10] M. Capek, L. Mroz, and R. Wegenkittl, “Robust and fast medical registration of 3D-multi-modality data sets,” In Proceedings of the International Federation for Medical and Biological Engineering, 2001, vol. 1, pp. 515-518. [11] A. Bardera, M. Feixas, and I. Boada, “Normalized similarity measures for medical image registration,” Medical Imaging SPIE 2004, vol. 5370, pp. 108-118.
