Variational segmentation of multi-channel MRI images - Image ...

VARIATIONAL SEGMENTATION OF MULTI-CHANNEL MRI IMAGES Homer H. Pien * C. S. Draper Laboratory, Cambridge MA, 02139, and College of Computer Science, Northeastern University, Boston MA, 02115 John M. Gauch t Department of EECS, University of Kansas, Lawrence KS, 66045

Magnetic resonance imaging (MRI) has become a valuable tool in the medical sciences; the non-invasive and high-resolution nature of MRIs have become major reasons for their pervasiveness. Because the size and shape of various anatomical structures are associated with the aging process, diseases, and injuries, the ability to segment and subsequently quantify volume and morphology is important. Due to the labor intensive and errorprone nature of manual segmentation, however, there is a tremendous need for automated segmentation techniques. The ability to accurately, robustly, and automatically segment MRI images is therefore crucial to maximizing the amount of information one can extract from MRI images. Segmentation of MRI images of the human brain is an especially difficult problem due to several reasons: the wide range of scales at which segmentation must occur; the partial volume effect; the low contrast between tissue classes, and the uneven response to the magnetic field. As such, in order to segment MRI images of a human brain into the three tissue classes (i.e.,

gray matter, white matter, and cerebral spinal fluid (CSF)), one must deal with low contrast and blurred edges, as well as non-uniform responses from pixels in the same tissue class. For these reasons, past segmentation schemes have relied on man-in-the-loop semiautomated segmentation. Recognizing the difficulty of singlechannel segmentation, researchers have recently looked to the image acquisition process itself in order to improve the segmentation process. One approach is to utilize the multi-channel nature of MRI data. Specifically, in addition to acquiring proton density (PO) data, channels characterizing different aspects of the molecular relaxation process - such as TI and T2 - can also be utilized. In this paper we develop a variational framework for performing multi-channel segmentation. This is accomplished by combining multi-channel data via the optimization of an energy functional incorporating two types of quantities: (i) a piecewise smooth estimate of the intensity of each channel, and (ii) an edge process function indicating the probability of the presence of an intensity discontinuity at each pixel. We emphasize that the edge function is estimated on the basis of all input channels simultaneously, instead of some ad hoc combination of edge pixels extracted from different channels independently. The objective of this research is to derive a formulation for segmenting MRI images, with particular emphasis on robust segmentation by incorporating multichannel data. Although the domain of application is that of MRI images of the human brain, it is hoped that the derived formulation can be useful in other applications as well (e.g., cardiac MR imaging, functional MRI, and multi-spectral images). Note that the purpose of this paper is segmentation, not classification. As such, only a description of how the image is partitioned, not how each region is labeled, is provided.

*Supportedin part by Draper Laboratory IR&D Project No. 451 +Supportedin part by NSF Grant No. IRE9109431

This paper is organized as follows. A brief review of MRI imagery, emphasizing the complementary nature of the different channels of data, as well as an intro-

ABSTRACT MRIs are effective for non-invasively imaging the interior of the human brain. Due to the large amount of data associated with typical MRI sessions, manual segmentation of the images of the human brain is prohibitive except in isolated cases. The various imaging and contrast artifacts common to MRIs, however, make automatic segmentation difficult. In this paper a segmentation algorithm incorporating multi-channel MRI data is described; this approach utilizes the variational calculus formulation to simultaneously compute piecewise smooth estimates of each channel, as well as a continuous “edge process” common to all the channels.

1. INTRODUCTION

0-8186-6950-0/94 $4.00 0 1994 IEEE

508

ror rate [3]. Thus despite the successes associated with semi-automated segmentation algorithms (see [ll,12]), our emphasis is on the automated segmentation of MR images.

duction to the motivation for studying human brain morphology, is provided in Section 2. A review of variational segmentation is provided in Section 3. The proposed formulation for multi-channel segmentation is discussed in Section 4, with experimental results shown in Section 5 . Section 6 contains our conclusions.

3. VARIATIONAL SEGMENTATION

One general approach to many image processing problems is to utilize the creation of an energy functional, the optimization of which provides a trade-off between fidelity to the observed data and various desirable constraints (e.g., smoothness, segmentation, etc.). To solve these energy functionals, Euler-Lagrange equations derived via variational calculus are used to determine the conditions for optimality. These variational formulations have been used extensively in the regularization of many ill-posed image processing and computer vision problems (see [13]). In particular, variational image segmentation was first demonstrated by Blake and Zisserman (see [14]); Mumford and Shah [15] later cast the segmentation problem into the continuous domain, using

2. MR IMAGING AND ANATOMICAL

CHANGES IN THE HUMAN BRAIN

A MR imaging system is comprised of a large magnet that is used to align hydrogen protons within the subject. During image acquisition, a RF-pulse is introduced in order to deflect the aligned protons. After the RF-excitation, these hydrogen protons are allowed to relax in order to come into re-alignment with the magnetic field. The amplitude of the signal produced by the resonating protons is the proton spin density (PSD, or PD for short). The time constants TI and T2 are used to characterize the relaxation process: TI measures the interactions of hydrogen protons with the surrounding magnetic field, and T2 measures interactions of neighboring protons [l, 21. TI, T2, and P D images highlight different aspects of the human brain. For example, TI tends to show the overall contrast of the different tissue classes of the brain, whereas T2 emphasizes variations in water content (e.g., strokes, bleeds, and other lesions) [3, 41. Segmentation of MR images is important because the size and shape of many anatomical structures are correlated with diseases, injuries, and the aging process. For example, Alzheimers and schizophrenia have been correlated with an enlargement of the brain fluid spaces [5]; schizophrenia has also been correlated with changes in the frontal lobe area [6, 71; cerebral edema after head injury has been shown to result in brain enlargement [3], and age-related decreases in brain volume in non-demented subjects have been shown [8]. In all these (and many other) cases, it is the volume of the structure instead of the cross-sectional area that is of importance. Despite our ability to visually perceive small changes in 2-D images (i.e., area), perception of volumetric changes through the mental accumulation of multiple cross sections is far more difficult [9]. Indeed, it has been shown that the volume of a brain tumor must differ by as much as 40% - 50% before such changes are recognized during clinical reviews of individual slices [3, lo]. Furthermore, due to the large quantity of data present in a full MRI scan of an adult head, manual segmentation is a time consuming and error prone activity. For example, it takes approximately 4 hours to process each scan slice via computer-assisted hand-drawn segmentation, with an inherently high er-

E

=

J Jn (f(z,Y) - d ( z , Y))2dzdY+ A J i-* lVf(2, Y)12dZdY + 4%

(1) Y)l

in which d denotes the data, f denotes the piecewisesmooth approximating surface, B the binary edge process, and R the domain of integration. Intuitively, the ( f ,8)-pair minimizing (1) has the properties of (i) f closely resembles the data d everywhere; (ii) f is smooth everywhere except along the boundaries, and (iii) the total length of the boundaries is minimized to prevent over-segmentation. One problem with (l), however, is that segmentation is performed using a 62nary edge term, which makes the problem highly nonconvex and hence computationally difficult. More recently, a continuous approximation to this binary edge term was suggested in [16], in which IBI is replaced by

where s now represents a continuous edge process indicating the probability of the presence of an edge at every pixel. To see why this is a continuous approximation of 181, note first that B acts as an impulse function - it has an amplitude of 1 wherever it’s more “cost effective” to incur a penalty of Y instead of incurring the penalty associated with a large gradient. Second, consider the behavior of minimizing (2) in one-dimension:

509

and let s p denote the minimizer of (3). This equation has as its Euler equation the differential equation

to the multi-channel case. Consider the energy functional given by

E

(4)

= //n[at(fi

a2(f2 - d2)2

with solutions

+ + A;lvf212(1- s)2 + +

- d ~ + )A:lVfi12(1 ~ -s ) ~

* * *

(7)

(5)

where di(i = 1, . . .,m)denotes m channels of registered MRI data. Intuitively, this equation states that each channel of data is approximated by a corresponding piecewise-smooth surface fi, and that whenever any fi contains a large gradient (i.e., a discontinuity), the edge term s is increased accordingly. The Euler equations for (7), expressed in terms of the derivative with respect to time (i.e., iteration number), are:

where only the second solution has finite energy. Thus, A p ( s p )approximates B by replacing the impulse with a function that has a width of p, a peak amplitude of 1, and a shape that decays exponentially on both sides of the peak. In precise terms, the relationship between A p ( s p )and IBI is such that A p ( s p )-+ IBI as p + 0. Using the continuous edge approximation of (2), a gradient-descent based method for segmentation of the form

E = /L((f

- d)2 + AlVfI2(1 - s

afl - - Cf,[(1--S) 2 v 2f l - Ta1( f 1 - 4 ) 87 A1 2(1 - S)(fl,SZ + f1,Sdl af2 - C f M - s)2v2f2 - 3 f 2 - d 2 ) a7

+

) ~

2(1 - S)(f2=S, was demonstrated in [17]. There are three terms in this functional - a data fidelity term, a smoothness term, and an edge penalty term. The smoothness term is based upon a first-derivative measurement of smoothness, weighted according t o the probability of the presence of an edge. This is the key notion in (6) - weighted , smoothing term is fully enforced wherby (1 - s ) ~ the ever s = 0, suppressed wherever s = 1, and varies continuously when 0 < s < 1. This mechanism prevents the problem of smoothing over discontinuities. The last term in (6) - the edge penalty term - prevents oversegmentation by (i) penalizing the presence of s, and (ii) penalizing isolated occurrences of s by enforcing a smoothness constraint on s. Because of the trade-off between increasing s whenever lVfl is large and the penalty incurred by increasing s, f and s act synergistically, and minimization of the energy functional (based upon the proper selection of weighting coefficients) yields an “optimal” balance between f and s. Although this approach to segmentation is similar in flavor t o the stochastic approach [18] and anisotropic diffusion [19], the variational approach can be readily generalized t o multi-spectral data, as we show next.

+ f2,sy)l

(8)

where the coefficients C f , and C, denote gradient descent step sizes. The boundary conditions for these Euler equations are:

where n denotes the direction normal to the image boundary aR. 5. RESULTS Figures l a through ICshow an example of registered multi-channel MRI images comprised of the 7’1, T2, and PD-weighted scans of the frontal view of a human brain, respectively’. This set of images form the inputs to our multi-channel segmentation algorithm. Figure 2 shows the s-function (i.e., edge image) resulting from equations (7) and (8). In setting the parameters of the algorithm, we noted that since 2’1 showed the best contrast between gray and white matter, the weight associated with the first channel (a1)

4. MULTI-CHANNEL SEGMENTATION

By using a different piecewise-smooth function to a p proximate each channel of data, (6) easily generalizes

Dataset No. 657, courtesy of the Massachusetts General Hospital’s Center for Morphometric Analysis

510

Figure 1. Multi-Channel MRI Data. (a) TI-weighted; (b) T’-weighted, and (c) PD-weighted images.

Figure 2. Segmentation Results. (a) An edge image showing responsiveness to dominant edges only; (b) an edge image showing greater responsiveness, and (c) the result of thinning (a). 6. CONCLUDING REMARKS

was set to 2, whereas the other two channels were given a weight of 1. The weight associated with the segmentation term, v ,can be set to a variety of values depending on the desired responsiveness of the edge process. Figure 2a shows the edge image s resulting from assigning v to a value of the same order-of-magnitude as the smoothness term J V f l J 2 ;Figure 2b shows an example of the results obtained when v is an order-ofmagnitude smaller than that of Figure 2a. Note that Figure 2a appears to do well in delineating CSF from gray- and white-matter, and that Figure 2b shows the more subtle details such as gray/white matter separation as well as the background noise. Lastly, a thinned version of Figure 2a is shown in Figure 2c.

In this paper we have put forth a mathematically rigorous formulation for the simultaneous segmentation of multi-channel MRI data. Although a thorough analysis of the efficacy of our approach has yet been performed, our preliminary results indicate significant promise. In particular, due to the ease with which the proposed approach can be generalized, we are in the process of developing and analyzing a simultaneous multi-channel and 3-D segmentation approach for processing MRI data. One issue of particular concern to us is that of using this approach on clinical data, in which the inter-slice resolution between T I ,Tz, and P D can be significantly different. Our expectation is that, because of the interpolative nature of the regularization framework, our

511

basic formulation is still valid, although the issue of inter-channel registration will need to be addressed. Lastly, a systematic analysis of the robustness, consistency, and repeatability of our segmentation approach with respect to the derived volumetric measurements is underway. We are attempting to quantify, with respect to manual segmentation, the accuracy associated with our automated segmentation scheme.

[8] Jernigan, T., G. Press, and J . Hesselink, “Methods for measuring brain morphologic features on magnetic resonance images” in Arch. Neurology, 47:27-32, Jan. 1990. [9] Gerig, G., J . Martin, R. Kininis, 0. Kubler, M. Shenton, F. Jolesz, “Automating segmentation of dual-echo MR data” in 12th Int’l Conf. on Information Processing in Medical Imaging - IPMI’91, A. Colchester and D. Hawkes (ed), SpringerVerlag, NY, 175-186, 1991.

7. ACKNOWLEDGEMENTS We wish to thank David Kennedy and Andy Worth of the Massachusetts General Hospital for the many insightful discussions and for making available their extensive MRI database. We also wish to thank Jayant Shah of Northeastern University, Mukund Desai of Draper, and Clem Karl of MIT for the many technical discussions on this topic.

[lo] Filipek, P., D. Kennedy, and V. Caviness, “Morphometric analysis of central nervous system neoplasms” in Annals of Neurology, 26:461, 1989. [ l l ] Filipek, P., D. Kennedy, V. Caviness, S. Rossnick, T . Spraggins, and P. Starewicz, “Magnetic resonance imaging-based brain morphometry: development and application to normal subjects” in Annals of Neurology, 25(1):61-67, Jan. 1989

8. REFERENCES

[12] Kennedy, D., P. Filipek, and V, Caviness, “Anatomic segmentation and volumetric calculations in nuclear magnetic resonance imaging” in IEEE Trans. Medical Imaging, 8( 1):l-7, March 1989.

[l] Filipek, P., D. Kennedy, and V. Caviness, Jr., “Neuroimaging in child neuropsychology” in Handbook of Neuropsychology, Vol. 6: Child Neuropsychology, I. Rapin and S. Segalowitz (ed), Elsevier, NY, 1992.

[13] Bertero, M., T. Poggio, and V. Torre, “111-posed problems in early vision” in Proc. IEEE, 76, 1988.

[2] Profio, A., Biomedical Engineering, Wiley, NY,

1993.

[14] Blake, A. and A. Zisserman, Visual Reconstructions, MIT Press, Cambridge, MA, 1987.

[3] Filipek, P. and D. Kennedy, “Magnetic resonance imaging: its role in the developmental disorders’’ in The Reading Brain: The Biological Basis of Dyslexia, D. Gray and D. Duane (ed), York Press, Timonium MD, 1991.

[15] Mumford, D. and J. Shah, “Boundary detection by minimizing functionals, I” in Proc. IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 1985.

[4] Worth, A., “Neural networks for automatic segmentation of magnetic resonance brain images,” Boston University Graduate School Ph. D. Dissertation, 1993.

[16] Ambrosio, L. and V. Tortorelli, “Approximation of functions depending on jumps by elliptic functionals via r-convergence” in Comm. Pure and Applied Mathematics, 43, 1990.

[5] Astari, M., J . Zito, B. Gold, J . Lieberman, M. Borenstein, and P. Herman, “Computerized volume measurement of brain structures” in Invest. Radiology, 25:798-805, July 1990.

[17] Shah, J . , “Segmentation by nonlinear diffusion, 11” in Proc. IEEE Computer Vision and Pattern Recognition Conference, IEEE, 1992.

[6] Andreasen, N., H . Nasrallah, and V. Dunn, “Structural abnormalities in the frontal system in schizophrenia” in Arch. of Gen. Psychiatry, 43~136-144,1986.

[18] Geman, S. and D. Geman, “Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images” in IEEE Trans. Pattern Analysis and Machine Intelligence, PAMI-6(6), Nov. 1984.

[7] DeLisi, L., A. Alexandropolous, N. Colter, “Reduced temporal lobe area: an MRI study of siblings with schizophrenia” in Schizophrenia Research, 1:169-170, 1988.

[19] Perona, P. and J . Malik, “Scale-space edge detection using anisotropic diffusion’’ in IEEE Trans. Pattern Analysis and Machine Intelligence, 12(7), July 1990.

512