The complex and variable nature of brain anatomy makes ... The active surface template uses an energy minimization scheme to find a globally consistent ...
Segmentation of the brain from 3D MRI using a hierarchical active surface template J. W. Snell,¶† M. B. Merickel,†§ J. M. Ortega,‡ J. C. Goble,¶† J. R. Brookeman§† and N. F. Kassell¶ †Department of Biomedical Engineering, University of Virginia, Charlottesville, VA 22908, U.S.A. ‡Department of Computer Science, University of Virginia, Charlottesville, VA, 22908, U.S.A. §Department of Radiology, University of Virginia, Charlottesville, VA 22908, U.S.A. ¶Department of Neurosurgery, University of Virginia, Charlottesville, VA 22908, U.S.A.
ABSTRACT The accurate segmentation of the brain from three-dimensional medical imagery is important as the basis for visualization, morphometry, surgical planning and intraoperative navigation. The complex and variable nature of brain anatomy makes recognition of the brain boundaries a difficult problem and frustrates segmentation schemes based solely on local image features. We have developed a deformable surface model of the brain as a mechanism for utilizing a priori anatomical knowledge in the segmentation process. The active surface template uses an energy minimization scheme to find a globally consistent surface configuration given a set of potentially ambiguous image features. Solution of the entire 3D problem at once produces superior results to those achieved using a slice by slice approach. We have achieved good results with MR image volumes of both normal and abnormal subjects. Evaluation of the segmentation results has been performed using cadaver studies.
1. INTRODUCTION The rapid development of three-dimensional medical imaging technologies has provided the ability to examine the interior structure and function of the human body in great detail. While the acquisition of these images is well developed, the methodology for visualizing and quantifying volumetric imagery remains immature. The objects of interest must be reliably identified and delimited in order to make volumetric measurements and to generate 3D perspective views. This paper deals with the partition, or segmentation, of 3D medical images into meaningful volumes of interest using a knowledge-based approach. An automatic segmentation procedure is desirable because the size and complexity of volumetric imagery makes manual delineation of structures both tedious and unreliable. The method presented employs a priori anatomical knowledge in the form of generalized deformable surface models in order to reduce required operator interaction and to improve segmentation results. Existing segmentation techniques which are based on local image features are not able to handle ambiguities caused by noisy or incomplete object boundary information. Therefore, user interaction is required by these methods in order to guide the segmentation process. By the incorporation of anatomical knowledge, systems may be designed which reduce or eliminate the need for interactive segmentation approaches. However, the representation of a priori knowledge must be flexible enough to accommodate the variations an shape and size which occur between individuals. The deformable active surface models described in this paper provide a mechanism for representing structure in a general way. Normal anatomy need not be assumed and the template surfaces may adaptively conform to a range of object sizes and shapes. In this way, even images exhibiting structural abnormality may be successfully segmented. The surfaces are deformed by forces which are derived from image features. Therefore, active surfaces serve to unify high-level knowledge and assumptions with low-level image information. In this way, segmentation and recognition of objects occur simultaneously instead of in discrete processing stages. The surface templates are instantiated within a standardized proportional coordinate system which is defined by userspecified landmarks. These landmarks are the only parameters input by the user. This allows a single generalized model to be coarsely registered with a corresponding image object and then adaptively deformed by the image data. Composite models may be constructed for cases where an object is naturally described as a collection of connected subparts. An application has been developed using the active surface template method for the segmentation of the brain from 3D MR images of the head. Comparisons with cadaver studies have produced encouraging results and suggest applications to diagnosis, surgical planning/simulation and volumetric measurement studies. The method is generally applicable to any type of 3D imagery where models of object surface structure can be constructed. A preliminary version of this work was presented at the SPIE ‘93 Medical Imaging Conference1.
2. ACTIVE SURFACE TEMPLATES The knowledge representation mechanism presented here is closely related to deformable templates2 and active contour models (“snakes”)3. The proposed surface models are based on the physically-based deformable models described in (4-7). The active surface is the three-dimensional analog of the active contour. It is an energy minimizing spline which is characterized by a set of intrinsic and extrinsic constraints. The intrinsic constraints govern the material properties of the surface, while the extrinsic constraints link the surface to the image data through “forces” which deform it. The equation of motion is given by ∂ v ∂ v ∂ v ∂ v ∂ v − w1 + + +2 4 2 2 4 ∂x2 ∂y2 ∂x ∂y ∂y ∂x 2
2
4
4
4
w2
= f ( v) ,
(1)
where w 1 and w 2 determine the elasticity and stiffness of the surface respectively. Currently, these coefficients must be determined empirically. The surface is linked to the image data by the extrinsic forces defined as f ( v) =
δP . δv
(2)
The image potential function P(v) has been defined in [3] as P ( v ) = w line I ( v ) − w edge ∇I ( v ) . 2
(3)
where I(v) is the image intensity function. The coefficient wline controls the attraction of the surface to ridges or valleys in the image intensity function while wedge controls the attraction of the surface to areas of high intensity gradient(edges). We have developed an extrinsic force formulation based on mathematical morphology in order to keep surfaces from becoming trapped in the interior of the brain. Our image potential function is based on a vector distance transform(VDT)8 which has the effect of reducing unwanted local minima which typically occur in Equation (3). An intensity threshold is used to divide the image into foreground and background regions from which the VDT is calculated. The gradient of the VDT always points toward the nearest foreground boundary and therefore tends to restore surfaces from the interior of the foreground. In order to solve the system given by Equation (1), the domain is discretized with equal spacing h between the grid points and the derivatives of the Laplacian and biharmonic parts of Equation (1) are approximated by finite differences 9. The system may written in matrix form as
AV = F
(4)
and because A is large and sparse it may be efficiently solved by iterative methods. We have used the simple Gauss-Seidel iterative method10, lagging the nonlinear terms in F. The iteration process is stopped when the maximum node movement falls below some threshold, which is typically on the order of the voxel size. 70000
Surface External Energy
60000
50000
40000
30000
20000
10
20
30
40
50
60
Iteration
Figure 1 - Shown above is a plot of the total surface energy of the brain surface model vs. iteration. The surface deforms in such a way as to take the path of steepest decent on the energy hypersurface.
The total external energy experienced by the surface model decreases monotonically with each iteration as shown in Figure (1). This is due to the fact that the forces which deform the surface always point in the direction of steepest descent on the energy hypersurface as stated by Equation (2).
2.1 Surface Model Construction
Figure 2 - Wireframe of the hierarchical brain surface model. Only one hemisphere is shown for clarity. The model is composed of five subsurfaces which are attached to each other by enforcing common boundary conditions.
Our preliminary brain surface template is formed from a series of manually traced contours specifying the surface at various elevations of a normal brain These contours are then interpolated to yield a continuous surface description. Surface descriptions were obtained in this way for five major structures: the left and right cerebrum, the left and right cerebellum and the brain stem. Together, these surfaces describe the entire outer surface of the brain as shown in Figure (2). The hierarchical nature of the model allows the computation to be broken into discrete subproblems which can be solved in parallel. The boundary conditions communicated between subsurfaces characterize the behavior of the model along these seams. For instance, the crease along the interhemispheric fissure is allowed to form by enforcing only C0 continuity along the boundary between left and right cerebral subsurfaces.
2.2 Surface Initialization
Figure 3 - The image volume is brought into a standard coordinate system by indicating the interhemispheric fissure and the anterior and posterior commissures (AC, PC).
We use the Talairach proportional coordinate system as a mechanism for warping the model surfaces into coarse registration with an individual image volume11. The orientation of the coordinate system is defined by the interhemispheric fissure and the anterior and posterior commissures, while the extent of the brain in each direction defines the scaling parameters as shown in
Figure (3). The brain surface model is defined in this normalized coordinate system so that once the landmarks and extrema are identified, the warped model provides an approximate initialization for the active surface template. Once initialized, the active surface template is iterated until it reaches an equilibrium configuration.
3. RESULTS We investigated the effectiveness of the segmentation procedure on 3-D magnetic resonance images of both cadavers and live subjects. This group of images included examples of both normal and abnormal anatomy. The three-dimensional magnetic resonance data sets used in our experiments were acquired with the 3D MP-RAGE pulse sequence12 on a 1.5 T Siemens Magnetom. This sequence employed a 180 degree inversion pulse to produce a strongly T1-weighted image with an inversion time of 500 ms. The 3D image was acquired with a matrix of 256x256x128 to give a voxel size of 1.0x1.0x1.3 mm. The acquisition time was 11 minutes. The cerebrospinal fluid (CSF) in the T1-weighted images appears dark and the brain appears bright. The brain and spinal cord are completely surrounded by CSF, so in our initial experiments we used the conventional intensity-based energy functional given by Equation (3). We set wedge equal to zero and wline such that the surface is attracted toward neighboring dark areas of the image volume. This approach causes the active surface to move from its initial position determined by the warped model into the nearest CSF space. While the resulting surface does not correspond to the actual brain surface, it does effectively separate the brain from other structures in the head because it lies within the CSF space. Voxels outside the surface are removed from the volume of interest and a subsequent thresholding operation eliminates any CSF voxels which are enclosed by the surface, leaving the relatively bright brain voxels. The intensity-based energy functional of Equation (3) can cause the surfaces to become trapped in the interior of the brain. This occurs because this function only produces forces in the immediate neighborhood of object boundaries. If part of a surface should be initialized such that it falls within the brain volume, there are no forces there to restore it to the brain surface. In order to counter this effect we employed the VDT which maps the euclidean distance to the nearest foreground boundary onto each voxel position. This function has non-zero gradient everywhere in the foreground and thus acts to push model surfaces toward the surface of the brain if they should fall within the brain volume.
Figure 4 - Comparison of dissection photograph (left) and surface shaded segmentation result (right). Volume of the dissected brain was measured by immersion to be 1172 ml. The volume of the brain segmentation result was 1104 ml.
The active surface template method was applied to 3D MR images of a cadaver and the resulting segmented brain volume was compared with photographs obtained after dissection. As shown in Figure (4), the photograph of the cortical surface correlates well with a surface shaded rendering of the segmentation result. Also, volume estimates for the cadaver brain and the segmented image agreed within 10%. The volume of the dissected brain was measured by immersion to be 1172 ml. The volume of the segmented brain image was 1104 ml. In an effort to fairly evaluate the active surface template segmentation method, a second cadaver experiment was
performed. The brain was imaged both within the whole head and after careful dissection. The brain was segmented from the post-dissection image volume by simply setting a threshold to remove the voxels corresponding to air. Since the brain was physically “segmented” from the head and then imaged, the post-dissection image serves as a “gold standard” for evaluating the performance of the active surface segmentation algorithm on the pre-dissection image.
Figure 5 - Comparison of pre and post-dissection segmentation results. The pre-dissection image was segmented by the active surface template method(left). A simple thresholding procedure was used to segment the post-dissection image(right).
The brain volume measured from the post-dissection image was 981.3 ml, while that measured by the active surface segmentation of the pre-dissection image was 1036.6 ml. Comparison of the volume rendered segmentation results of both cases is shown in Figure (5). Available image volumes of living subjects have also been successfully segmented. These include images exhibiting both normal and abnormal brain structure. A typical rendered result is shown in Figure (6). The active surface algorithm segments not only the gross brain, but subvolumes corresponding to each of the subsurfaces. This allows volume rendering software to display each substructure with different surface properties (color, opacity, specularity, cut plane permeability, etc.). We are in the process of extending the approach with a more complete and realistic model. A neuroanatomical atlas in digital form could serve as the knowledge base for a more detailed model than the one used currently(13,14). The gross segmentation of the brain could then be subsequently refined by the segmentation of progressively smaller structures.
Figure 6 - Typical segmentation result on a normal volunteer. The active surface template labels the left/right cerebrum, left/right cerebellum and brain stem/spinal cord independently. Operations can be done selectively on each label. The image above right shows the left cerebrum made permeable to the cut plane operation.
4. DISCUSSION The active surface template segmentation algorithm presented here is based on the central idea that a priori knowledge of anatomy is required to facilitate the robust segmentation of complex objects in medical imagery. Additionally, the representation of this knowledge must be flexible enough to accommodate a range of anatomical morphology. Existing medical image segmentation schemes typically use relatively local image information in bottom-up strategies. While knowledge is also used, it is implicitly encoded in the form of programming heuristics. Those approaches which do use knowledge-based methods tend to use rigid representation of global shape which limits them to a narrow subset of possible cases. The system we have developed encodes anatomical knowledge in the form of deformable surface templates. These models place expectations on object shape and surface smoothness in order to constrain the segmentation process, yet retain freedom to deform in response to local images features. This approach is related to active contour approaches which take a slice-by-slice approach to 3D segmentation15. Our approach is to solve the segmentation problem in a fully three-dimensional way, instead of using a 2D approach. This allows all of the image information and geometric constraints to be employed at once. The active surface representation is based on the active contour technique and so exhibits the same regularized behavior. Areas of poor contrast or noise are bridged by the enforced smoothness constraints. On the other hand, the smoothness constraints can sometimes be a liability when the objects to be segmented exhibit areas of high curvature. This limitation has not proven to be a problem in separating the brain from the head. The active surface technique is efficient because it selectively interrogates the image data instead of searching the entire image exhaustively. Surface deformation is affected by local image features only. This makes it unnecessary to process the entire image volume because image features are only required in those areas actually traversed by the surface. Effective caching schemes
can greatly reduce processing time and storage requirements for image features.
Figure 7 - Simulated craniectomy allows the underlying cortical surface to be inspected in order to more effectively evaluate burr hole placement.
The application of active surface templates to brain segmentation suggests many possible applications. These include both visualization and volumetric analysis. Direct 3D visualization of the brain and surrounding structures is an effective way to communicate complex spatial relationships. This kind of visualization also enables more interactive and intuitive planning of surgical procedures. Figure (7) demonstrates the planning of a craniectomy. The cortical surface can be previewed in order to more accurately plan burr hole placement. After further experimentation, we hope to show that accurate and repeatable volumetric measurements of brain volumes can also be made with this technique. Due to the limited user interaction required and the low computation time (~4 min on an HP Apollo 700 series workstation), this type 3D segmentation becomes clinically useful as the basis for visualization, volumetric analysis and surgical planning/navigation.
5. REFERENCES 1.
Snell, J.W. et al, “Model-Based Segmentation of the Brain from 3-D MRI Using Active Surfaces,” Proceedings of the SPIE Conference on Medical Imaging, Newport Beach, (1993).
2.
Blake, A. and A. Yuille, Active Vision, MIT Press, Cambridge, MA, (1992).
3.
Kass, M., A. Witkin and D. Terzopoulos, “Snakes: Active Contour Models,” Int’l J. of Computer Vision, 1(4), 321-331, (1987).
4.
Terzopoulos, D., A. Witkin, and M. Kass., “Constraints on Deformable Models: Recovering 3D Shape and Nonrigid Motion,” Artificial Intelligence, 36, 91-123, (1988a).
5.
Terzopoulos, D., and A. Witkin., “Physically Based Models with Rigid and Deformable Components,” IEEE CGA, 41-51, (1988b).
6.
Terzopoulos, D. and D. Metaxas., “Dynamic 3D Models with Local and Global Deformations: Deformable Superquadrics,” IEEE Trans. PAMI, 13(7), 703-714, (1991).
7.
Terzopoulos, D., “Regularization of Inverse Visual Problems Involving Discontinuities,” PAMI, 8, 413-424, (1986).
8.
Mullikin, JC., “The Vector Distance Transform in Two and Three Dimensions,” CVGIP, 53, 76-87, (1991).
9.
Lapidus, L. and Pinder, G., Numerical Solution of Partial Differential Equations in Science and Engineering, John Wiley & Sons, (1982).
10. Golub, G. and J. Ortega, Scientific Computing and Differential Equations, Academic Press, (1991).
11. Talairach, J. and P. Tournoux., Co-Planar Sterotaxic Atlas of the Human Brain - 3-Dimensional Proportional System: An Approach to Cerebral Imaging, G.T. Verlag, New York, (1988). 12. Mugler, III, J.P. and J.R. Brookeman, “Three-Dimensional Magnetization-Prepared Rapid Gradient-Echo Imaging (3D MPRAGE),” Mag Res Med, 15, 152-157, (1990). 13. Greitz, T, C.C. Bohm, S. Holte and L. Eriksson, “A Computerized Brain Atlas: Construction, Anatomical Content, and Some Applications,” J. Comp. Ass’t Tomography, 15(1), 26-38, (1991). 14. Höhne, K.H., bomans, M.Riemer, R. Shubert et al, “A Volume-based Anatomical Atlas,” IEEE CGA, 12(4), 72-78, (1992). 15. Grzeszczuk, R.P. and D.N. Levin, “Segmenting Images by Stochastic contour Optimization,” Proceedings of the Society of Magnetic Resonance in Medicine, New York, (1993).
