3D Active Shape Models Integrating Robust Edge ...

3D Active Shape Models Integrating Robust Edge Identification and Statistical Shape Models Brent C. Munsell1 , Martin Styner2,3 , Pahal Dahal1 , Heather C. Hazlett3 , Jijun Tang1 , and Song Wang1 1 2

University of South Carolina, Department of Computer Science, Columbia SC University of North Carolina, Department of Computer Science, Chapel Hill NC 3 University of North Carolina, Department of Psychiatry, Chapel Hill NC

Abstract. Based on the Point Distribution Model (PDM), Active Shape Model (ASM) is an iterative algorithm used to detect structures of interest from images. However, current ASM methods are sensitive to image noise that may trap the ASM to false edges and/or lead to a structure not within the shape space defined by the PDM. Such problems are particularly serious when segmenting 3D anatomical surface structures from 3D medical images. In this paper we propose two strategies to improve the performance of 3D ASM: (a) developing a robust edge-identification algorithm to reduce the risk of detecting false edges, and (b) integrating the edge-fitting error and statistical shape model defined by a PDM into a unified cost function. We apply the proposed ASM to the challenging tasks of detecting the left hippocampus and caudate surfaces from an subset of 3D pediatric MR images and compare its performance with a recently reported atlasbased method.

1 Introduction Image segmentation is a fundamental problem in medical image analysis. An accurate segmentation of the desired anatomical structures from medical images can aid clinical researchers or physicians in the diagnosis of diseased conditions, their possible prognosis, and the effective planning of surgical interventions. For example, given a patients history of epileptic seizures, segmentation of the hippocampus allows physicians to perform meaningful volumetric comparisons to that of a hippocampus affected with mesial temporal sclerosis (MTS). Evidence of an asymmetric MTS hippocampal volume assists physicians with the diagnosis of mesial temporal lobe epilepsy, and its possible treatments [1]. Segmentation of the hippocampus from medical images has also been investigated by clinical researchers in areas such as Alzheimer disease [2, 3], amnesic syndromes [4], and schizophrenia [5–7]. Active shape model (ASM) [8] is one of the widely used methods for medical image segmentation. The primary advantage of the ASM is its capability to detect structures with a desired shape, which is usually defined by a probabilistic Point Distribution Model (PDM). However, two significant limitations of the ASM can adversely affect its segmentation performance in practice. First, ASM is an iterative algorithm that deforms an initialized shape instance (in the form of boundaries in 2D and surfaces in 3D) to fit some identified image edges. Strong noise in many medical images usually introduce

false edges that may trap the shape instance to incorrect locations. Techniques such as the construction of gray level active appearance models (AAM) for each point along the shape instance [9, 10], or statistical shape metrics based on inter-model landmarkbased distances [11] have been used to address this limitation. However, the former requires high-quality training images which can be very expensive, while the later does not consider the wealth of information provided by the parameterized shape model. The second limitation is to ensure that the structure found by the ASM bears a shape defined within the shape space of the PDM. In the traditional ASM, this is usually achiveved by directly rescaling the obtained shape back to the shape space independent of an image information. In this paper we propose two strategies to address the limitations described above. Specifically, we propose a strategy to reduce the risk of noisy edges identification and a strategy to integrate the edge-fitting error and statistical shape model defined by a PDM into a unified cost function, which is of a quadratic form and its optimum can be efficiently calculated.

2 PDM, ASM and Problem Description The statistical shape models used in active shape model (ASM) are called point distribution model (PDM) [8]. The PDM is usually constructed from a set of shape instances, e.g., a set of shape surfaces, say S1 , S2 , . . . , Sm , in the 3D case. Specifically, n corresponded landmark points v ji = (xji , yji , zji ), i = 1, 2, . . . , n are first identified from each shape surface Sj and this way, each shape surface Sj can be represented by these landmarks: a 3n-dimensional vector v j = (v j1 , v j2 , . . . , v jn )t . If these shape surfaces (and landmarks) have been normalized by removing the size, orientation, and location differences, a PDM is constructed as a multivariate Gaussian distribution N (¯ v , D), where m m 1 X 1 X v ¯= vj , D = (v j − v ¯)(v j − v ¯)t . (1) m j=1 m − 1 j=1 This probabilistic PDM models the shape-deformation space of the considered structure. Specifically, any valid landmark based shape instance u, given N (¯ v , D) can be written as u=v ¯+

3n X

bi pi ,

(2)

i=1

where pi is the i-th eigenvector of D and bi is the deformation along pi . In some prior √ literatures, |bi | ≤ 3 λi , i = 1, 2, . . . , 3n (λi is the eigenvalue along pi ) is used as a criteria to decide whether u is a valid shape in the shape space defined by PDM N (¯ v , D). Given a PDM, ASM operates on a 3D image frame as follows: (a) Placing an initial estimate of the desired landmark-based shape surface u = (u1 , u2 , . . . , un ) in the image. This is achieved by applying an global transform T (¯ v : t, s, θ) to the PDM mean shape v ¯, where t , s, and θ indicate the translation, scaling and rotation transform parameters respectively; (b) Searching along the normal directions for each of the

n landmarks in u, identifying strong image edges with high intensity-gradient magnitude; (c) Updating u to the ASM approximated shape u0 by moving all the n landmarks to their corresponding identified edges while ensuring u0 to be a valid shape in the PDM N (¯ v , D); (d) Repeating (b) and (c) to further update the shape surface until convergence. We can see that, in the above framework there are two major ASM operations which are edge identification and shape deformation. As shown in Fig. 1(a), edge identification is achieved by constructing a search profile along the normal direction of each landmark ui , where i = 1, 2, · · · , n. Specifically, 2k + 1 locations are sampled along this profile and the magnitude of the voxel intensity gradient is calculated at each sampled location. The profile location with the strongest intensity-gradient magnitude αi is then chosen as the identified edge point. This strongest edge point can then be written as u ˆi = ui + αi ni , where ni is the unit normal vector at ui . In shape deformation, we need to deform u to u0 , which updates both the global transform T (¯ v : t, s, θ) and the local-transform parameters b = (b1 , b2 , . . . , b3n )t shown in Eq. (2). This shape deformation seeks a balance between two goals: (a) u0 fits better to the identified edges u ˆ = (ˆ u1 , u ˆ2 , . . . , u ˆn )t , and (b) u0 still bears a valid shape in the PDM N (¯ v , D). In some current 3D ASM implementations, the deformation from u to u0 is simply achieved by modifying the identified edges u ˆ√to fit the given PDM: representing u ˆ in the form of Eq. (2) and if any b is larger than 3 λi or smaller i √ √ √ than −3 λi , bi is rescaled [8] to fit within the limits 3 λi or −3 λi of the PDM shape space.

b i=

b i=

3

λ

α=k α=2 α=1

λ

ui

0

pi ni

b i =

−3

α=−1 α=−2 α= −k

(a)

(b)

Fig. 1. An illustration of the edge identification and shape deformation in ASM. (a) Search for strong edges along the profile of landmark ui . The profile is uniformly sampled into 2k+1 points symmetric over ui for calculating intensity-gradient magnitude. √ √(b) A step of shape deformation by rescaling the parameter bi that is out of the range [−3 λi , 3 λi ].

However, two problems exist in the above ASM algorithm. First, the strongest edge with the largest intensity gradient magnitude along the profile might not be the true edge of the desired structure. In practice, the strongest edges could be image noise or other neighboring structures. Previous research has shown that incorrect edge identification may result in very poor performance of the ASM [11]. Second, direct modification of u ˆ for shape deformation may also result in poor ASM performance, especially when the identified edges u ˆ contains much noise and the initial estimate of the shape surface

is relatively far from the desired structure. In the worst case, such modification may keep the shape surface around the initial location without any further deformation. In the next section, we presented two strategies to address these two problems.

3 Proposed Method 3.1 Robust Edge Identification Instead of independently detecting the strongest edge u ˆi for each landmark ui along the normal direction, we consider a neighboring-consistency property to improve the robustness of the edge identification: if two landmarks ui and uj are very close to each other, it is very likely that the detected edges u ˆi and u ˆj are close as well. Particularly, our proposed edge-identification algorithm consists of three steps: First, for each landmark ui , we find all its m(i) neighboring landmarks defined on the triangular surface mesh of u, as shown in Fig. 2(a). Let dij be the Euclidean distance between the landmark ui and its j-th neighbor, where i = 1, 2, . . . , n and j = 1, 2, . . . , m(i). We can see that dij describes the spatial proximity from ui to its neighbors. Later we will apply dij to weight the edge-identification consistency between ui and its j-th neighbor. neighbor 4 d i4 d i3

neighbor 1

1

ui

st

2

nd

3

rd

−2

8

4

neighbor 1

8

9

neighbor 2

9

8

neighbor 3

9

neighbor 4

8

4

th

5

th

1

ui

st

2

nd

3

rd

−2

8

4

neighbor 1

8

9

neighbor 2

9

8

4

neighbor 3

9

4

neighbor 4

8

7

2

7

6

5

6

5

4

8

6

5

6

−2

5

4

th

5

th

7

2

7

6

5

6

5

4

8

6

5

4

6

−2

5

4

d i1

ui d i2

neighbor 3

neighbor 2

k= 1

(a)

2

3

(b)

4

5

k= 1

2

3

4

5

(c)

Fig. 2. An illustration of the robust edge identification algorithm. (a) Identify the neighbors of ui and calculate the their distances. (b) The first five strongest edges identified for ui and its five neighbors. The elements show the offset values of the identified edges along their respective profiles. (c) The optimal offset value α∗i = 8 minimizes the edge cost and we therefore choose uˆi = ui + 8ni .

Second, for landmark ui and each of its neighboring landmarks, we construct a vector e that contains the locations of the first several strongest edges in a descending order. For example, ei in Fig. 2(b) contains the first five strongest edges along the profile of ui , where ei1 = −2 indicates that the strongest edge is located at ui − 2ni and ei5 = 2 indicates that the fifth strongest edge is located at ui + 2ni . We further define a weight function to quantify the strength of each edge by its rank: the k-th strongest edge is simply set to have a weight of k in this paper. Based on the information collected in the two steps above, the third step aims to more robustly determine the most likely edge u ˆi for landmark ui . In this paper, we formulate the problem as identifying the optimal offset αi∗ along the normal direction

that shows the best consistency for landmark ui and all its neighbors. Basically, for each offset value eik , we find the rank in j-th neighbor of ui and denote this rank to be r(i, j, k). The edge cost corresponding to the offset value eik is defined to be m(i)

φ(eik ) = k

X

r(i, j, k) · dij .

j=1

We finally choose αi∗ to be the offset value eik that minimizes the edge cost φ(eik ). Figure 2(c) gives an example of detecting the most likely edges for ui by locating the offset that minimizes this new edge cost. We can see that, in this example, the most likely edge identified for ui is different from the highest-ranked edge (the one with the largest intensity gradient magnitude) along the normal direction ni . 3.2 Shape Deformation Based on a Unified Cost Function Each iteration of shape deformation aims to update the shape surface u to u0 by minimizing the fitting error to the identified edges u ˆ while keeping u0 within the shape space of the given PDM. As discussed before, such deformation consists of updating both a global transform T (t, s, θ) and a local transform described by parameters b. For the global transform, we simply find the T (t, s, θ) between the PDM mean shape vector v ¯ and the identified edges u ˆ [8]. For local transform, we find u0 that minimizes a cost function n X

ˆi k2 + β(u0 − T (¯ v : t, s, θ))t DT−1 (u0 − T (¯ v : t, s, θ)) ku0i − u

(3)

i=1

where u0i is constrained to be along the normal direction of T (ui : t, s, θ), i.e., the globally transformed version of ui . DT−1 is the inverse of DT which is the updated covariance matrix after the global transform, i.e. DT = T ·D·T t . We can see that finding the optimal solution for local transform is a typical quadratic programming problem, which can be solved efficiently. In the above cost function, β > 0 is a coefficient that balances the contributions from the detected edges and the PDM.

4 Experiments We applied the proposed ASM by segmenting the left hippocampus and caudate surfaces in MRI images from an pediatric autism study [12] from which we randomly selected a subset of 10 IR-prepped SPGR T1 weighted MRI brain scan images. The resolution of an image is 256 × 256 × 192 with both the in-plane and inter-slice voxel spacing set at 1.0mm. The hippocampus PDM was constructed using 42 adult hippocampus surfaces, each of which contains 642 identified corresponded landmarks. These corresponded landmarks are identified by using a minimum-description-length (MDL) implementation [13]. The caudate PDM was constructed using 85 caudate surfaces, each of which contains 742 corresponded landmarks. The corresponded caudate

landmarks are identified by using the spherical-harmonic-descriptors (SPHARM) implementation [14]. The initial estimate and placement of the hippocampus and caudate surfaces in these 10 pediatric images are manually performed and visually verified using the InsightSNAP software. Nine consecutive points with one voxel intervals were uniformly sampled along the normal direction of each landmark to locate the strong edges. The first five strongest edges are then picked for robust edge identification, as detailed in Section 3.1. The ASM is ran until convergence or when an maximum of 10 iterations is reached using an balance coefficient of β = 5 × 10−5 . Convergence is achieved when both the global and local transform vary minimally. We evaluate the segmentation results by comparing them with expertly segmented left hippocampus and caudate surfaces. The hippocampus was segmented using an semi-automatic method [15], and the caudate was segmented either via manual outlining, or semi-automatic geodesic curve evolution [16]. We measure their consistency using the Pearson correlation and dice coefficients [17]. The Pearson correlation coefficient cP measures the volumetric correlation between manual and ASM segmentations on all 10 images, while the dice coefficient cD assesses the structure similarity by measuring their volumetric overlap. The results are show in Table.1. In this table, we also include the segmentation performance of the a recently reported atlas-based method [17], which is based on 20 pediatric images from the same autistic study. Table 1. Segmentation performances of the proposed method and a recent atlas-based method compared against expertly segmented hippocampus and caudate data. Left Hippocampus Left Caudate Segmentation Methods cP cD cP cD Proposed ASM 0.7373 0.7792 0.8452 0.8318 Atlas-based Method [17] 0.1800 0.7500 0.7500 0.8400

Although the performance of the atlas-based method is obtained from more images that the proposed method, we can see that the proposed ASM shows a comparable cD performance to and better cP performance than those of the atlas-based method. Fig. 3 shows the initially-placed shape surfaces and the ones segmented by the proposed ASM in the coronal, transverse and sagittal planes. Lastly, the proposed ASM does require an reasonable initial PDM placement but is not very sensitive.

5 Conclusion In this paper, we presented two new strategies to address the limitations of the current 3D ASM methods. The preliminary study shows promising results: the proposed 3D ASM produces comparable or even better segmentation results than a recently reported atlas-based method. In the future, we plan to further investigate the sensitivity of the initial shape placement and apply the proposed ASM on segmenting other anatomic structures.

Fig. 3. Row 1: Initially placed left caudate. Row 2: Left caudate segmented by the proposed ASM. Row 3: Initially placed left hippocampus. Row 4: Left hippocampus segmented by the proposed ASM. Columns 1, 2 and 3 show the structures in the coronal, transverse, and sagittal planes respectfully.

Acknowledgement This work was funded, in part, by NSF-EIA-0312861.

References 1. Hogan, R.E., Mark, K.E., Wang, L., Joshi, S., Miller, M.I., Bucholz, R.D.: Mesial temporal sclerosis and temporal lobe epilepsy: Mr imaging deformation-based segmentation of the hippocampus in five patients. Radiology 216 (2000) 291–297 2. Jack, C.R., Petersen, R.C., Xu, Y.: Rate of medial temporal lobe atrophy in typical aging and alzheimers disease. Neurology 51 (1998) 993–999 3. Frisoni, G.B., Laakso, M.P., Beltramello, A.: Hippocampal and entorinhal cortex atrophy in frontotemporal dementia and alzheimers disease. Neurology 52 (1999) 91–100 4. Press, G.A., Amaral, D.G., Squire, L.R.: Hippo-campal abnormalities in amnesic patients revealed by high-resolution magnetic resonance imaging. Nature 341 (1989) 54–57 5. Csernansky, J.G., Joshi, S., Wang, L.: Hippocampal morphometry in schizo-phrenia by high dimensional brain mapping. Proc Natl Acad Sci USA 95 (1998) 11406–11411 6. Shenton, M.E., Kikinis, R., Jolesz, F.A.: Abnormalities of the left temporal lobe and thought disorder in schizophrenia: a quantitative magnetic resonance imaging study. New England Journal of Medicine 327 (1992) 604–612 7. Shenton, M.E., Gerig, G., McCarley, R.W., Szekely, G., Kikinis, R.: Amygdala - hippocampal shape differences in schizophrenia: the application of 3d shape models to volumetric mr data. Psychiatry Research Neuroimaging 115 (2002) 15–35 8. Cootes, T., Taylor, C., Cooper, D., Graham, J.: Active shape models - their training and application. Computer Vision and Image Understanding 61(1) (1995) 38–59 9. Cootes, T., Taylor, C.: Statistical models of appearance for computer vision. Technical Report, University of Manchester, Imaging Science and Biomedical Engineering, http://www.wiau.man.ac.uk (1999) 10. Heimann, T., Wolf, I., , Meinzer, H.P.: Active shape models for a fully automated 3d segmentation of the liver -an evaluation on clinical data. In: International Conference on Medical Image Computing and Computer Assisted Intervention. Volume 2. (2006) 41–48 11. Lekadir, K., Merrifield, R., Yang, G.Z.: Outlier detection and handling for robust 3-d active shape models search. IEEE Transactions on Medical Imaging 26 (2007) 212–222 12. Hazlett, H.C., Poe, M.D., Gerig, G., Smith, R.G., Piven, J.: Cortical gray and white brain tissue volume in adolescents and adults with autism. Biological Psychiatry 59 (2006) 1–96 13. Heimann, T., Wolf, I., Williams, T., Meinzer, H.P.: 3d active shape models using gradient descent optimization of description length. In: Information Processing in Medical Imaging Conference. (2005) 566–577 14. Styner, M., Lieberman, J.A., McClure, R.K., Weingberger, D.R., Jones, D.W., Gerig, G.: Morphometric analysis of lateral ventricles in schizophrenia and healthy controls regarding genetic and disease-specific factors. Proceedings of the National Academy of Science 102 (2005) 4872–4877 15. Styner, M., Xu, S.C., El-Sayed, M., Gering, G.: Correspondence evaluation in local shape analysis and structural subdivision. In: IEEE Symposium on Biomedical Imaging. (2007) 16. Yushkevich, P.A., Piven, J., Hazlett, H.C., Gimpel, R.S., Ho, S., Gee, J.C., Gerig, G.: User guided 3d active contour segmentation of anatomical structures: Significantly improved efficiency and reliablility. NeuroImage 31 (2006) 1116–1128 17. Gouttarda, S., Styner, M., Joshi, S., Smith, R.G., Cody, H., Gerig, G.: Subcortical structure segmentation using probabilistic atlas priors. In: Proceedings of the SPIE Medical Imaging Conference. (2007)