Unsupervised Learning of Shape Complexity: Application to Brain Development Ahmed Serag1, , Ioannis S. Gousias1,2 , Antonios Makropoulos2, Paul Aljabar3 , Joseph V. Hajnal3 , James P. Boardman4 , Serena J. Counsell3 , and Daniel Rueckert1 1
4
Department of Computing, Imperial College London, UK
[email protected] 2 Centre for the Developing brain, Imperial College London, UK 3 Centre for the Developing brain, Kings College London, UK Simpson Centre for Reproductive Health, Royal Infirmary of Edinburgh, UK
Abstract. This paper presents a framework for unsupervised learning of shape complexity in the developing brain. It learns the complexity in different brain structures by applying several shape complexity measures to each individual structure, and then using feature selection to select the measures that best describe the changes in complexity of each structure. Then, feature selection is applied again to assign a score to each structure, in order to find which structure can be a good biomarker of brain development. This study was carried out using T2-weighted MR images from 224 premature neonates (the age range at the time of scan was 26.7 to 44.86 weeks post-menstrual age). The advantage of the proposed framework is that one can extract as many ROIs as desired, and the framework automatically finds the ones which can be used as good biomarkers. However, the example application focuses on neonatal brain image data, the proposed framework for combining information from multiple measures may be applied more generally to other populations and other forms of imaging data. Keywords: Shape complexity, dimension reduction, feature selection, brain development, biomarker extraction, neonatal, MRI.
1
Introduction
Machine learning techniques, particularly ’dimensionality reduction’, play an important role for converting data from a high to a low-dimensional representation more suitable for further processing steps such as regression or biomarker extraction. This category of methods have recently been popular in the field of medical image analysis where a typical structural magnetic resonance (MR) image, for instance, which contains around a million voxels may be well represented in a significantly lower dimension space. For instance, typical shape complexity
Corresponding author.
S. Durrleman et al. (Eds.): STIA 2012, LNCS 7570, pp. 88–99, 2012. c Springer-Verlag Berlin Heidelberg 2012
Unsupervised Learning of Shape Complexity
89
analysis methods in the brain summarize shape complexity, like cortical folding, by single/few numbers. One class of descriptors quantify surface complexity alone. For example, Gyrification Index (GI) [1] measures the ratio of the lengths of a planar curve to its envelope, Convexity Ratio (CR) [2] measures the ratio of the area of the surface to the area of the convex hull of the surface, and FD [3] captures the rate of increase in surface area over multiscale representations of the surface. Isoperimetric ratio (IPR) [2] has been used to measure the ratio of the surface area to the two-third power of the volume enclosed by the surface. Also, Average curvedness (AC) [4] measures the deviation of the surface from a plane. Some folding descriptors capture a part of the shape complexity spectrum by summing up specific measures for all surface patches. For example, Average Shape index (AS) [4] sums up shape measures, Gaussian Curvature Norm (GCN) [2] sums up degrees of hemisphericity and saddle-likeness, Mean Curvature Norm (MCN) [2] sums up degrees of hemisphericity and cylindricity, and Intrinsic Curvature Index (ICI) [5] sums up degrees of hemisphericity. While previous studies of shape complexity showed that the degree of cortical folding is highly correlated with brain development, the degree of complexity associated with developmental changes in other brain structures may provide an additional marker of development. For example, there is a significant effect of prematurity on thalamic development [6, 7], hence the shape complexity of the thalamus could be a useful marker of brain development. In this work, we aim to complement the analysis of the complexity of the cortical shape in early brain development with the study of the complexity of different brain structures, which can offer an additional marker for mapping neurodevelopmental changes. The proposed framework learns the complexity of different brain structures by applying different shape complexity measures to each individual structure, and then using feature selection to select the measures that best describe the changes in complexity of each structure. The feature selection algorithm, Laplacian Score (LS), is used to weigh applied measures per structure to embed them into low-dimensional representation. Then, LS is applied again to assign a score to each structure, in order to find structures that can be good biomarkers of brain development. The framework is illustrated in Figure 1.
2 2.1
Materials and Methods Subjects and Image Acquisition
This study was carried out using T2 weighted MR images from 224 premature neonates. The age range at the time of scan was 26.7 to 44.86 weeks postmenstrual age (PMA), with mean and standard deviation of 37.5 ± 4.8 weeks. All subjects were born prematurely, with mean age at birth 29.2 ± 2.7, range 24.1 − 35.2 weeks PMA. The images were acquired on 3T Philips Intera system with the following parameters: T2-weighted fast spin-echo (FSE), TR = 8700 ms, TE = 160 ms, flip angle = 90◦ , acquisition plane = axial, voxel size = 1.15 x 1.18 x 2 mm, FOV = 220 mm and acquired matrix = 192 x 186.
90
A. Serag et al. Feature Extraction
Shape descriptor 1
Shape 1 Shape descriptor m Shape descriptor 1
Shape 2 Shape descriptor m Shape descriptor 1
Feature Selection
Biomarker Extraction
Shape descriptor m Shape descriptor 1
Shape descriptor m Shape descriptor 1
Shape s Shape descriptor m
Fig. 1. Proposed Framework
2.2
Preprocessing
A first step in the proposed framework is the masking of non-brain tissues in all images in the database. The Brain Extraction Tool (BET) is used [8] to remove all non-brain tissue in each image. Then, images were corrected for intensity inhomogeneity using the N4 algorithm [9]. Twenty brain images at term equivalent age (39.39-44.86 weeks PMA) from the database were manually segmented by an expert (I.S.G.) into 50 ROI [10]. The 50 labels were propagated to the rest of the images of the database using the the spatio-temporal atlas 1 constructed in [11] and the LISA algorithm [12]. Then, propagated labels were combined by vote fusion [13]. On the assumption that paired structures that appear in both the left and right hemispheres have the same characteristics, we applied a common label to each pair. This step reduced the initial segmentation from 50 to 26 ROIs (see Tab. 1), since brainstem and corpus callosum are unpaired structures. To make sure that the our measurements are not affected by volumetric changes, every segmented structure from all images was affinely aligned to the corresponding segmented structure in a reference template (the 37th week template of the atlas from [11]). This reference was selected as the target age to reduce the degree of deformation required from the other age groups as it lies in the median of the age range for the group. For the purpose of analyzing the changes of shape complexity over time, we generated a triangle mesh of each structure surface with the Marching Cubes algorithm [14]. 1
http://www.brain-development.org
Unsupervised Learning of Shape Complexity
91
Table 1. List of manually segmented ROIs Index Anatomical Description of ROI 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
2.3
Hippocampus Amygdala Anterior temporal lobe, medial part Anterior temporal lobe, lateral part Gyri parahippocampalis et ambiens anterior part Superior temporal gyrus, middle part Medial and inferior temporal gyri anterior part Lateral occipitotemporal gyrus, gyrus fusiformis anterior part Cerebellum Brainstem, spans the midline Insula Occipital lobe Gyri parahippocampalis et ambiens posterior part Lateral occipitotemporal gyrus, gyrus fusiformis posterior part Medial and inferior temporal gyri posterior part Superior temporal gyrus, posterior part Cingulate gyrus, anterior part Cingulate gyrus, posterior part Frontal lobe Parietal lobe Caudate nucleus Thalamus Subthalamic nucleus Lentiform Nucleus Corpus Callosum Lateral Ventricle
Feature Extraction
This study employs six of previously developed complexity measures, and these measures are applied to different brain structures, in particular the 26 ROI shown in Tab. 1. This means that each ROI to be described by six features/measures. These measures are described as follows: Shape Index (SI). The SI characterizes the deviation of the shape of an object from a sphere. The SI of a sphere is one. For every other shape it is greater than one (see Fi.g 2). The SI is defined as: SI =
√ A √ 3 V
(1) N with A being the area of the surface, and V being the volume. The normalization factor N is defined as the square root of the area of a sphere divided by the cube root of the volume of a sphere. Convexity Ratio (CR). The CR is a measure of the degree of convexity of an object. CR takes a minimum value of 1.0 for any convex body. The CR is defined as: A CR = (2) Ach with A being the area of the surface, and Ach is the area of its convex hull.
92
A. Serag et al.
1.0
1.32
1.22
1.44
1.51
Fig. 2. Shape Index (SI) as a measure of complexity
Intrinsic Curvature Index (ICI). The ICI is an intrinsic measure of convexity, i.e., how much of the surface is lying on one side of its tangent plane. The ICI is defined as: ICI =
1 K + Af 4π
(3)
f acets
where K + = KV if KV > 0 or else K + = 0; KV is Gauss curvature estimate at a vertex of a triangular facetted, and Af is the area of facet f .
Extrinsic Curvature Index (ECI). The ECI is as a measure of cylindrical parts on the surface. The ECI is defined as: ECI =
1 (H (H 2 − K) + H 2 − K)Af 2π
(4)
f acets
where H being the mean curvature of a surface, K being the Gauss curvature of a surface, and Af is the area of facet f . Mean curvature L2 Norm (MCN). The MCN is a measure of bending of the surface. The MCN is defined as: 1 M CN = Hf2 Af (5) 4π f acets
where Af being the area of facet f , and Hf being the mean curvature of f . Gauss curvature L2 Norm (GCN). The GCN is an intrinsic measure of how much of the surface has constant Gauss curvature. The GCN is defined as 1 A Kf2 Af (6) GCN = 4π f acets
where Af being the area of facet f , Kf being the Gauss curvature of f , and A being total area of the surface.
Unsupervised Learning of Shape Complexity
3
93
Feature Selection
As we have many ROIs, most of the changes related to shape complexity of each developing structure are not completely known. It is well known that the cortex exhibits folds, however most of other structures exhibit unknown complexity changes. In general, we would like to build an unsupervised learning framework that learns the complexity of each structure without any prior knowledge about the type of changes such structure exhibits. Therefore, after applying several measure of complexity to each structure, feature selection is used to select the measures that best describe the changes in complexity per structure. We use Laplacian Score (LS) [15] for feature selection as it is an unsupervised feature selection method. LS does not only prefers to those features with larger variances which have more representative power, but also prefers to selecting features with stronger locality preserving ability. Let each image in the database to be represented by s shapes and every shape i denote the i-th sample xi of the r-th is described by m shape descriptors. Let fkr feature (descriptor) of the k-th shape, where i = 1, ..., n; k = 1, ..., s; r = 1, ..., m. 1 2 n T , fkr , ..., fkr ] . For the r-th feature of the k-th shape, the feature vector fkr = [fkr 1 i Define μkr = n i fkr . All features were normalized to standard scores, with μ = 0 and σ = 1. The LS of the r-th feature of the k-th shape Lkr , which should be minimized, is computed as follows [15]:
j 2 i ij (fkr − fkr ) Sij Lkr = i 2 i (fkr − μkr ) Dii
(7)
where D is a diagonal matrix with Dii = j Sij , and Sij is defined by the neighborhood relationship between samples xi (i = 1, ..., n) as follows: Sij =
e−
||xi −xj ||2 t
0
if xi and xj are neighbors, otherwise
(8)
where t is a constant to be set, and xi and xj are neighbors means that either xi is among K nearest neighbors of xj , or xj is among K nearest neighbors of xi . As mentioned earlier, we are interested in features that best describe the changes in complexity of each shape. Hence, each k shape to be represented by a weighted sum of all the m shape descriptors applied to that shape/ROI: Fki =
r
i fkr ∗ Lkr
(9)
where the Lkr represents the LS to weigh the measure r applied to shape k. Now, the k-th shape is described by the feature vector Fk = [Fk1 , Fk2 , ..., Fkn ]T .
94
3.1
A. Serag et al.
Biomarker Extraction
After having each ROI represented by a scalar feature vector, which is the weighted sum of the different measures applied to such ROI, we apply the LS feature selection algorithm again to compute Lk for each shape/ROI. The assumption is that, the higher the score, the more likely the ROI is a good biomarker.
4
Results
The result of the propagation of the twenty labeled images from term equivalent age to the younger ages is shown in Fig. 3. Unlike direct registration [16], LISA provides an accurate and consistent registration, particularly when the timeinterval between scans increases [12]. It can be seen that direct registration performed poor at strong intensity boundaries, like GM-CSF, comparing to LISA which performed better.
LISA
Direct
LISA
Direct
LISA
28 weeks
32 weeks
36 weeks
40 weeks
Fig. 3. Comparisons of label propagation using direct registration and LISA. LISA provides accurate results, particularly at strong intensity boundaries
Unsupervised Learning of Shape Complexity
95
90
Volume after Affine Normalization (cm3)
80 70 60 50 40 30 20 10 0 1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 Region Index
Fig. 4. Box plot of the volume of ROIs after affine registration to the reference template
To make sure that the shape complexity measures are not affected by volumetric changes, the volumes of each ROI after affine registration to the reference template are shown in Fig. 4. The figure shows that volumes of each brain structure, within the cohort of study, are consistent after the affine registration step. Table 2 shows the ranked ROIs based on computed LS. Figure 5 shows the top four ranked ROIs, based on computed LS. For each ROI, the weighted sum of the six shape descriptors applied to that ROI is plotted against post-menstrual age at scan (in weeks). These ROIs are Caudate Nucleus (CN), Lentiform Nucleus (LN), Thalamus (TH) and Cerebellum (CE). These four regions (CN, LN, TH, CE) have the highest scores (0.90, 0.89, 0.87, 0.77) among the 26 structures. This suggests that these regions are useful biomarkers of brain development in the age range of interest, which is consistent with the findings from clinical studies [17–19]. We noticed that these regions exhibit faster complexity change in the period 28-37 weeks PMA, comparing to the period 37-45 weeks PMA where the rate of change is hardly noticeable. For better understanding of the results, Fig. 6 shows different regions of interest (and corresponding scores) at different stages of development. It can be seen that the top four ranked regions (CN, LN, TH, CE) undergo significant changes over time, and by term equivalent age, they look quite different from their shape at the younger age. Brainstem (BS) lies close to the middle of the
96
A. Serag et al. Table 2. Ranked ROIs based on computed Laplacian score (LS) ROI Caudate Nucleus Lentiform Nucleus Thalamus Cerebellum Medial and inferior temporal gyri posterior part Medial and inferior temporal gyri anterior part Superior temporal gyrus, middle part Frontal lobe Amygdala Superior temporal gyrus, posterior part Occipital lobe Brainstem, spans the midline Gyri parahippocampalis et ambiens anterior part Hippocampus Parietal lobe Cingulate gyrus, anterior part Lateral Ventricle Insula Lateral occipitotemporal gyrus, gyrus fusiformis posterior part Lateral occipitotemporal gyrus, gyrus fusiformis anterior part Cingulate gyrus, posterior part Anterior temporal lobe, medial part Subthalamic nucleus Anterior temporal lobe, lateral part Gyri parahippocampalis et ambiens posterior part Corpus Callosum
2
Z−score (Caudate Nucleus) FCN
Z−score (Lentiform FLN Nucleus)
LS 0.90 0.89 0.87 0.77 0.72 0.71 0.70 0.68 0.68 0.68 0.66 0.63 0.62 0.61 0.59 0.56 0.55 0.48 0.47 0.46 0.46 0.40 0.37 0.35 0.34 0.33
1 0
30
35 40 Age (weeks PMA)
30
35 40 Age (weeks PMA)
30
35 40 Age (weeks PMA)
2
Z−score (Cerebellum) FCE
Z−scoreF(Thalamus) TH
0
−2
2 1 0
1 0
−1
−1
−2
1
−1
−1
−2
2
30
35 40 Age (weeks PMA)
−2
Fig. 5. The weighted sum of the six shape descriptors applied to CN , LN, TH and CE is plotted against age at time of scan. A fitted Gaussian kernel with σ = 2 (black line) and 95% confidence intervals (dashed lines) are also shown.
Unsupervised Learning of Shape Complexity
97
Caudate Nucleus (0.90)
+X
Lentiform Nucleus (0.89)
-Z
Thalamus (0.87)
+Z
Cerebellum (0.77)
+Z
Brainstem (0.63)
-X
Corpus Callosum (0.33)
+X
30w
36w
40w
Age
Fig. 6. Different shapes (and corresponding scores) at different stages of development
table of rankings with a score of 0.63. Having a look at Fig. 6, BS shape does not change much over time comparing to the regions with higher scores. In addition, Corpus Callosum (CC), which was ranked last in the table of rankings, does not undergo complicated changes, away from thinning and elongation.
5
Discussion and Conclusions
This paper presented an approach for unsupervised learning of shape complexity. The framework can be used for automated extraction of biomarkers of early brain development in MRI. It learns the complexity of different brain structures by applying several shape complexity measures to each individual structure, and then using feature selection to select the measures that best describe the changes in complexity of each structure. Then, feature selection is applied again to assign a score to each structure, in order to find which structure can be a good biomarker of brain development. The used feature selection algorithm, Laplacian Score, showed to perform well in our task as it is an unsupervised algorithm which ranks the features according to their variance and locality preserving power. This study was carried out using T2 weighted MR images from 224 premature neonates (the age range at the time of scan was 26.7 to 44.86 weeks post-menstrual age).
98
A. Serag et al.
The advantage of the proposed framework is that one can extract as many ROIs as desired, and the framework automatically finds the ones which can be used as good biomarkers. However, the example application focuses on neonatal brain image data, the proposed framework for combining information from multiple measures may be applied more generally to other populations and other forms of imaging data. The framework can be useful when analyzing data collected from different sources where the target variable (age) is not consistently defined. Also, gestational age estimation is subjected to minor inaccuracy (4-6 days) [20]. Hence, we chose to use an unsupervised approach and not to use age at all in computing the ranking. As the results are very promising and suggest that there are some anatomical structures that can be useful markers of early brain development, the proposed framework undergoes further analysis to understand the complexity changes of brain structures and how this can be related to biological changes. Future work includes exploring more shape descriptors, both local and global, for more precise biomarkers extraction. This might give more precise measurements that capture subtle changes during brain development. Ethical Statement Ethical permission for this study was granted by the Hammersmith and Queen Charlotte’s and Chelsea Research Ethics Committee (07/H0707/101). Written parental consent was obtained prior to scanning. This study is part of a wider study of the effect of preterm birth on brain development. Acknowledgments. The authors are grateful for the support received from the ’Chloe Svider bursary’.
References 1. Zilles, K., Armstrong, E., Schleicher, A., Kretschmann, H.J.: The human pattern of gyrification in the cerebral cortex. Anatomy and Embryology-Berlin 179(2), 173– 179 (1988) 2. Batchelor, P.G., Castellano Smith, A.D., Hill, D.L.G., Hawkes, D.J., Cox, T.C.S., Dean, A.F.: Measures of folding applied to the development of the human fetal brain. IEEE Transactions on Medical Imaging 21(8), 953–965 (2002) 3. Luders, E., Narr, K.L., Thompson, P.M., Rex, D.E., Jancke, L., Steinmetz, H., Toga, A.W.: Gender differences in cortical complexity. Nature Neuroscience 7(8), 799–800 (2004) 4. Awate, S.P., Win, L., Yushkevich, P.A., Schultz, R.T., Gee, J.C.: 3D Cerebral Cortical Morphometry in Autism: Increased Folding in Children and Adolescents in Frontal, Parietal, and Temporal Lobes. In: Metaxas, D., Axel, L., Fichtinger, G., Sz´ekely, G. (eds.) MICCAI 2008, Part I. LNCS, vol. 5241, pp. 559–567. Springer, Heidelberg (2008) 5. Van Essen, D.C., Drury, H.A.: Structural and functional analyses of human cerebral cortex using a surface-based atlas. Journal of Neuroscience 17(18), 7079–7102 (1997)
Unsupervised Learning of Shape Complexity
99
6. Srinivasan, L., Dutta, R., Counsell, S.J., Allsop, J.M., Boardman, J.P., Rutherford, M.A., Edwards, A.D.: Quantification of deep gray matter in preterm infants at term-equivalent age using manual volumetry of 3-tesla magnetic resonance images. Pediatrics 119(4), 759–765 (2007) 7. Ball, G., Boardman, J., Rueckert, D., Aljabar, P., Arichi, T., Merchant, N., Gousias, I., Edwards, A., Counsell, S.: The effect of preterm birth on thalamic and cortical development. Cerebral Cortex (2011) 8. Smith, S.M.: Fast robust automated brain extraction. Human Brain Mapping 17, 143–155 (2002) 9. Tustison, N.J., Avants, B.B., Cook, P.A., Yuanjie, Z., Egan, A., Yushkevich, P.A., Gee, J.C.: N4ITK: Improved N3 Bias Correction. IEEE Transactions on Medical Imaging 29, 1310–1320 (2010) 10. Gousias, I.S., Hammers, A., Counsell, S.J., Rueckert, D., Edwards, A.D.: A new resource for imaging research in neonates: Manually defined brain atlases. Acta Paediatrica 98, 146–147 (2009) 11. Serag, A., Aljabar, P., Ball, G., Counsell, S., Boardman, J., Rutherford, M., Edwards, A., Hajnal, J., Rueckert, D.: Construction of a consistent high-definition spatio-temporal atlas of the developing brain using adaptive kernel regression. NeuroImage 59(3), 2255–2265 (2012) 12. Serag, A., Aljabar, P., Counsell, S.J., Boardman, J.P., Hajnal, J.V., Rueckert, D.: Lisa: Longitudinal image registration via spatio-temporal atlases. In: The 9th IEEE International Symposium on Biomedical Imaging (2012) 13. Heckemann, R.A., Hajnal, J.V., Aljabar, P., Rueckert, D., Hammers, A.: Automatic anatomical brain mri segmentation combining label propagation and decision fusion. Neuroimage 33(1), 115–126 (2006) 14. Lorensen, W.E., Cline, H.E.: Marching cubes: A high resolution 3D surface construction algorithm. Computer Graphics 21(4), 163–169 (1987) 15. He, X., Cai, D., Niyogi, P.: Laplacian score for feature selection. In: NIPS (2005) 16. Rueckert, D., Sonoda, L.I., Hayes, C., Hill, D.L.G., Leach, M.O., Hawkes, D.J.: Nonrigid registration using free-form deformations: application to breast MR images. IEEE Transactions on Medical Imaging 18(8), 712–721 (1999) 17. Rutherford, M.: MRI of the Neonatal Brain. Saunders Ltd. (2001) 18. Limperopoulos, C., Soul, J.S., Gauvreau, K., Huppi, P.S., Warfield, S.K., Bassan, H., Robertson, R.L., Volpe, J.J., du Plessis, A.J.: Late gestation cerebellar growth is rapid and impeded by premature birth. Pediatrics 115(3), 688–695 (2005) 19. Ment, L.R., Hirtz, D., Huppi, P.S.: Imaging biomarkers of outcome in the developing preterm brain. Lancet Neurology 8(11), 1042–1055 (2009) 20. Engle, W.A.: A. A. of Pediatrics Committee on Fetus, and Newborn, Age terminology during the perinatal period. Pediatrics 114(5), 1362–1364 (2004)
