Abstract. This paper introduces an automated, pathological class level annotation system for volumetric brain medical images. Existing work focuses mainly on ...
An automated pathological class level annotation system for volumetric brain images Thien Anh Dinh1, Tomi Silander, PhD1, C. C. Tchoyoson Lim, MD2, Tze-Yun Leong, PhD1 1 National University of Singapore, Singapore; 2 National Neuroscience Institute, Singapore Abstract This paper introduces an automated, pathological class level annotation system for volumetric brain medical images. Existing work focuses mainly on annotating medical images at the region of interest level. Our system does not require annotated region level training data nor assumes perfect segmentation results for the regions of interest; the time and effort needed for acquiring training data are hence significantly reduced. This capability of handling incomplete and uncertain data, however, poses additional technical challenges in dealing with very high dimensional and noisy data that would result in overly fitting classifiers. We propose a framework that combines a regularized logistic regression method, and a kernel-based discriminative method to address these problems. Regularized methods provide a flexible selection mechanism that is well-suited for such datasets. Evaluation of the system has shown promising results in classifying pathological classes for 3-D computer tomography images of traumatic brain injury patients. Introduction Medical imaging plays a significant role in the healthcare process; a huge number of digital images are generated every day. There is an increasing need for efficiently accessing and retrieving relevant images for teaching, research and diagnosis. By indexing the correspondence of keywords and images, traditional text-based image retrieval system (TBIR) can directly and quickly solve this problem. TBIR, however, requires substantial manual work to annotate the images with keywords. Content-based retrieval system (CBIR), on the other hand, allows users to use a query image to search for images with similar low-level visual features. Due to its reliance on the low-level features, however, CBIR is not suitable for answering abstract or high-level queries. Automatic annotation technology combines the advantages of both TBIR and CBIR by annotating images with their semantic content and allowing users to perform text-based search on image databases. The images and their corresponding semantic labels are automatically associated by the machine. There are two types of medical image annotation: anatomical annotation and pathological annotation. Training data for anatomical annotation could be obtained from the DICOM headers of the images. Pathological training data which usually contains semantic labels, however, is harder to obtain. Moreover, to obtain training data for traditional supervised machine learning methods, pathological regions in the image need to be manually annotated. The cost and time needed for manual region labeling is one of the most difficult challenges in automated pathology-level annotation. In this work, we introduce a system that does not require labeling of each abnormal region, but works with training data that are class-labeled at the image set level. This requirement is very helpful in practice. Class-labeled training data could be obtained from medical text report either manually or automatically using natural language processing techniques. In addition, our system does not assume perfect segmentation results as in previous work. This capability of dealing with incomplete and uncertain data poses additional challenges in dealing with very high dimensional and noisy data. Due to the curse of dimensionality, we need a lot more training data to avoid overfitting the classification system. Especially, in medical domain, it is not always possible to obtain more training data. To overcome these challenges, we propose a regularized logistic regression approach to annotate pathology-level information of volumetric brain images medical images. We consider the annotation task as a classification problem for each patient’s image scan. Instead of annotating each abnormal region on the training image, our method works on class-labeled training data that consists of several images from each patient, all of which are assumed to belong to the same pathological class. The main advantages of this approach include the ability to handle non-annotated training datasets, and requiring minimal labeling of the training datasets.
The proposed approach could be applied to annotate pathological classes to any volumetric brain images created by computer tomography (CT), magnetic resonance imaging (MRI), etc... To illustrate this framework, we build a system to automatically annotate CT brain images for traumatic brain injury (TBI). TBI is known to be a major cause of death and disability worldwide. The disease is caused by damage to the brain resulting from external mechanical force such as falls, vehicle accidents and violence. CT has shown to be an effective tool to investigate TBI. There are several types of hemorrhages (or hematoma): extradural hematoma (EDH), subdural hematoma (SDH), intracerebral hemorrhage (ICH), subarachnoid hemorrhage (SAH) and intraventricular hematoma (IVH). Radiologists, trained specialist and senior residents usually categorize TBI. Related work One of the earliest attempts to annotating CT brain images is from Cosic and Longaric [1] who proposed a rulebased approach to labeling intracerebral brain hemorrhage (ICH) on CT brain images. Liao et al. [2] implemented a pathology-level image classifier for CT brain images using a decision tree-based approach. Their training data consisted of 48 images classified into three hematoma types: epidural, subdural and intracerebral; the, authors assumed that all images have hemorrhage regions. Zhang and Wang [3] proposed to classify normal and abnormal CT brain images by using global image features such as intensity, shape, texture and symmetry. They used the C4.5 decision tree algorithm and radial basis function neural network for the classification. Annotated training data requires substantial effort. To address the problem of lacking annotated training data, Gong et. al [4] proposed to use a traditional machine translation paradigm to find the correspondences between pathology labels and the regions of interest. They used an unsupervised approach to label the images with keywords extracted from the associated reports. This work, however, did not address the case in which the extracted regions of interest from segmentation are not real hemorrhage regions. In addition, this work focused on classifying individual region of interests (ROI) instead of annotating the pathological classes of single cases (each containing a series of images). In our previous work [5], we proposed a generative-model based approach to solving this same class of problem. In this approach the relationships among all the features have to be carefully modeled in order to achieve decent performance. If the relationships are not obvious, discriminative approaches such as support vector machines (SVMs) [6] often yields better performance. In contrast, our current work aims to annotate a 3-D CT brain image with a hematoma type that is the most representative pathology for a patient by using regularized logistic regression feature selection technique and kernelbased discriminative methods. The proposed approach works without annotated region-level training data and does not assume perfect segmentation results of the regions. Data 191 CT brain scans of patients with TBI were retrospectively selected from the database of the Neuroradiology Department in a tertiary referral hospital specializing in neurological diseases in Singapore. Institutional Review Board approval was obtained for this anonymized dataset for retrospective data mining. The dataset consists of three main types of TBI: SDH (110 cases), EDH (40 cases) and ICH (41 cases). Each 3-D brain scan is a stack of 18-30 images called slices (Figure 1). Each slice is an image with a resolution of 512 × 512 pixels. Each scan (not each image/slice) was manually assigned a hematoma type, which was extracted from its relevant medical text report. Since medical report is usually subjective, inconsistent and difficult for automated keyword extraction, this class-assignment was done manually. However, this process is much less time-consuming than the process of annotating images. Methods Our proposed image classification system consists of two parts: a feature extraction component and a classification component. The feature extraction component segments out all potential hemorrhage regions and extract their corresponding description vectors. Since each TBI case could contain a different number of hemorrhage regions, each case is represented by a set of features. Because of the variable size of the feature vectors, discriminative
approaches could not be directly apply to learn the model. Hence, we need to transform a set of features into uniform-length vectors. This transformation will create a very high dimensional dataset that could severely impact the performance of the classification system. Therefore, we propose to use regularized logistic regression method to reduce dataset’s dimensionality and apply SVM to learn a classification model. The main considerations of our chosen approach are to select relevant features for each class and at the same time avoid overfitting the classifier due to the limited number of training samples. Unlike other feature selection methods, regularized logistic regression provides a framework to easily adjusting the selection criteria in an interpretable manner.
Figure 1: 3-D CT brain scan with 19 slices In the following, we describe our system in more detail. Feature extraction component The feature extraction component extracts a list of potential regions from a brain scan using an automated segmentation technique. Due to the limitations of the segmentation technique, potential regions can be either artifacts or true hemorrhage regions. In this work, for illustration purpose, we adopt a segmentation method adapted from Gong et al. [7]. This technique is shown to achieve reasonable results for content-based retrieval purposes. Other segmentation techniques could potentially be used as well. The segmentation process could be summarized into four main stages: removing skull area from an image by using a thresholding technique and a simple hole filling algorithm [8], removing artifacts and reducing noise from the image using a wavelet technique, normalizing images (pose correction, intensity and size normalization) and segmenting out potential hemorrhage region by an automated thresholding algorithm. Figure 2 illustrates the segmentation process described above. In traumatic brain injury, location and shape of hemorrhage regions are important to differentiate various types of hemorrhages. For instance, EDH usually has biconvex shape with well-defined margin. Meanwhile, SDH often appears with crescentic shape and less regular inner margin. Hence, for each segmented region, we extract its shape descriptors and location as described in the Table 2 automatically using a regionprop method from MATLAB. Particularly, features S, MajA, MinA, Ecc, E are important indicators for differentiating convex, non-convex and crescent appearance of a region.
However, they do not explicitly provide overview or global information about the patient’s brain. To capture the overview or global descriptions of the extracted, potential hemorrhage regions (above), we define the features as shown in Table 1.
(d)
(a)
(b)
(c)
(d)
Figure 2: Preprocessing and segmentation process. (a) original image, (b) image after skull removal (c) image after normalization process, (d) segmented region Table 1: Description of global features Feature name N WL1, WL2, WZ H 1, H 2, H 3, H 4
Feature description Total number of extracted regions Weighted average centroid of extracted regions (weighted by size of a region) Histogram of number of extracted regions (slice index is split into four ranges: regions with Z
