Available online at www.sciencedirect.com
Computerized Medical Imaging and Graphics 32 (2008) 678–684
Automatic bone age assessment based on intelligent algorithms and comparison with TW3 method夽 Jian Liu a,e , Jing Qi b , Zhao Liu c , Qin Ning d , Xiaoping Luo a,∗ a
Department of Pediatrics, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan 430030, China b Department of Neurology, First Affiliated Hospital of Guangxi Medical University, Nanning 530021, China c College of Computer Science and Technology, Wuhan University of Science and Technology, Wuhan 430081, China d Department of Infectious Disease, Tongji Hospital, Huazhong University of Science and Technology, Wuhan 430030, China e Department of Pediatrics, Tongji Medical College Affiliated Xiangfan Hospital, Huazhong University of Science and Technology, Xiangfan 441021, China Received 12 December 2007; received in revised form 3 July 2008; accepted 7 August 2008
Abstract Purpose: New algorithms are proposed to improve the validity, accuracy and practicality of automatic bone age assessment (ABAA). Materials and methods: The concept of object-based region of interest (ROI) was proposed. Thirteen RUS (including radius, ulna and short finger bones) ROIs and seven carpal ROIs were appointed respectively according to Tanner–Whitehouse (TW3) method. Five features including size, morphologic features and fusional/adjacent stage of each ROI were extracted based on particle swarm optimization (PSO) and input into ANN classifiers. ANNs were built upon feed-forward multilayer networks and trained with back-propagation algorithm rules to process RUS and carpal features respectively. About 1046 digital left hand-wrist radiographs were randomly utilized half for training ANNs and the rest for ABAA after manual reading by TW3 method. Results: BA comparison between observers indicated that the S.D. of RUS BA was larger than that of carpal BA (S.D. = 4.40, 2.42 respectively), but interestingly, both CVs were 4.0, and both concordance rates were very high (95.5% and 94.2%), and both differences between observers were not significant (both P > 0.05). We found by comparison between results of ABAA and manual readings that RUS BA had larger S.D.s than carpal BA between two methods, but the CVs were very similar in the case of carpal BA < 9 years and RUS BA ≥ 9 years (CV = 3.0, 3.1 respectively), apart from a comparatively larger CV for RUS BA < 9 years (CV = 3.5). Both parts of ABAA system, RUS and carpal, had very high concordance rates (97%, 93.8% and 96.5%) and no significant difference compared with manual method (all P > 0.05). Conclusions: PSO method made image segmentation and feature extraction more valid and accurate, and the ANN models were sophisticated in processing image information. ABAA system based on intelligent algorithms had been successfully applied to all cases from 0 to 18 years of bone age. © 2008 Elsevier Ltd. All rights reserved. Keywords: Computer-assisted diagnosis; Bone age assessment; Particle swarm optimization; Tanner–Whitehouse (TW3) method; Neural network model
1. Introduction Bone age assessment (BAA), a common radiological examination to determine any discrepancy between skeletal age and chronological age, plays a vitally important role in diagnostic and therapeutic investigations of endocrinological and growth 夽 The study was supported in part by The National Basic Research Program of China (2005CB522507). ∗ Corresponding author at: Department of Pediatrics, Tongji Hospital, 1095 Jiefang Avenue, Wuhan 430030, China. Tel.: +86 27 83662393; fax: +86 27 83662393. E-mail address:
[email protected] (X. Luo).
0895-6111/$ – see front matter © 2008 Elsevier Ltd. All rights reserved. doi:10.1016/j.compmedimag.2008.08.005
disorders in children, and quantitative assessment of skeletal maturity is also useful for predicting adult’s height [1,2]. Two commonly used reference standards for BAA are Greulich–Pyle method [1] and Tanner–Whitehouse (TW3) method [2]. The former is an atlas-driven method and is based on visually comparing a nondominant hand-wrist radiograph with a number of atlas patterns. Bone age (BA) is assessed on the basis of the pattern, which most accurately resembles the clinical image according to the physicians’ perception. The TW3 method uses a detailed shape analysis of several bones of interest, leading to their individual classification into one of several stages. Scores derived from stages of each interesting bone are summed to compute the assessment. Both methods are time-consuming, and experi-
J. Liu et al. / Computerized Medical Imaging and Graphics 32 (2008) 678–684
679
tion of hands and wrists and incomplete images of fingers or wrists. Thus 1046 cases were enrolled. Subjects, aged from 3 months to 25 years of chronological age, mainly came from central China, who took X-ray photographs for growth and development evaluation. Distribution of age and sex of enrolled subjects see Fig. 1. Females accounted for 56.4% and right handedness 86.5%. Enrolled cases were divided randomly into two groups, half for training ANNs, and the rest for ABAA. All digital images were acquired with the Kodak Direct View DR 7100 system. Fig. 1. Age and sex distribution of enrolled subjects.
enced pediatric endocrinologist or radiologist is needed to carry out the work. The subjectivity of Greulich–Pyle method and the considerable complexity of TW3 method make the automatic bone age assessment (ABAA) a highly desirable goal to assist the radiologist or endocrinologist in performing a more objective, fast and accurate analysis without the intrinsic variability of human activities. Several ABAA methods had been developed based on features extracted from region of interest (ROI) of hand-wrist radiographic image [3–5]. ROI segmentation and boundaries identification, however, are extremely challenging tasks, especially to carpal ROI. The appointed ROI is neither accurate nor objective. In addition, relevant knowledge rules used for bone age deduction and pattern recognition are hard to acquire and be expressed accurately. In order to surmount these drawbacks, two intelligent algorithms, particle swarm optimization (PSO) and artificial neural networks (ANN) were used in present study for fully automatic bone image segmentation, feature extraction and relevant knowledge processing. 2. Materials and methods 2.1. Dataset derivation and division A collection of 1131 digital left hand-wrist radiographic images created from January 2006 to June 2007 was exported from the data bank of Department of Radiology, Tongji Hospital (Wuhan, China). Eighty-five (7.5%) photographs were excluded from the dataset for a variety of reasons such as malformed bones, bone age significantly exceeding 18 years, incorrect posi-
2.2. Atlas-driven method by operators All 1046 photographs were read by two experienced pediatric endocrinologists. The appearance of 20 bones of a given radiograph was compared with TW3 atlas and the nearest match was selected. From these bones stages, two overall maturity scores are obtained, by summing up the 13 RUS scores and the 7 carpal scores respectively. Different ossification levels may appear pathologically or physiologically between RUS system and carpal system. As a result, carpal BA less than 9 years and RUS BA of all cases were assessed respectively. 2.3. Procedures for ABAA based on intelligent algorithms The ABAA system was based on intelligent algorithms (Fig. 2). At data entry stage, ROIs were searched by PSO method. Numerical features of RUS and carpal were computed respectively from each image and stored in the database. In the meantime, manually assessed BA of RUS and carpal and other information of the same image were also input to the database. Then two ANNs, RUS ANN and carpal ANN, were trained separately by the referred features from the database and relevant knowledge was represented and obtained in networks. When input RUS or carpal features of query image most accurately matches that of certain BA in database, computer-assisted diagnosis (CAD) system then output the BA. The ABAA system was accomplished under Windows Server 2003 on an x86-based machine (Intel Pentium D 945 CPU/1GB RAM). Moulds of feature extraction, PSO and ANN, were developed under Microsoft Visual Studio 2005 and the programming language is C++. Microsoft SQL databases, managed by Microsoft SQL Server 2005, were used to store images data and features data.
Fig. 2. Procedures of ABAA system based on intelligent algorithms.
680
J. Liu et al. / Computerized Medical Imaging and Graphics 32 (2008) 678–684
Fig. 3. Object-based ROIs appointments. Each ROI is made up of target area and reference frame: epiphysis and carpals were assigned as target areas and adjacent bones as reference frame: (A) 1–13 are RUS ROIs; (B) 14–20 are carpal ROIs.
2.4. ROI appointments
2.6. ROI segmentation and feature extraction based on PSO
Two independent BAA systems had been divided in this study according to TW3 method, RUS system (comprising 13 bones, i.e. radius, ulna and short finger bones), and carpal system (comprising seven carpal bones). It has been asserted that the carpal bones give different information about the maturity process than do the finger bones, and that the difference between RUS BA and carpal BA may be of differential diagnostic significance [2]. Thus 20 ROIs were derived according to these bones, 13 ROIs belonging to RUS system and other 7 ROIs to carpal system. RUS ROIs were labeled from 1 to 13 in order: radius, ulna, metacarpals I, III, V; proximal phalanges I, III, V; middle phalanges III, V; distal phalanges I, III, V (Fig. 3A). Carpal ROIs from 14 to 20 in order: capitate, hamate, triquetral, lunate, scaphoid, trapezium, trapezoid (Fig. 3B). In order to diminish the influence of confounding factors and obtain independent features of skeletal maturity, we proposed each ROI be made up of two parts: the target area and its reference area, and both of which were objects in the same image. Epiphysis and carpals were assigned as target areas for they are more representative of skeletal maturity, while the adjacent bones served as reference frames.
Particle swarm optimization (PSO) is one of the evolutionary computation techniques for parameter optimization primarily introduced by Kennedy and Eberhart in 1995 [6,7]. It represents a population-based adaptive optimization technique that is influenced by several “strategy parameters”. Choosing reasonable parameter values for the PSO is crucial for its convergence behavior, which depends on the optimization task [8]. The potential solutions, called particles, “fly” through the problem space by learning from the current optimal particle and its own memory. It can search the multi-dimensional complex space efficiently through cooperation and competition among the individuals in a population of particles [6]. We think that the ROI segmentation can be transformed into an optimization process in a multi-dimensional information space and thus can be solved by PSO. Different edges will emerge and develop concomitantly with the change of segmentation threshold. In addition, selection of some edges is more reasonable than others when they represent a bone object. The complete process of ABAA includes image preprocessing, edge detecting, template matching, features extracting and decision making. In order to extract closely relevant BA features from images, the relevant image processing, such as edge detecting, template matching and features extracting, should cooperate with each other. The process of edge detection is very important. To match templates with objects and extract features from images, we previously utilized an algorithm for image template matching based on PSO (Fig. 4) [9]. We designed an edge set model to store the information of image edge detection in
2.5. Dataset fields recorded Each record had 25 fields to register demographic data such as gender and chronological age, bitmap of hand-wrist image, features of 20 ROIs, along with RUS BA and carpal BA assessed by pediatric endocrinologists. The images have been transformed into gray level images in the dataset, and some fields were designed to record the value of ROI features, which could be used in training ANN classifiers and bone age assessment.
Fig. 4. Architecture of ROI segmentation and feature extraction.
J. Liu et al. / Computerized Medical Imaging and Graphics 32 (2008) 678–684
681
Fig. 5. ROI matching example: (A) original image; (B) and (C) ROI during matching; (D) ROI matching result; (E) edge set.
the algorithm. Edge detecting will be processed only when and where it is necessary, while detecting the whole image edge is not necessary. The edge set will be revised progressively in the course of matching (Fig. 5). For each object-based ROI, no matter RUS system or carpal system, each target area will present ordinally three key stages in the course of bone maturation, i.e. absent, separated and adjoining/fused. Thus bone templates were designed compatible with the three stages.
2.7. Features selection The size and morphologic feature of ROI target area, together with the fusion stage between epiphyses and metaphyses indicate significant information of skeletal maturity [1]. The absolute size of ROI, however, is invaluable due to variable scale of the original image. Consequently, we utilized relative ratio such as S0 /(S0 + S1 ), a ratio of ROI target area to ROI area to eliminate the influence of scale. On the other hand, the morphologic feature of ROI bears no relation to the image scale. In present study, the morphologic feature of RUS ROI was described by
Table 1 ROI features selection Features
RUS ROI
Carpal ROI
Fea1 Fea2 Fea3 Fea4 Fea5
S0 /(S0 + S1 ) Sg /S1 X0 /Y0 X0 /Y1 X0 /X1
S0 /(S0 + S1 ) Sg /S1 C0 /D0 D0 /Y1 C0 /C1
X0 /Y0 , the ratio of x axis variable to y axis variable of ROI target area. A ratio of circumference to diameter, C0 /D0 , was designed to reflect the morphologic feature of carpal ROI target area. In addition, Sg represented the gap area between target area of ROI and adjacent objects, and Sg /S1 indicated the fusing or adjoining degree between target area and reference frame (Fig. 6). Five relative ratios were selected as the features of each ROI (Table 1). For every ROI-i (1 ≤ i ≤ 20), we suppose its target area is G2i and its neighbor bone area is G2i+1. The graph G2i could be transformed into graph G2i as follows: (1) O2i = (x, y) = 1/N(x, y); //get the centre of graph G2i .
Fig. 6. Examples of variable selection: (A) RUS ROI-5; (B) carpal ROI-16. Symbols: X, width of the epiphysis or metaphysis; Y, thickness of epiphysis or length of the bone; S, area; C, circumference; D, diameter; Sg , gap area between ROI target area and adjacent objects. Subscript: 0 refers to target area of ROI, 1 to reference area of ROI.
682
J. Liu et al. / Computerized Medical Imaging and Graphics 32 (2008) 678–684
Table 2 Dispersion tendency analyses on BA discrepancy between observers
RUS BA Carpal BA RUS–carpal*
BA (year)
Case
MD (m)
S.D. (m)
CV (%)
95% CI (m)
CR (%)
t
P
0–18 0.05). The CVs of discrepancy between manual readings are larger than that between ABAA and mean manual readings (Tables 2 and 3).
J. Liu et al. / Computerized Medical Imaging and Graphics 32 (2008) 678–684
683
Table 3 Dispersion tendency analyses on BA discrepancy between ABAA and manual readings BA (year)
Case
MD (m)
S.D. (m)
CV (%)
95% CI (m)
CR (%)
RUS BA
≥9
