IEEE - 31661
Segmentation of Thyroid gland in Ultrasound image using neural network Hitesh Garg M.E Scholar, Dept. of CSE, PEC University of Technology, Chandigarh, UT, India
[email protected] Abstract— The thyroid gland is highly vascular organ, and lies in the anterior part of the neck just below the thyroid cartilage. Ultrasound imaging is most commonly used to detect and classify abnormalities of the thyroid gland. Other modalities (CT/MRI) are also used. There is a challenge to segment ultrasound medical image which is often blurred and consists of noise as other modalities like CT contains ionizing radiations and expensive. Thus, there is a need to apply a method to automated segment well the objects for future analysis without any assumptions about the object’s topology are made. Various methods or techniques are used for automatic segmention of thyroid gand but the application of neural network in image processing provides a better solution to segmentation problem. In this paper we use Feedforward neural network to classify the region using feature extraction and then segment it. Experiment and results are shown. Index Terms—Feed forward neural network, Feature extraction, Image processing, Thyroid segmentation.
I.
INTRODUCTION
Image processing [8] is a vast area inclusive of many subareas but fundamentally can be viewed as a single block in which input is an image or a video and output is either an image or a video or set of parameters associated with the image. Its use widely in medical imaging processing helps radiologist in diagnosis a problem and its saves time which consumes during manually diagnosis. Since radiologist meet hundreds of cases every day, thus diagnosis manually is timeconsuming and laborious. Now-a-days, computer aided diagnosis is an active research area which assists the radiologist and also gives accurate judgment. Computer aided diagnosis includes the basic principle of image processing i.e. image acquisition, image pre-processing, image segmentation or any other task according to one’s need. In medical image processing, there are various imaging modalities like X-ray, MRI, CT scan, OCT, ultrasound etc. Ultrasound is the foremost tool for the diagnosis of thyroid gland. In this paper we use Feed forward neural network [4] to automatic segment thyroid gland. Before segmentation various feature extractions techniques [16] are used to get features of the thyroid and non-thyroid region. These are further used to input the feed forward neural network and to train the neural network The rest of this paper is ordered as follows. In Section II, image pre-processing i.e. image enhancement [15], feature extraction [18],feed forward neural network[4]. Section III presents the experiment results..Conclusions are given in Section IV. II.
THYROID SEGMENTATION
Different image acquisition techniques have different types of noise. Ultrasound image has speckle distribution in addition to structure and grain noise.
Alka Jindal Dept. of Information Technology, PEC University of Technology, Chandigarh, UT, India
[email protected] om
US Image v Image Enhancement w
1
Manually Segment
Feature extraction
thyroid and non-thyroid
"
region
Training of Feedforward neural
I
Classification
I Segmented result of thyroid region Fig. 1 Steps of thyroid segmentation
After the image has been acquired it is pre-processed to get a noise suppressed and an enhanced image. It needs to be processed for further analysis i.e here we need to segment the thyroid gland. Image segmentation is a vital step in image processing. Various segmentation algorithms available are active contour models (ACMs) [10], Watershed [11], Clustering [9] etc but use of neural network [5] attracted due to its better performance and results. There are four major steps in our proposed method for thyroid segmentation which are as follows: 1) Locating the apparent thyroid region and image enhancement; 2) Feature extraction; 3) Training Feed forward neural networks; 4) Thyroid region classification. These processes are described in detail in the following steps. 1) Locating apparent enhancement
thyroid
Region
and
Image
A low visual quality thyroid US images greatly affects the segmentation results. A image pre-processing [15] step is thus required for noise repression, contrast enrichment, identical intensity, outlier eradication, bias compensation, time/space filtering to make the image noise-suppressed. Noise is the result of errors in image acquisition process that result in pixel values that do not reflect the true intensities of the real scene. The image pre-processing steps are as follows: a) Locating the apparent thyroid region; b) Applying an AWMF [13] to reduce speckle noise; c) Morphological operations [8] to enhance the
4th ICCCNT 2013 July 4 - 6, 2013, Tiruchengode, India
IEEE - 31661
filtering result; These steps are explained in detail in the following section: a) Locating apparent Thyroid Region: The thyroid gland in a thyroid US images always lies in between the bright part and the dark part of the image. Two indication values (X1 and X2) are used to locate the apparent thyroid region. X1 is the row index with the largest average intensity in the horizontal projection of the US image. X2 is the first row index with an average intensity of zero from the top to bottom in the horizontal projection of the US image. The apparent thyroid region is located between the X1 th row and the X2th row of the US thyroid image. b) Adaptive Weighted Median Filter (AWMF):- AWMF [13] is able to remove the outliers without reducing the sharpness of image. Thus AWMF is used to remove inevitable speckle noise in US images. The main function of the median filter is to run through the pixel unit by unit, swaping each unit with the median of neighboring units. AWMF is conducted on a fixed moving mask with the weights adjusted according to the local statistics. For a cover with a size of M × M, the weight coefficient wi,j at position (i,j) is given by v
t.i
u
using reduced feature set instead of full size input. Initially features are extracted from the thyroid region manually selected by the physician and also from non-thyroid region. Extracted features are then used in feed forward neural network [4] to train the network. Eight texture features were extracted from selected thyroid region and non-thyroid region and these were Haar wavelet [20], mean, standard deviation, variance, entropy, skewness, kutosis, energy[18]. In our proposed method, we use two Feature extraction techniques for Radiographic texture analysis, First-order histogram based features and Multi-scale features. First-order histogram based features: First order histogram [18] is a graphical representation of pixels intensity values found in that image. First-order statistics can be calculated from the histogram of pixel intensities of that US images and these depend only on individual pixel values and not on the interaction or co-occurrence of neighboring pixel values Assume that US image is a function f(x,y) of two space variables x and y, x=0,1,…,K-1 and y=0, 1,...,L-1. The function f(x,y) can take discrete values i = 0,1,…,M-1, where M is the total number of number of intensity levels in the image. The number of pixels in the whole image is shown by the intensity-level histogram function which has this intensity:
V-x,y
K-l
where μχ_γ and a^y are the mean and variance of the M x M window centred in the (x,y) pixel respectively. w0 is the central weight, g is a scale factor, [■] is round-to-nearest function, and D is the Euclidean distance from the pixel point to the centre of the mask. If the weights are negative, they are set to zero. c) Morphological Operation: Closing and opening operators [8] were applied to the image to remove the redundancy enhanced by AWMF.
L-
h(i) = )
Htffey),'),
x=0
y=0
where S(j,i) is the Kronecker delta function J=i j Ψί
ίΟ'.Ο
The histogram of intensity levels basically shows the spread of pixels. Calculation of the grey-level histogram involves single pixels. Dividing the values h(i) by the total number of pixels in the image one obtains the approximate probability density of occurrence of the intensity levels p(Q =h(i)/KL, (a)
(d)
(b)
(e)
(c)
(f)
The shape of the histogram gave many clues as to the character of the image. For example, a normal shape of histogram is small on either side with peak in middle which reflects that majority of pixel intensity values exists in middle. The shape of histogram can be positively skewed, negatively skewed or normal depending upon the distribution of pixel values. The mean takes the arithmetic average of a set of pixels intensity values found in that US images. M-l
Fig. 2. (a) – (c) Orignal images (d) – (f) Results after image preprocessing steps
2)
i = 0,1, ...,M- 1
μ = >
φ(ί)
Feature Extraction
Feature Extraction technique [16] is mainly used to extract important features from original image which contains redundant data but not much information. The objective of using feature extraction techniques is to transform the input data into a reduced representation set of features so as to extract relevant data. If features extracted are carefully selected then it is obvious that feature set will extract relevant information from input data in order to perform desired task
Variance describes how far the pixels intensity values lies from expected values i.e. mean. Μ —ί
y [x — fi) p{i) ent to which whic a pixel values of that Skewness measure the extent images lean to one side of the mean. The value of skewness is
4th ICCCNT 2013 July 4 - 6, 2013, Tiruchengode, India
IEEE - 31661
zero if the histogram is symmetrical about the mean, and is otherwise either positive or negative depending whether it has been skewed above or below the mean. µ3 is an indication of symmetry. i«3
σ~3 Υ ( ί - μ ) 3 ρ ( ί )
Kurtosis is a measure of whether pixels of an US images that are extracted from reduced feature set are peaked i.e. positive or flat i.e. negative relative to normal distribution of the histogram.
μ4
One level haar wavelet is applied for haar wavelet transform. After decompose mean and variance of LL-subband is computed as LL-subband contains an approximation of original imge while other contains missing details.
σ"4 Υ ( ί - μ ) 4 ρ ( 0
The quality and accuracy of segmentation ultimately depend on the type of features used. Mainly images that contain number of textured regions are best segmented using frequency-based features, whereas images consist of smoother regions can more easily be segmented using local mean and variance of intensity levels. As US images consist of smoother region thus we use histogram features.
3
3) Energy measures the number of repeated pairs. The value of energy is expected to be high if the occurrence of repeated pixel pairs is high. The value of energy is 1 for a constant image
i,j
where P ij symbolize the number of occurrences of grey levels i and j within the given window. Entropy measures randomness that can be used to characterize the texture of the input image. The value of entropy is expected to be high if gray levels are distributed randomly throughout the image.
Σ„.„„,
SEGMENTATION USING FEED-FORWARD NEURAL NETWORK
The remarkable ability of the neural network [12] to provide meaningful information from complex and imprecise data makes its applicability wide in extracting patterns and objects that are too difficult to notify by humans or by other computer techniques. In our proposed work, we used a feed-forward neural network [4], trained with back propagation [1], for extracting pattern. Three different layers in the network are input, hidden and output layer. The connections in the feed forward neural network are unidirectional, which means signals or information being processed can only pass through the network in a one direction, starting from the input layer(s), passing through the hidden layer(s) to the output layer. Depending upon the number of inputs and extracted features, input and hidden layer contains enough number of neurons. Network accepts the input in vector form: Y i = [x i,1 , xi,2, . . . , xi,m ]
Λ)
Multiscale features: For calculating multiscale features[18], Haar wavelet transforms method is adopted which is timefrequency method and it possesses a capability of space localization of pixel intensity values. Haar Wavelet Features (HWT):- It provides a certain sequence of rescaled "square-shaped" functions which together form a wavelet family[21]. The mean and variance of different frequency subbands are thus calculated. For a function f, the HWT is defined as:
where xi,m is the mth feature of the ith pixel. In our applications a feed-forward network with a single layer of hidden units is used with a sigmoid activation function for the neurons. The architecture of FF neural network is shown in Fig3.
/->0>D|cD) D
b C
= =
(b1b....,bm) (C1,C2,
■■■■>CK/2)
Where D is the decomposition level, b is the approximation subbands and c is the detail subband. J2n +
J2n-1
V2 J2n ~
J2n-
V2~"
forn
= 1,2,..., K/2
, : _ : :
,K/2
Fig3. Architecture of Feedforward network
The function of this network can be alienated into two phases:
4th ICCCNT 2013 July 4 - 6, 2013, Tiruchengode, India
IEEE - 31661
1. The training phase Training phase-During the training phase the weights in the FFNet is modified with Back Propagation Algorithm [2]. As a training algorithm, the purpose of back-propagation is to adjust the network weights so the network produces the desired output in response to every input pattern in a predetermined set of training patterns [3]. LevenbergMarquardt back propagation (LM) [6] is a network training function that updates weight and bias values according to Levenberg-Marquardt optimization [7]. It is often the fastest back propagation algorithm and is highly recommended as a first-choice supervised algorithm even though it requires more memory as compared to other algorithms. The differences between the actual outputs and the idealized outputs are propagated back from the top layer to lower layers to be used at these layers to modify connection weights. In Levenberg-Marquardt algorithm [7] neural network training weights are adjusted with the following rule:
w„
w(n) - (J(n)TJ(n)
AP +AN TP
Sensitivity
A~P TN
Specificity
TP
TPrate
FPrate
A~N
(TP + FN) TN
:
(TN + FP)
Table I shows the performance of the proposed method with/without multiscale feature. It shows that multiscale feature significantly improves the segmentation results. Table II illustrate the quantifiable measurements of the proposed method. All the average values are higher than 89%, thus exhibit the efficiency of the proposed method.
de1
de1
de1
dw1 de2
dw2 de2
dwm de2
dw1
dw2
■
den
den
■
dw1
dw2
dwm den dwm
where μ will ensure that matrix inversion will always produce a result and depends on evaluation of sum of squared errors. If the error is decreased, parameter μ is divided by some scalar Θ or if decreases then multiplied by scalar. We set scalar Θ to be proportionate to the magnitude of the error e'e i.e. θ= 0.01(e* e). Denote the weights as vector w, network output o and desired output as vector od . Error vector is defined as e = o Od·
2. The classification phase In this phase the extracted feature of image is taken as input and is transformed from input layer to output layer .Now classification can occur by selecting the category associated with the output unit that has the largest output value. III.
TP + TN
Accuracy
+ M/)- 1 /(n) T e(n)
Here, J is Jacobi matrix, which is defined as
/
and false positive rate. The five measuring indices are defined as follows:
RESULTS
The proposed algorithm is now applied to the images in the data set and the required region of thyroid gland is extracted. Various experiments were performed to show the ability of the proposed method. An additional 10 US images taken from five patients were used to train the feed forward neural network for segmenting thyroid regions from US images. In the 10 training US images, a total of 45 training patterns, including 20 thyroid tissues and 25 non-thyroid tissues were extracted by an experienced physician, which were used in training the feed forward neural network. There are total of 15 inputs for image vector and thus network contains 15 input neurons and 15 hidden neurons and one output neuron. In order to illustrate the segmentation performance of the proposed method, five standardized measurements were adopted, which are as follows: Accuracy, Sensitivity, Specificity, True positive rate
TABLE I EFFICIENCY OF PROPOSED METHOD WITH AND WITHOUT MULTISCALE FEATURE EXTRACTION With multiscale feature Without multiscale Measuring Indices extraction feature extraction Accuracy
96.51
94.45
Sensitivity
89.06
96.60
Specificity
98.90
93.91
False negative rate
10.93
3.40
False positive rate
1.09
6.09
TABLE II SEGMENTATION EFFICIENCY OF PROPOSED METHOD False US Accuracy Sensitivity Specificity False negative positive image rate rate Case1 97.44 93.11 99.32 6.89 0.68 Case2 Case3 Case4
94.90 96.94
80.63 95.84
98.82 97.43
19.37 4.16
1.18 2.57
97.02
85.98
99.83
14.02
0.17
Case5
96.27
89.75
99.14
10.25
0.86
Average
96.51
89.06
98.90
10.93
1.09
Experiment was performed on 10 US images and result of few images are shown in Fig 4. Fig. 4. (a) – (c) Original US images. (d) – (f) Segmented thyroid region using proposed method. (g) – (i) manually outlined by a physician. Value of various measuring indices is shown. Table I shows the
4th ICCCNT 2013 July 4 - 6, 2013, Tiruchengode, India
IEEE - 31661 performance of the proposed method with/without multiscale feature
[6]
[7]
[8] (a)
(b)
(c)
[9]
[10]
[11] (d)
(e)
(f)
[12]
[13] (g)
(h)
(i)
Fig. 4. (a) – (c) Original US images. (d) – (f) Segmented thyroid region using proposed method. (g) – (i) manually outlined by a physician.
IV.
DISCUSSIONS AND CONCLUSION
[14]
[15]
US images are the most common and mostly the first diagnostic tool for imaging diseases. Then too, much research is done on MRI and CT images. Rather than to manual segment thyroid region by physicians is time consuming, a automated approach for segmentation thus remains convenient. The proposed method takes intensity of pixels and texture as the criteria for segmentation of the image. This is a hybrid approach. It first removes the noise then various pixel based features are extracted to estimate texture and then FF neural network is used for intensity based classification. The experiment results show that learning by examples methodology of neural network provides better result. Further, Future work can be proposed for volume estimation of segmented thyroid region.
[16]
REFERENCES
[21]
[1] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, Learning representations by back-propagating errors, Nature, 323, 533–536, 1986. [2] P. J. Werbos, Back-propagation: Past and future, in Proceedings of International Conference on Neural Networks, vol. 1, pp. 343–354, San Diego, CA, 1988. [3] Robert Hecht Nielsen, Theory of the back propagation neural network in Proceedings 1989 IEEE IJCNN, pp. 1593–1605, IEEE Press, New York, 1989. [4] M. T. Hagan and M. Menhaj, Training feedforward networks with the Marquardt algorithm, IEEE Transactions on Neural Networks, 5(6), 989–993, 1994 [5] B. M. Wilamowski. Y. Chen, and A. Malinowski, Efficient algorithm for training neural networks with one hidden layer, in 1999 International Joint Conference on Neural
[17]
[18]
[19]
[20]
Networks (IJCNN’99), pp. 1725–1728, Washington, DC, July 10–16, 1999. #295 Session: 5.1. B. M. Wilamowski and H. Yu, Improved computation for Levenberg Marquardt training, IEEE Transactions on Neural Networks, 2 1 , 930–937, 2010 Arnold Reynaldi, Samuel Lukas, Helena Margaretha,Backpropagation and Levenberg-Marquardt Algorithm for Training Finite Element Neural Network, 2012 UKSim-AMSS 6th European Modelling Symposium, 978-0-7695-4926-2/2012 Gonzalez, R.C., and Woods R.E, (2003) “Digital Image Processing,” Pearson Education, 2nd Edition D. R. Chen, R. F. Chang, W. J. Wu, W. K. Moon, and W. L. Wu, “3-D breast ultrasound segmentation using active contour model,” Ultrasound Med. Biol., vol. 29, no. 7, pp. 1017–1026, Jul. 2003. M.Kass, A.Witkin, and D.Terzopoulos, “Snakes:Active contour models,” Int. J. Comput. Vision, vol. 1, no. 4, pp. 321–331, 1987. L. Vincent and P. Soille, “Watersheds in digital spaces: An efficient algorithm based on immersion simulation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 13, no. 6, pp. 583– 598, 1991. F.M. Ham and I. Kostanic, Principles of Neurocomputing for Science and Engineering. New York: McGraw-Hill, 2001 T. Loupas, W. N. McDicken, and P. L. Allan, “An adaptive weighted median filter for speckle suppression in medical ultrasonic images,” IEEE Trans. Circuits Syst., vol. 36, pp. 129–135, Jan. 1989. Y. Chen, R.Yin, P. Flynn, and S. Broschat, “Aggressive region growing for speckle reduction in ultrasound images,” Pattern Recognit. Lett., vol. 24, pp. 677–691, Feb. 2003. R. C. Gonzalez and R. E. Woods, Digital Image Processing, 2nd ed. Englewood Cliffs, NJ: Prentice-Hall International Edition, 2002 C. M.Wu, Y. C. Chen, and K. S. Hsieh, “Texture features for classification of ultrasonic liver images,” IEEE Trans. Med. Imag., vol. 11, no. 2, pp. 141–152, Jun. 1992. J. C. R. Seabra, L. M. Pedro, J. F. e Fernandes, and J. M. Sanches, “A 3-D ultrasound-based framework to characterize the echo morphology of carotid plaques,” IEEE Trans. Biomed. Eng., vol. 56, pp. 1442–1453, 2009. A. Materka, M. Strzelecki, Texture Analysis Methods – A Review, Technical University of Lodz, Institute of Electronics, COST B11 report, Brussels 1998 The Handbook of Pattern Recognition and Computer Vision (2nd Edition), by C. H. Chen, L. F. Pau, P. S. P. Wang (eds.), pp. 207-248, World Scientific Publishing Co., 1998 Hiremath, P. S. and Shivashankar, S. “Texture classification using Wavelet Packet Decomposition ”, ICGSTs GVIP Journal, 6(2), pp. 77-80(2006). P.S.Hiremath*, S.Shivashankar, Wavelet based features for texture classification, GVIP Journal, Volume 6, Issue 3, December, 2006.
4th ICCCNT 2013 July 4 - 6, 2013, Tiruchengode, India
