Available online at www.sciencedirect.com Available online at www.sciencedirect.com
Procedia Engineering
Procedia Engineering 00 (2011) 000–000 Procedia Engineering 15 (2011) 2885 – 2891 www.elsevier.com/locate/procedia
Advanced in Control Engineering and Information Science
Apple Grading Method Based on Features Fusion of Size, Shape and Color Xianfeng Li 1, Weixing Zhu 2 a* 1 2
School of Information Engineering, Yancheng Institute of Technology, Yancheng, China School of Electrical and Information Engineering, Jiangsu University, Zhenjiang, China
Abstract As grading results of apples based on the single feature such as size, shape or color are not accurate, this paper proposes a multi-feature information fusion method based on BP neural network and D-S evidential theory to improve the accuracy of apple grading. Firstly, size, shape and color features are extracted from the processed images of apples. Secondly, apples are classified with each kind of feature by BP network classifier and as independent evidences, the outputs of classifiers are combined to construct the basic probability assignment (BPA). Finally, using D-S fusion rules of evidences to make the decision and achieve the final grading result. The experimental results have shown that the decision information fusion method based on size, shape or color features has good performance on accuracy compared to the single feature-based method in apple grading.
© 2011 Published by Elsevier Ltd. Open access under CC BY-NC-ND license. Selection and/or peer-review under responsibility of [CEIS 2011] Keywords-feature extraction; fourier descriptors; BP neural network; D-S theory; decision fusion; apple grading
1. Introduction Quality grading of fruits is an important item in post-harvest handling and marketing, it is mainly performed according to shape, size, color and any other appearance features of the fruit [1]. Advancements in the areas of computer image processing and machine vision make it possible that fruits can be graded automatically instead of manually [2]. However, most of automatic grading methods have been always performing based only on single feature, which has drawbacks such as uncertainty, inobjectivity and * Weixing Zhu. Tel.: +86-0515-88168193 E-mail address:
[email protected].
1877-7058 © 2011 Published by Elsevier Ltd. Open access under CC BY-NC-ND license. doi:10.1016/j.proeng.2011.08.543
2886 2
XianfengLi, Li Weixing and Weixing / Procedia Engineering 15 (2011) 2885 – 2891 Xianfeng Zhu/Zhu Procedia Engineering 00 (2011) 000–000
inconsistency [3]. D-S evidential theory is a theory of uncertainty that was first developed by Dempster and extended by Shafer [4]. It have secure foundation of mathematics, can provide an optimal results from a set of options by simple reasoning without prior probability, and can more explicitly represent uncertainty, imprecision and ignorance of evidence than traditional probability theory [5]. Feature extraction is a key issue in apple grading. Fourier Descriptors (FD), which can descript the closed curve of object’s contour is most frequently used in classification tasks of two-dimensional objects. The advantage of FD lies in good extraction and easy description of object features that are invariant to such transformations as: shifting, scaling and rotations of classified objects [6]. The goal of this paper is to build a method which can realize multi-feature fusion in apple grading in order to improve the accuracy and stability of correct grading rate. The overall process of the method this paper proposed can be seen in Fig.1. color
apple
size
image
shape
and
sub
BPA1
N-N1
D-S
apple
sub
fusion
grade
N-N2
feature extraction
BPA2 BPA build
Fig. 1. Structure diagram of grading system.
2. Features Extraction of Apples The apple images used in this work, e.g. Fig.2 (a), are taken from a top-view camera. In order to extract features easily and efficiently, first, we get binary image by using Otsu segmentation method, then the binary image is filtered by morphology operation and the final binary image is shown in Fig.2 (b). To make the size and shape analysis of the target image further, the boundary tracking algorithm of 8-neighborhood search is utilized to track the boundary contour of apple image so as to acquire the boundary series in certain orientation. As shown in Fig.2 (c), the object boundary curve obtained is closed and continuous, keeping the geometry information of original contours in a good situation. From Fig.2 (c) we know that the contour of apple image is a closed curve, so we can seek the centroid of apple’s shape by using moments of this shape. The formula is as below: M ( p , q ) = ∑ i p j q f (i , j ) ( i , j )∈S
(1)
Where, M stands for the moment under different values of p and q; p, q=0,1,2,…; (i, j) is the coordinate of pixel point of apple image; f(i, j)represents the quality of the pixel. The calculation formula of shape’s centroid coordinate O(Cx, Cy) can be described by: Cx = M (1,0) / S , C y = M (0,1) / S ,Where, S=M(0,0) , M(1,0) = ∑if (i, j) , M(1,0) = ∑ jf (i, j) . Then, we can calculate θ, the angle of the main axis by the formula as follows: tan 2 θ + {[M (2,0) − M (0,2) / M (1,1)]}tan 2 θ = 1 B
r(θ) O
(a)
(b)
(c)
(d)
θ
A
O
θ P0
(e)
Fig. 2. Image processing (a) Original image; (b) Binary image; (c) Contour image (d) centroid of image; (e) coordinates of FD
2887 3
Xianfeng Li and – 2891 Xianfeng Li,Weixing WeixingZhu Zhu/ /Procedia ProcediaEngineering Engineering1500(2011) (2011)2885 000–000
2.1. Features extraction of size and shape As shown in Fig.2 (d), line AB is the max diameter of the apple image, we define length value of line AB as the parameter Dmax to describe the size of the apple. In Fig.2 (d), select A as the initial discrete point (point P0, shown in Fig.2 (e)) which is the point with relatively fixed position on the apple shape. Take P0 as the initial position and process the curve with discretization. The discrete sequence obtained with initial point P0 is immune to the influences of target translation, rotation, etc. Then, divide the boundary curve into N parts which are same in length, so N equal-diversion points are obtained, denoted by P0, P1, …, PN-1. In such a way the boundary sequence point (xk, yk) (k=0,1,⋯,N-1) arranged in the anticlockwise direction can be obtained. Connect Pk with centroid O and evaluate the radius sequence r(k) anticlockwise. The following is obtained: (2) r ( k ) = OPk = ( xk − C x ) 2 + ( yk − C y ) 2 r(k) is a periodic sequence with a periodic N (N=64 in this study), wherein, r(0) =rmin. In order to avoid the influence of size on the result, the normalization is applied in this study. Let rmin be the minimum of the amplitude sequence, rmax be the maximum of the amplitude sequence. The sequence r(g) obtained after the normalization to r(k) is as follows: (3) r ' (k ) = [r (k ) − rmin ] /[rmax − rmin ] The discrete Fourier transform (DFT) of radius sequence is as follows: F r ( h ) = DFT [ r ( k )] =
63
∑r
'
(k )e
−j
2π kh 64
(4)
k =0
Where, the harmonic order k=0,1,…63, frequency h=0,1,…63, F(h) is called the FD of boundary. In order to reduce the computational cost, fast fourier transform (FFT) [7]was used in this study. The FFT formula can be described as: ⎡ Fr ( 0 ) ⎤ ⎡ w 0 ,0 ⎥= ⎢ M ⎢ M ⎥ ⎢ ⎢ ⎢⎣ F r ( 63 ) ⎥⎦ ⎢⎣ w 63 , 0
Where, wk , h = e
−j
2π kh 64
L O L
w 0 , 63 ⎤ ⎡ r ' ( 0 ) ⎤ ⎥ ⎥⎢ M ⎥⎢ M ⎥ w 63 , 63 ⎥⎦ ⎢⎣ r ' ( 63 ) ⎥⎦
(5)
, h=0,1,…63.
As to the apple in Fig.2, when the FFT is carried out to the radius sequence obtained after the tracking to the boundary, the relationship curve between frequency h and amplitude F(h) is obtained as in Fig.3 which shows each harmonic of FD amplitude distribution with the frequency variation, indirectly reflecting the shape information of primary boundary curve. 3000
am plitude/|Fr(h)|
2500 2000 1500 1000 500 0 1
8
15
22
29
36
43
50
57
64
frequency/h
Fig. 3. F(h) value change
It can conclude from Fig.3 that the energy of FD mainly concentrates near F(0) and F(63), however, the value of most terms in the middle is small. Furthermore, the amplitude decreases fast with the
2888 4
XianfengLi, Li Weixing and Weixing / Procedia Engineering 15 (2011) 2885 – 2891 Xianfeng Zhu/Zhu Procedia Engineering 00 (2011) 000–000
frequency h grows. Most coefficients which have little influence on the basic shape feature are, therefore, discarded so as to reduce the dimension of eigenvector. Tests indicate that the first five components of FD have almost the same shape as the primary image, thus it is considered to be able to represent the apple shape without losing the necessary shape information. We choose F(i) (i=0,1,…,4) and Dmax (the max diameter of the apple image) to compose one sixdimensional vector α=(a1,a2,a3,a4,a5,a6), and we can regard it as parameters of shape and size features described in apple image. 2.2. Feature extraction of color In HIS color space, which is close to the visual perception of color, the image color information was only related to hue (H) component, so we convert RGB image into HIS space. ⎧⎛ 255 o −1 2 R − G − B ⎞ ⎟× ⎪⎜⎜ 270 + tan ⎟ 360 G > B 3 ( G B ) − ⎠ ⎪⎝ ⎪⎛ ⎞ 2 R G B 255 − − ⎪ ⎟× H = ⎨⎜⎜ 90o + tan −1 G t(t=0.6);Bel(A)-m(Θ)>ε(ε=0.5);m(Θ)
