Plant Leaf Recognition Using Zernike Moments and Histogram of ...

Plant Leaf Recognition Using Zernike Moments and Histogram of Oriented Gradients Dimitris G. Tsolakidis, Dimitrios I. Kosmopoulos, and George Papadourakis Department of Informatics Engineering, Technological Educational Institute of Crete, GR-71004, Heraklion, Greece [email protected], [email protected], [email protected]

Abstract. A method using Zernike Moments and Histogram of Oriented Gradients for classification of plant leaf images is proposed in this paper. After preprocessing, we compute the shape features of a leaf using Zernike Moments and texture features using Histogram of Oriented Gradients and then the Support Vector Machine classifier is used for plant leaf image classification and recognition. Experimental results show that using both Zernike Moments and Histogram of Oriented Gradients to classify and recognize plant leaf image yields accuracy that is comparable or better than the state of the art. The method has been validated on the Flavia and the Swedish Leaves datasets as well as on a combined dataset. Keywords: Leaf recognition, Zernike moments, Histogram of oriented gradients, support vector machine.

1

Introduction

Plants play a vital role in the environment. There is huge volume of plant species worldwide and their classification has become an active area of research [1]. A plant database is of obvious importance for archiving, protection and education purposes. Moreover, recognition of plants has also become essential for exploiting their medicinal properties and for using them as sources of alternative energy sources like bio-fuel. The classification of plant leaves is a useful mechanism in botany and agriculture [3]. Additionally, the morphological features of leaves can be employed in the early diagnosis of certain plant diseases [5]. Plant recognition is a challenging task due to the huge variety of plants and due to the many different features that need to be considered. There are various ways to recognize a plant, like flower, root, leaf, fruit etc. Recently, computer vision and pattern recognition techniques have been applied towards automated process of plant recognition [2]. Leaf recognition plays an important role in plant classification and its key issue lies in whether the chosen features have good capability to discriminate various kinds of leaves. Computer aided plant recognition is still challenging due to improper models and inefficient representation approaches. A. Likas, K. Blekas, and D. Kalles (Eds.): SETN 2014, LNCS 8445, pp. 406–417, 2014. © Springer International Publishing Switzerland 2014

Plant Leaf Recognition Using Zernike Moments and Histogram of Oriented Gradients

407

A lot of work has been focused on the shape description of the leaves. In the past decade, research on contour-based shape recognition [18-19] was more active than that on region-based [17]. In [15] they introduced a multiscale shape-based approach for leaf image retrieval. The leaf represented by local descriptors associated with margin sample points. Within this local description, they studied four multiscale triangle representations: the well-known triangle area representation (TAR), the triangle side lengths representation (TSL) and two other representations, triangle oriented angles (TOA) and triangle side lengths and angle representation (TSLA). In this research they used 1-NN as classifier. In [16] they proposed a contour-based shape descriptor, named Multiscale Distance Matrix (MDM), to capture shape geometry while being invariant to translation, rotation, scaling, and bilateral symmetry and to classify the plants they used 1-NN as classifier. The color information was incorporated in the plant identification in [11] and [12] and RBPNN was used as classifier. However most researchers avoid using color, mainly due to its dependency on the illumination. Some other researchers focused on green leaves and ignored color information on the leaf. In [10] they used PNN to classify 32 species of plants. All the plants they used in their research had green leaves. Also in [24] Zulkifli used General Regression Neural Networks (GRNN) and invariant moment to classify 10 kinds of plants. They did not include color features to the classifier. Furthermore, in [14] they used K-SVM to classify 32 species of plants and they also did not use any color features. This paper differs from the previous approaches due to the fact that we propose a method for recognizing leafs using as shape descriptor the Zernike Moments (ZM) and as a descriptor for the interior of the leaf the Histogram of Oriented Gradients (HOG). Support Vector Machine (SVM) has been used as a classifier, which is among the best methods for discriminative models. Experimental results on Flavia dataset [21] indicates that the proposed method yields an accuracy rate of 97.18%, on Swedish Leaves dataset [22] 98.13%. When we combine both Flavia and Swedish Leaves the obtained accuracy is 97.65%. To our knowledge these results are similar or better than the state of the art, and it is the first time someone combines these two popular databases. An overview of the method is given in Fig. 1. More specifically we perform a preprocessing step, then we extract a feature vector per image and finally we do the classification of the image. The rest of the paper is organized as follows. In the next section we describe the preprocessing steps. In section 3 we outline the feature extraction method and in section 4 we outline the classification method. In section 5 we give the experimental results of our method and section 6 concludes this paper.

408

D.G. Tsolakidis, D.I. Kosmopoulos, and G. Papadourakis

Fig. 1. Proposed method for leaf classification

2

Image Pre-processing

2.1

Convert RGB Image to Binary Image

Each RGB image is firstly converted into grayscale. We do not employ color due to its dependency on the illumination, so after that we calculate a threshold using Otsu’s method and using this threshold level we convert the grayscale image to binary, so that we can have the leaf image in white and background in black. All images are scaled in 512x512 resolution. 2.2

Eliminating the Petiole

Some leaves have petioles so we have to eliminate them, because they can distort the overall shape. For that we use the Distance Transform operator which applies only to binary images. It computes the Euclidean distance transform of the binary image. For each pixel in binary image, the distance transform assigns a number that is the distance between that pixel and the nearest nonzero pixel of binary image. 2.3

Center the Image

After converting the image to binary we find the connected components of image and use the centroid property to find the center of mass of the region, so we can move the image to the center. This is necessary for the Zernike Moments calculation.


409

Fig. 2. a) RGB image, b) Grayscale image, c) Binary image and petiole elimination, d) Centered binary image, e) Cropped Grayscale image

3

Feature Extraction

In this paper we use Zernike Moments (ZM) on centered binary images and Histogram of Oriented Gradients (HOG) on cropped grayscale images to extract and calculate features of leaf image. 3.1

Zernike Moments (ZM)

We use Zernike Moments (ZM) to extract features using the shape of leaf. The computation of Zernike moments from an input image consists of three steps: computation of radial polynomials, computation of Zernike basis functions, and computation of Zernike moments by projecting the image on to the basis functions. The procedure for obtaining Zernike moments from an input image begins with the computation of Zernike radial polynomials. The real-valued 1-D radial polynomial Rnm is defined as | | /

,

,

,

(1)

where, ,

,

!

1 !

| | 2

!

| | 2

!

(2)

410


In (1), n and m are generally called order and repetition, respectively. The order n is a non-negative integer, and the repetition m is an integer satisfying n − |m| = (even) and |m|≤n. The radial polynomials satisfy the orthogonal properties for the same repetition, ,

,

0 , otherwise

,

Zernike polynomials V(ρ,θ) in polar coordinates are formed by ,| |

e

1

(3)

where (ρ,θ) are defined over the unit disk, j = √ 1 and is the orthogonal radial polynomial defined in equation (1). Finally, the two dimensional Complex Zernike Moments for a NxN image are defined as, 1

π

,

,

(4)

where f(x,y) is the image function being described and * denotes the complex conjugate [6]. 3.2

Image Reconstruction Using Zernike Moments (ZM)

Let f(x,y) be an image function with dimension NxN, their moments of order n with repetition m are given by, N

N

,

(5)

,

Expanding this using real-values functions, produces: N

,

C

cos

S

sin

C 2

(6)


411

composed of their real (Re) and imaginary (Im) parts: C

2Im

S bounded by

∑ ∑

2Re

y

∑ ∑

,

cos ,

sin

(7) (8)

1.

a)

b)

Fig. 3. a) Original image, b) Reconstructed image from Zernike Moments

Based on [23] the Zernike Moments have the following advantages: • Rotation Invariance: The magnitudes of Zernike Moments are invariant to rotation. • Robustness: They are robust to noise and minor variations in shape. • Expressiveness: Since the basis is orthogonal, they have minimum information redundancy. 3.3

Histogram of Oriented Gradients (HOG)

The Histograms of Oriented Gradients (HOGs) are feature descriptors used in computer vision and image processing for the purpose of object detection (e.g., [20]). The technique counts occurrences of gradient orientation in localized portions of an image. This method is similar to that of edge orientation histograms, scaleinvariant feature transform descriptors, and shape contexts, but differs in that it is computed on a dense grid of uniformly spaced cells and uses overlapping local contrast normalization for improved accuracy. The essential concept behind the Histogram of Oriented Gradient descriptors is that local object appearance and shape within an image can be described by the distribution of intensity gradients or edge directions. The implementation of these descriptors can be achieved by dividing the image into small connected regions, called cells, and for each cell compiling a histogram of gradient directions or edge orientations for the pixels within the cell. The combination of these histograms then represents the descriptor. For improved accuracy, the local histograms can be contrast-normalized by calculating a measure of the intensity across a larger region of the image, called a block, and then using this value to normalize all cells within the block. This normalization results in

412


better invariance to illumination changes or shadowing [8]. Since the HOG descriptor operates on localized cells, the method upholds invariance to geometric and photometric transformations, except for object orientation. The concept of HOG is particularly appealing for the classification of leaves due to the nature of their region, which contains many visible veins (see Fig. 4). Those veins are very informative about the specific class the leaf belongs to. Furthermore, the gradients that HOG capitalizes upon, are unlike the color features that some researchers use (e.g., [12]), generally robust to illumination changes and color differences due to the leaf maturity.

Fig. 4. Veins of leaf

4

Classification

For the classification we have employed the Support Vector Machine (SVM), which is appropriate for the task of discriminative classification between different classes [9]. SVM can simultaneously minimize the empirical classification error and maximize the geometric margin. SVM can map the input vectors to a higher dimensional space where a maximal separating hyperplane is constructed. Two parallel hyperplanes are constructed on each side of the hyperplane that separate the data. An assumption is made that the larger the margin or distance between these parallel hyperplanes the better the generalization of the classifier will be. We consider a set of n data points of the form , , where yn is , , ,………… , , , either 1 or -1 indicating the class to which xn the point belongs. Each xn is a p-dimensional real vector. We want to find the maximum-margin hyperplane that divides the points having yn = 1 from those having yn = -1. Any hyperplane can be written as the set of points x satisfying w.x – b = 0,

(9)

where “.” denotes the inner product and w the normal vector to the hyperplane. The parameter determines the offset of the hyperplane from the origin along the normal vector w. If the training data are linearly separable, we can select two hyperplanes in a way that they separate the data and there are no points between them, and then try to


413

maximize their distance. The region bounded by them is called "the margin". Slack variables allow a tradeoff between misclassification of samples and overall performance. More details can be found in [9].

5

Experimental Results

To test the proposed method we first used Flavia dataset, then we used the Swedish Leaves dataset and at last we combined both the Flavia and Swedish Leaves dataset. We have 47 species from both datasets, 32 species from Flavia dataset and 15 species from Swedish Leaves dataset. In Flavia dataset we used 50 samples per species. For each species we used 40 samples for training and 10 for testing. In Swedish Leaves dataset first we used 50 samples per species for training and 25 samples for testing and second we used 25 samples for training and 50 for testing to compare results with other methods. After the calculation of Zernike Moments and Histogram of Oriented Gradients on both datasets, we concatenated the features from both datasets, but this time we used 50 samples per species from Swedish dataset. After that, we used 40 samples per species for training and 10 samples for testing. The feature extraction from both datasets is the same. To compare and see if the difference lies in features or in classifier, we also computed chebyshev moments and concatenated with Histogram of Oriented Gradients features. After the training and testing results, we saw that the accuracy was lower than the other method. Firstly, the gradient was simply obtained by filtering it with two one-dimensional filters: horizontal: (-1 0 1) and vertical: 1 0 1 . The second step was to split the image into 4x4 cells and for each cell one histogram is computed according to the number of bins orientation binning. The numbers of bins we used is 9. The result of the feature vector of HOG is a 144 dimensional vector for every image leaf. This vector is a computation of 4x4x9 and that’s how we got the 144 dimensional vector. Different normalization schemes are possible for a vector V, containing all histograms of a given block. The normalization factor Nf that we used is: L2-norm: Nf = / || ||

(10)

All images were normalized with respect to orientation, so that the major axis appeared vertically. This establishes a common reference for the HOG descriptors. We computed Zernike Moments up to order n=20 and the result was a 121 dimensional vector for every image leaf, which was normalized by computing the complex modulus. In this paper we used linear SVM as classifier. Before we started the classification we concatenated Zernike Moments (ZM) and Histogram of Oriented Gradients (HOG) arrays and then we normalized our data into range [0, 1]. After obtaining the best C parameter for our training model we calculated the overall accuracy of our system. Using Zernike Moments (ZM) up to order 20 and Histogram of Oriented Gradients (HOG) as features, resulted in a highest accuracy of 97.18% for using only Flavia dataset, 98.13% for using only Swedish Leaves dataset and 97.65% for using both datasets, which are listed in Table 1. The test and training samples were randomly selected as indicated in Tables 2, 3.

414


Fig. 5. Overview of the Swedish Leaf Dataset

Fig. 6. Overview of the Flavia Leaf Dataset Table 1.

Dataset

Features

Classifier

Flavia

ZM + HOG

Linear SVM

Best Accuracy 97.18%

Flavia + Swedish Leaves

ZM + HOG

Linear SVM

97.65%

Swedish Leaves

ZM + HOG

Linear SVM

98.13%

Table 2. Comparison based on the Swedish Leaves database

Method [16] [16] [16] [16] [15] [15] [15] [15] Our Our

Features MDM-CDC MDM-CDA MDM-CDM MDM-RA TSLA TSL SPTC + DP TAR ZM + HOG ZM + HOG

Classifier

Training Data

Testing Data

Best Accuracy

1-NN

25 samples

50 samples

91.07%

1-NN

25 samples

50 samples

91.33%

1-NN

25 samples

50 samples

91.20%

1-NN 1-NN 1-NN 1-NN 1-NN SVM SVM

25 samples 25 samples 25 samples 25 samples 25 samples 25 samples 50 samples

50 samples 50 samples 50 samples 50 samples 50 samples 50 samples 25 samples

91.60% 96.53% 95.73% 95.33% 90.40% 95.86% 98.13%


415

Table 3. Comparison based on the Flavia database

Method

[11]

[12]

[10]

[14]

[14]

Our

Features Shape Features Vein Features Color Features Texture Features Pseudo-Zernike Moments Shape Features Vein Features Color Features Texture Features Zernike Moments Geometrical Features Morphological Features Geometrical Features Morphological Features Geometrical Features Morphological Features ZM + HOG

Classifier

Training Data

Testing Data

Best Accuracy

RBPNN

40 samples

10 samples

95.12%

RBPNN

40 samples

10 samples

93.82%

PNN

40 samples

10 samples

90.30%

K-SVM

40 samples

10 samples

96.20%

SVM

40 samples

10 samples

94.50%

SVM

40 samples

10 samples

97.18%

Our method seems to outperform other methods on the Flavia dataset, which is currently the most popular benchmark and is very close to the best reported in the Swedish Leaves. That could be attributed to the fact that we use some of the best descriptors for the shape (contour) and for the internal structure of the leaf (veins). These are actually the essential visual properties that need to be captured. It is expected that the better the state of the art in this representation becomes, the higher the recognition accuracy will be.

6

Conclusion

A new approach of plant classification based on leaves recognition is proposed in this paper. The approach consisted of three phases namely the preprocessing phase, feature extraction phase and the classification phase. We could classify plants via the leaf images loaded from digital cameras or scanners.

416


Zernike Moments (ZM) and Histogram of Oriented Gradient have been used for feature extraction and Linear SVM has been adopted for classification. Compared to other methods, this approach is of comparable or better accuracy on the Flavia and the Swedish Leaves databases. To our knowledge it is also the first time that these two databases are combined.

References 1. Chaki, J., Parekh, R.: Plant Leaf Recognition using Shape based Features and Neural Network classifiers. International Journal of Advanced Computer Science and Applications (IJACSA) 2(10) (2011) 2. Pan, J., He, Y.: Recognition of plants by leaves digital image and neural network. In: International Conference on Computer Science and Software Engineering, vol. 4, pp. 906–910 (2008) 3. Cotton Incorporated USA. The classification of Cotton (2005), http://www.cottoninc.com/ClassificationofCotton 4. Singh, K., Gupta, I., Gupta, S.: SVM-BDT PNN and Fourier Moment Technique for Classification of Leaf Shape. International Journal of Signal Processing, Image Processing and Pattern Recognition 3(4), 67–78 (2010) 5. Kumar, N., Pandey, S., Bhattacharya, A., Ahuja, P.S.: Do leaf surface characteristics affect agro bacterium infection in tea. J. Biosci. 29(3), 309–317 (2004) 6. Suard, F., Rakotommamonjy, A., Bensrhair, A., Broggi, A.: Pedestrian Detection using Infrared images and Histograms of Oriented Gradients. In: Intelligent Vehicles Symposium 2006, Tokyo, Japan, June 13-15 (2006) 7. Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines. Cambridge University Press (2000) 8. Wu, G., Bao, F.S., Xu, E.Y., Wang, Y.-X., Chang, Y.-F.: A Leaf Recognition Algorithm for Plant Classification Using Probabilistic Neural Network. In: IEEE 7th International Symposium on Signal Processing and Information Technology (2007) 9. Kulkarni, A.H., Rai, H.M., Jahagirdar, K.A., Kadkol, R.J.: A Leaf Recognition System for Classifying Plants Using RBPNN and pseudo Zernike Moments. International Journal of Latest Trends in Engineering and Technology 2, 1–11 (2013) 10. Kulkarni, A.H., Rai, H.M., Jahagirdar, K.A., Upparamani, P.S.: A Leaf Recognition Technique for Plant Classification Using RBPNN and Zernike Moments. International Journal of Advanced Research in Computer and Communication Engineering 2(1), 1–5 (2013) 11. Du, J.-X., Wang, X.-F., Zhang, G.-J.: Leaf shape based plant species recognition. Applied Mathematics and Computation 185(2), 883–893 (2007) 12. Arunpriya, C., Selvadoss Thanamani, A.: An Efficient Leaf Recognition Algorithm for Plant Classification using Kernelized Support Vector Machine. International Journal of Computer Science and Management Research 2(2) (February 2013) ISSN 2278-733X 13. Mouine, S., Yahiaoui, I., Verroust-Blondet, A.: A Shape-based Approach for Leaf Classification using Multiscale Triangular Representation, Thesis. Le Chesnay, France CReSTIC Université de Reims, FRANCE (2013) 14. Hu, R.-X., Jia, W., Ling, H., Huang, D.: Multiscale Distance Matrix for Fast Plant Leaf Recognition. IEEE Transactions on Image Processing 21(11), 4667–4672 (2012) 15. Zhang, D., Lu, G.: Review of shape representation and description techniques. Pattern Recognition 37(1), 1–19 (2004)


417

16. Belongie, S., Malik, J., Puzicha, J.: Shape Matching and Object Recognition Using Shape Context. IEEE Trans. Pattern Analysis and Machine Intelligence 24(4), 509–522 (2002) 17. Backes, A.R., Bruno, O.M.: Shape classification using complex network and Multi-scale Fractal Dimension. Pattern Recognition Letters 31(1), 44–51 (2010) 18. Dalal, N., Triggs, B.: Histograms of Oriented Gradients for Human Detection. In: CVPR, pp. 886–893 (2005) 19. Mingqiang, Y., Kidiyo, K., Joseph, R.: A Survey of Shape Feature Extraction Techniques. In: Yin, P.-Y. (ed.) Pattern Recognition, p. 25 (2008) 20. Zulkifli, Z.: Plant Leaf Identification Using Moment Invariants & General Regression Neural Network., Master Thesis, University Technology Malaysia (2009) 21. Mukundan, A.: A Comparative Analysis of Radial-Tchebichef Moments and Zernike Moments. In: BMVC 2009, pp. 1–7 (2009) 22. Söderkvist, O.J.O.: “Computer Vision Classification of Leaves from Swedish Trees”, Master Thesis, Computer Vision Laboratory Linköping University (2001)