Int. J. Engg. Res. & Sci. & Tech. 2013
J Satheesh Kumar and D Surya Prabha, 2013 ISSN 2319-5991 www.ijerst.com Vol. 2, No. 2, May 2013 © 2013 IJERST. All Rights Reserved
Research Paper
THREE DIMENSIONAL OBJECT DETECTION AND CLASSIFICATION METHODS: A STUDY D Surya Prabha1 and J Satheesh Kumar1*
*Corresponding Author: J Satheesh Kumar,
[email protected]
Computer vision is an interesting research area where three dimensional object detection and classification is a significant research problem. Better object detection methods have higher demand on agricultural based applications. Fruit detection and classification is a challenging research application under 3D object analysis method. This paper primarily focusing on detailed study about various methods and algorithms used for 3D object analysis. This paper also discusses three levels of three dimensional fruit recognition and classification such as, image acquisition of low level processing, image segmentation of middle level processing and object recognition of high level processing which are highly suggestible for real time environment. Keywords: Recognition, 3D object, Classification
INTRODUCTION
preprocessing operations on the images. This stage can be understood and done without any
Object classification and recognition is a challenging task for computer vision in real time environment. It is a complicated job as it requires understanding of visual depiction in an image. It also needs to understand the relationship between objects seen in an image and also to identify or classify objects in an image. Computer vision involves three level of analysis such as, low level, mid level and high level processing for image analysis. Low-level processing is used for minimal
prior knowledge about objects in an image. Midlevel processing operates directly on image for extraction of attributes as outputs from an image. These attributes may be either partitioned image, labeled regions in an image or classification of the objects in an image. High-level processing performs maximum abstraction of information from an image. It performs image analysis by operating on the texture, region and objects of an image.
abstraction of information by performing 1
Department of Computer Applications, School of Computer Science and Engineering, Bharathiar University, Coimbatore.
This article can be downloaded from http://www.ijerst.com/currentissue.php 33
Int. J. Engg. Res. & Sci. & Tech. 2013
J Satheesh Kumar and D Surya Prabha, 2013
IMAGE ACQUISITION METHODS
Object classification can be attained efficiently with a prior knowledge about the objects in an image. Though these three level concepts of computer vision are required for object analysis in an image, high level concept plays a vital role in object classification and object recognition. Image can be represented either in two dimensional or in three dimensional plane. In real time environment, objects will be mostly represented in 3D layout with plane background. The food industry has high demand for computer vision in automating quality inspection, fruit sorting, classification and recognition. Automation of food industry helps to increase the accuracy and reduce human error, cost and time. Significant task of computer vision is to recognize optimally matured fruits from a tree and grade it for further process. Fruit classification is a primary and challenging phase where fruits have to be separated from the background environment and further these fruits have to be sorted based on its maturity.. In earlier days, most of the research focuses on fruit quality analysis by using 2D representation of binary, gray scale or color images. The main drawback of 2-D analysis is lack of accuracy due to non-convergence of entire scene or object. Three dimensional digitizing is an emerging tool in the agro-industries for better analysis of fruits. Fruit categorization based on 3D approach will be the great support for farmers and exporters in terms of quality.
Real time image acquisition is one of the important aspects of computer vision and many novel techniques have been updated periodically. It involves the process of acquiring images through a source that automatically capture images and store as a stream of files. These files can be accessed immediately or later. Various advanced image acquisition methods like infrared imaging method, thermal imaging method, collecting images from videos, multiple consecutive shots from different angles in an image and 3D imaging method are widely in use. Among these methods, 3D imaging method has an advantage of being invariant to illumination conditions and has the capability to revolve 3D object to 2D data. 3D images can be acquired through two or more cameras placed at different prescribed points around an object to gather and align a sequence of consecutive images. In another 3D image acquisition technique, camera is fixed at a single position and object is rotated around, to capture images at different angles. 3D imaging sensors can be broadly classified into active and passive sensors. The active 3D sensors use a structured light pattern or laser light and a camera for acquiring 3D object (Kapach et al., 2012). 3D scanners are one of the commonly used active sensors with a projector for light emission that emits multiple light strips (laser or infra-red light) over a 3D object and the camera captures light patterns that are distorted from different angles. The passive 3D sensors use 2D images or video to reconstruct 3D images indirectly from them (Sinoquet et al., 1998).
Organization of this paper is as follows. Section 2 deals detailed discussion on image acquisition methods. Image segmentation method and morphological operations are explained in section 3 and section 4. Section 5 discusses image representation and description methods. Object recognition procedures are focused in section 6 and section 7 concludes the paper.
Passive sensors suffer from the drawback of acquiring non-textured and visually impaired
This article can be downloaded from http://www.ijerst.com/currentissue.php 34
Int. J. Engg. Res. & Sci. & Tech. 2013
J Satheesh Kumar and D Surya Prabha, 2013
plane, satisfying condition 0 . These stored images in the system undergo image preprocessing to enhance image quality. Image enhancement is the process of improving quality of an image by reducing noise and distortion in an image. Image is smoothed for the better performance of pixel labeling in an image. Median filtering is a standard method used for filtering an image. It refers to the most frequent occurrence of pixels in an image. This filtering is reliable and has a better smoothing effect in an image that assists in image analysis as in Figure 1.
regions from a scene. Though active sensors overcome these drawbacks, it suffers from noise problem. In recent years, there are more research papers related to combining implementation of both the active and passive sensors to acquire images efficiently in real time environment. Combined sensors would be more suitable for capturing 3D view of fruit for analyzing the defects and to analyze the quality. 3D image of fruit obtained from the sensor in spherical coordinate system is transformed into Cartesian coordinate system for digitization. Formula used for converting spherical coordinate system into Cartesian coordinate system is defined by the formula
IMAGE SEGMENTATION METHODS Image segmentation is an important module of middle level processing used to segment fruit from the background. It is an essential step in fruit sorting which helps to detect fruit and also assists to identif y the f ruit. Numerous segmentation methods are involved in detecting fruit in real time environment. Methods such as, edge detection, thresholding, histogram, level set, Markov random fields and clustering methods are used for fruit sorting and identification. Edge detection is commonly used method for detecting
a r sin cos b r sin sin c r cos
...(1)
where, r is a radial distance from origin (x) to a prescribed point (y), ‘’ is a zenith angle between the zenith reference direction and the ‘xy’ line, satisfying condition 0 and ‘ ’ is a azimuth angle between the azimuth reference direction and orthogonal projection of line ‘xy’ for reference
Figure 1: (a) Original Image (b) Enhanced Image Using Median Filtering (Figure1(a) Image Courtesy: Russell-Farm)
This article can be downloaded from http://www.ijerst.com/currentissue.php 35
Int. J. Engg. Res. & Sci. & Tech. 2013
J Satheesh Kumar and D Surya Prabha, 2013
methods for pixel distribution measurements. Adaptive histogram equalization is a common technique used to enhance the contrast in an image. This enriches the quality of image and eases image analysis as in Figure 2b.
edges of an object. Edges are identified based on the abrupt change of the pixel value in an image. This basic concept of segmentation is calculated based on detectors developed using gradient values. Detectors such as, Robert, sobel, prewitt, LoG, zero crossing, gabor, and canny are frequently used edge detectors. LoG operator derived from second order of derivatives make an edge detection better than other existing methods as in Figure 2a. Histogram is used to measure the pixel distribution in an image. It is used to identify the constant region of distribution that belongs to fruit. Global histogram, Local histogram, adaptive histogram, histogram equalization are some of the commonly used
Based on histogram distribution in an image, threshold is set to identify fruit region from the background. Pixel values are similar for constant color region in an image. This helps to differentiate fruits having constant pixel value from the background. Global thresholding, local thresholding and adaptive thresholding are some of the common methods used in fruit object selection. Variable thresholding method was applied to partition fruit region as in Figure 2c.
Figure 2: (a) Log Edge Detection Method Applied Output Image, (b) Histogram Equalisation Applied Output Image, (c) Variable Thresholding Applied Output Image, (d) Clustering method applied output image
This article can be downloaded from http://www.ijerst.com/currentissue.php 36
Int. J. Engg. Res. & Sci. & Tech. 2013
J Satheesh Kumar and D Surya Prabha, 2013
with the help of structuring element. Contours with narrow, thin and weak edge strength are combined together in closing operation with the help of structuring element. Opening operation uses erosion followed by dilation. Closing operation uses dilation followed by erosion. Erosion and dilation are the two essential concepts of morphological methods. Structuring element is a set of elements used for the manipulation of erosion and manipulation operation. Erosion (Equation 2) and dilation (Equation 3) are defined mathematically as
Supervised and Unsupervised method of clustering based machine learning techniques can be applied to differentiate fruit from the background (Mondal et al., 2012). Most commonly applied clustering method to detect fruit is the conventional unsupervised k-means clustering method as in Figure 2d. Number of cluster selection is an important feature in this algorithm. This method is applicable to fruit images of different color space model. Markov random field is a Bayesian approach used for labeling the fruit region in an image. A predefined set is fixed to label similar pixels in an image. Markov chain is a chain of random variables that depends on the preceding variable. Variable is the pixel value in an image which depends on the neighbor pixels. Same labels are marked for similar pixels on the basis of pixel appearance with nearest neighbor relationship.
X Ө Y = {z|(Y)z X } ^ X Y = {z|(Y )z X f }
...(2) ...(3)
where f is empty set and Y is the structuring element applied over the original image X. Morphological filter masks are used in performing the dilation and erosion operations. Kernel with 3 X 3 window is used as filter mask to perform morphological operation. Three properties can be derived from the eroded and dilated image by performing image subtraction. Third property output had a better object identification when morphological operation was applied to an input image. Output of morphological operation and properties vary based on the filter mask selection as shown in Figure 3(c) and 3(d). Boundary extraction, hole filling, extraction of connected components, Convex Hull, Thinning, Thickening, Skeletons and Pruning are the widely used morphological algorithms for object identification and description.
MORPHOLOGICAL METHODS Morphological operator is used for extracting objects in an image from which shape and size can be analyzed. The term morphology defines the physical structure of an object. Identifying physical structures helps to mark the physical shape of objects present in an image. Identif ication of object is only a part of morphological operation. The basic operation of morphological operation is to represent the objects in an image and to describe them. Morphological opening and closing are used for smoothing and cleaning the boundary of the objects available in an image by opening and closing holes in the contours of an image as shown in Figure 3(a) and 3(b) (Stajnko et al., 2009). Opening and closing are two contrast terms in morphological methods. Contours with small, narrow and weak edge strength are broken in opening operation
IMAGE REPRESENTATION AND DESCRIPTION METHODS Main attributes used in fruit sorting are color, size, texture and shape of a fruit. These features alone cannot be considered as an important attributes
This article can be downloaded from http://www.ijerst.com/currentissue.php 37
Int. J. Engg. Res. & Sci. & Tech. 2013
J Satheesh Kumar and D Surya Prabha, 2013
in fruit sorting as these attributes are similar for multiple fruits. Shape of the fruit is also considered as a major attribute for sorting the fruits (Jimenez et al., 2000). Shape description is a process of extracting required feature from an image. Objects in an image can be represented in a numeric form. Shape of the fruit in an image is transformed into graphical representation. Statistical moments, boundary descriptors and regional descriptors are common methods used for sorting fruits. Statistical moments like mean, variance and smoothness can be used to measure the color features in the objects in an image (Gonzalez et al., 2010). Area, perimeter, eccentricity, Euler numbers and Euclidean measures can be used to extract size features in an image. Chain code, polygon, invariant
moments, B-spline, fourier descriptors and shape descriptors are used for extracting the shape features in an image. Chain code is one of the simple methods to classify objects in an image. It is a directional code that uses fragments of straight line based on the direction and length of line. Segments of straight line code used to form boundary in a unique direction is referred as a freeman chain code. Labels are assigned in all the direction of boundary lines. Starting point determines the accuracy of chain code. Fourier descriptor uses the Discrete Fourier Transforms and extends the functionality to identify the shape of objects in an image Figure 4(a) and 4(b). Numerous descriptors are combined together to determine
Figure 3 (A) Opening Operation Applied Output Image, (B) Closing Operation Applied Output Image, (C) and (D) Morphological Operation Applied Output Image with Different Filter Masks
This article can be downloaded from http://www.ijerst.com/currentissue.php 38
Int. J. Engg. Res. & Sci. & Tech. 2013
J Satheesh Kumar and D Surya Prabha, 2013
the shape of an object. Planar shapes, closed boundaries and open curve shapes can be easily extracted from Fourier descriptors. Skeleton is an important approach that reduces the object into a thin graphical representation as shown in Figure 4(c). Medial axis transformation is used to define the skeleton of an object in an image. Object region is identified by calculating the distance between the interior points and boundary points in an image.
are far away from centroid. Eigen axis points are determined using chain code algorithm. Centroid to boundary distance plotting is simplest and best method for recognizing shape of an object as shown in Figure 4(d). Straight, slightly curved and curved boundaries can be easily identified using this method. Rotating and scaling transformation methods can be applied in these boundary signatures but this method is invariant to translation. Shape signatures and skeletons based approach is always suggestible for better performance in sorting the fruits.
Shape signature is used to represent shape of an object in the form of 1D function. Shape signatures are formed by plotting points from centroid to boundary distance (Moreda et al., 2012). Another method for shape signature is done using plotting points from Eigen axis which
OBJECT RECOGNITION Acquired, preprocessed, segmented, identified and recognized fruit object enters object
Figure 4: (a) Original Image (b) Fourier Descriptor Applied Output Image (c) Skeleton Applied Output Image (d) Shape Signature Applied Output Image
This article can be downloaded from http://www.ijerst.com/currentissue.php 39
Int. J. Engg. Res. & Sci. & Tech. 2013
J Satheesh Kumar and D Surya Prabha, 2013
recognition as shown in Figure 5, the higher level concept of computer vision. Object recognition is used to recognize objects based on some learning and training concepts. Patterns and classif iers are the tool applied in object recognition. Patterns use certain descriptors to describe the pattern of an object. Objects are recognized and classified based on these pattern recognizers. Vectors, strings and trees are common collection of pattern recognizers involved in this method. Pattern matching is an object recognition technique used to match the objects based on their classes. Classifiers are used for this purpose of matching objects. These classifiers are also broadly classified into supervised and unsupervised methods. Numerous classifiers are available for pattern matching like minimum distance classifier, statistical classifiers, Bayesian classifiers and neural network classifiers (Vibhute and Bodhe, 2012). Minimum distance classifier is a simple method as it uses the concept of Euclidean measure to calculate the distance between the unknown and predefined patterns (Holalad et al., 2012).
making. Decision making situations are uncertain and unpredictable. It may be required to take simultaneous decisions within a short period of time. Bayes decision theory is a statistical concept of taking decisions using probability and cost effect on the decision. In fruit sorting, Bayes decision theory is used to sort the fruits like lemon, amla, apple and banana. Principal Component Analysis (PCA) is a commonly preferred statistical method used for recognizing objects in a higher level concept of computer vision. PCA is a pattern recognition method used face recognition, image compression and in various pattern recognition applications. In fruit sorting, PCA is used to classify and categorize the fruits in a fully automated computer vision system. It uses statistical tools like standard deviation, eigen vectors and covariance matrix. Support Vector Machines (SVM) is used to label the objects in an image by use of learning supervised and unsupervised learning technique. SVM can be trained to recognize objects in real time environment. SVM is a popularly used method for biomedical applications and it can also be integrated to work with other applications. Neural network is a machine learning technique used to classify different objects in an image. This method is inspired by the neural structure of the biological
Bayes decision theory is an important concept that supports the complicated task of decision
Figure 5: Flow Chart Depicting Object Recognition Methods
This article can be downloaded from http://www.ijerst.com/currentissue.php 40
Int. J. Engg. Res. & Sci. & Tech. 2013
J Satheesh Kumar and D Surya Prabha, 2013
nervous system. Quadratic Discriminant classifier, Logistic Regression, Support vector machines and k- Nearest Neighbors are the common statistical classifiers used for the purpose of sorting and recognizing fruits. Scale Invariant Feature Transform (SIFT) is an object recognition method that are trained to compare the objects with the predefined and stored objects in a database. Key features are extracted from an object and are stored in the database. Each object present in reference image is compared with the predefined objects in the database. Euclidean distance measures of vector is applied for matching the objects in image. Speeded Up Robust Features (SURF) is an object recognition method used to detect and describe the objects present in an image. This method is faster, accurate and efficient than the SIFT method in object recognition.
Recognition System Using Minimum Distance Classifier”, Journal of Information Engineering and Applications, Vol. 12, No .6, pp. 1-11. 3.
Locating Fruit on Trees”, Transaction of the ASAE, Vol. 43, No. 6, pp. 1911-1920. 4.
Harvesting Robots – State of the Art And Challenges Ahead”, International Journal Computational Vision and Robotics, Vol. 3, Nos 1/2, pp. 4-34. 5.
Mondal K, Dutta P and Bhattacharyya S (2012), “Fuzzy Logic Based Gray Image Extraction and Segmentation”, International Journal of Scientific and Engineering Research, Vol. 3, No. 4, pp.1-14
Three dimensional object recognition and classification methods have higher influence on agricultural based applications. Implementation of better and accurate object recognition and classification algorithms will have higher impact on fruit export management. This paper deals detailed study on various recognition, detection and classification methods used for 3D objects. This paper also explains importance of these methods on agricultural applications such as, fruit quality and grading analysis.
6.
Moreda G P, Munoz M A, Ruiz-Altisent M and Perdigones A (2012), “Shape Determination of Horticultural Produce Using Twodimensional Computer Vision – A Review”, Journal of Food Engineering, Vol. 108, pp. 245-261.
7.
Sinoquet H, Thanisawanyangkura S, Mabrouk H and Kasemsap P (1998), “Characterization of the Light Environment in Canopies Using 3D Digitising and Image Processing”, Annals of Botany, Vol. 82, pp.
REFERENCES
2.
Kapach K, Barnea E, Mairon R and Edan Y (2012 ), “Computer Vision for Fruit
CONCLUSION
1.
Jimenez A R, Ceres R and Pons J L (2000), “A Survey of Computer Vision Methods for
203-212.
Gonzalez C R, Woods R E and Eddins S L (2010), Digital image processing using MATLAB, 2nd Edition, Tata McGraw-Hill.
8.
Stajnko D, Rakun J and Blanke M (2009), “Modelling Apple Fruit Yield Using Image Analysis for Fruit Colour, Shape and Texture”, European Journal of Horticultural Science, Vol. 74, No. 6, pp. 260-267.
Holalad H S, Warrier P and Sabarad A D (2012), “An FPGA Based Efficient Fruit
This article can be downloaded from http://www.ijerst.com/currentissue.php 41
Int. J. Engg. Res. & Sci. & Tech. 2013
9.
J Satheesh Kumar and D Surya Prabha, 2013
Vibhute A and Bodhe S K (2012), “Applications of Image Processing in Agriculture: A Survey”,
International Journal of Computer Applications, Vol. 52, No. 2, pp. 34-40.
This article can be downloaded from http://www.ijerst.com/currentissue.php 42
