Food Anal. Methods DOI 10.1007/s12161-015-0097-7
Detection of Early Rottenness on Apples by Using Hyperspectral Imaging Combined with Spectral Analysis and Image Processing Baohua Zhang & Shuxiang Fan & Jiangbo Li & Wenqian Huang & Chunjiang Zhao & Man Qian & Ling Zheng
Received: 16 October 2014 / Accepted: 12 January 2015 # Springer Science+Business Media New York 2015
Abstract Detection of early rottenness on apples is still a challenging task for the automatic grading system due to the highly similarity between the rotten and sound tissues both in spectral and spatial domains. This research was conducted to develop an algorithm for detecting the early rottenness on apples by using hyperspectral reflectance imaging system combined with spectral analysis and image processing. In spectral domain, chemometric and pattern recognition methods were conducted for spectral analysis. In order to select the candidate optimal wavelengths that carry the most important information for distinguishing the rottenness from the sound tissues, successive projections algorithm (SPA) was conducted on the full range of average spectra extracted from the (regions of interest (ROIs) of sound and rotten tissues. The efficiency of the selected candidate optimal wavelengths for rottenness detection was also testified by using a binary particle least square discriminant analysis (PLS-DA) classifier in the spectral domain. In spatial domain, combined image processing methods were conducted for spatial analysis. In order to verify that the images at the optimal wavelengths were efficient and develop a robust detection algorithm, both principal component analysis (PCA) and minimum noise fraction (MNF) combined with conventional image processing methods were conducted on the images at the optimal wavelengths for image processing. Finally, the whole detection algorithm based on SPA-PLS-DA-MNF by using hyperspectral B. Zhang : S. Fan : J. Li : W. Huang : C. Zhao (*) : M. Qian : L. Zheng Beijing Research Center of Intelligent Equipment for Agriculture, Beijing Academy of Agriculture and Forestry Sciences, 100097 Beijing, China e-mail:
[email protected] B. Zhang : C. Zhao State Key Laboratory of Mechanical System and Vibration, Shanghai Jiao Tong University, Shanghai 200240, China
imaging combined with spectral analysis and image processing was designed and testified by the 120 testing apples. The results with 98 % overall detection accuracy indicated that the proposed algorithm was efficient and suitable for the rottenness apple detection. This research provides a foundational basis to develop the fast online inspection or real-time monitoring system for rottenness detection on apples both in the postharvest processing line or storage shelf.
Keywords Hyperspectral imaging . Rottenness detection . Spectral analysis . Image processing
Introduction Apple is one of the major fruits produced and consumed worldwide. About 70 million tons of apples have been produced worldwide since 2010, and China produced almost half of this total. With increased concerns for fruits of high quality and safety standards, the need for automatic, accurate, rapid, and objective quality inspection continues to grow (Shang et al. 2014; Patel et al. 2012; Wang et al. 2014; Zhang et al. 2015; Ding et al. 2014). Rottenness is one of the most serious and frequently occurred external defects on apples. Rottenness can be caused by many factors, such as physical damage happened in the postharvest processing and transportation stage and fungal infection. Rottenness on apples not only influences their appearance quality but also has high risk to spread the infection to the whole patch and cause great economic losses (Gomez-Sanchis et al. 2008). Hence, rotten apples should be detected and weeded out at the postharvest handing and processing stage.
Food Anal. Methods
Nowadays, the detection of rottenness on apples is mainly conducted manually during the apples being transported on the sorting lines under the ultraviolet illumination. However, rotten apple detection manually is a time-consuming and labor intensive work due to the large amount of production, high variability of apple surface color, and high similarity between the rotten and sound tissues both in spectral and spatial domains (Mehl et al. 2004; Gomes and Leta 2012; Kim et al. 2007). Besides, the ultraviolet illumination is harmful for the health of sorting workers. Over the past decades, parallel to the computer science development, the rapid and nondestructive automatic inspection of quality of fruits and vegetables based on computer vision, image processing, and spectral analysis techniques arises as a scientific and powerful tool in food processing industry. Computer vision, which has been widely used for the quality monitoring and inspection in food industry, is an engineering technology that imitates the vision of the human eyes by capturing images using three filters centered at red (R), green (G), and blue (B) wavelengths (Lorente et al. 2012). However, the color and texture of early rotten tissues and sound tissues are highly similar, and the color of sound tissues is highly various; it is impossible or very difficult to detect the early rottenness on apples by using the traditional computer vision systems for lack of spectral and multiconstituent information in conventional color RGB images. Therefore, very few researches that deal with the detection of apple rottenness through traditional computer vision systems have appeared in the literature (Gomes and Leta 2012; Zhang et al. 2014; Patel et al. 2012; Cubero et al. 2011). Hyperspectral imaging is an emerging technique that integrates both spectroscopic and imaging techniques into one system. With the help of wavelength-dispersive devices, high-resolution cameras, and the recent advances in hardware and software of a computer, hyperspectral imaging has been developed as an efficient inspection tool for the quality and safety monitoring of a variety of food and agricultural products (Liu et al. 2014; Yu et al. 2014a, b; Cho et al. 2013; Lorente et al. 2012; Qin et al. 2013; ElMasry et al. 2012; He et al. 2014; He and Sun 2014). A typical hyperspectral image is composed of a set of monochromatic images corresponding to almost continuous wavelengths. Some unobvious defects in RGB color images are always easy to identify in one or several waveband images. Therefore, the hyperspectral imaging system has the natural advantage compared to the traditional computer vision systems (Zhang et al. 2014), and this kind of imaging system makes it possible to conduct a more sophisticated spectral and spatial analyses to extract features that are not easy to detect in RGB color images. However, hyperspectral imaging is often used to collect images with high spatial and spectral resolutions for fundamental research. The process usually involves a significant amount of time for image acquisition and processing under laboratory condition
(Qin et al. 2013). In order to realize the rapid online detection of the rottenness on apples, it is important and necessary to select the optimal wavelength images that carry the most important information to distinguish the rottenness from the sound tissue. Various attempts have been made to select the optimal character wavelengths and process hyperspectral images in previous studies. Li et al. (2011) investigated to identify the common defects on oranges by using hyperspectral reflectance imaging system. Six wavelengths in visible spectra and near-infrared regions were selected as the optimal wavelengths by using PCA transform. Finally, PCA transform and band ratio methods based on the selected wavelengths were used to develop the final detection algorithm. There are many other approaches available to select optimal wavelengths for the detection of early bruise (Xing et al. 2005; ElMasry et al. 2008; Baranowski et al. 2012; Zhang et al. 2014), rottenness in mandarins (Gomez-Sanchis et al. 2008), physical damages of pears (Lee et al. 2014), defects in loquat (Yu et al. 2014a, b), defects, and contamination (Mehl et al. 2004). However, few researches have been conducted with the respect to early rottenness detection on apples by using hyperspectral imaging combined with spectral analysis and image processing. In our study, visible and near-infrared hyperspectral imaging combined with spectral analysis and image processing was firstly used to investigate the feasibility for detection of early rottenness on apples. Several steps have to be fulfilled in order to achieve the above main objective: (1) Acquiring the hyperspectral images (in the spectral region of 400– 1000 nm) of apples, (2) analyzing the hyperspectral images by chemometric methods in spectral domain for optimal wavelength selection, (3) establishing a binary classification model to verify the efficiency of the selected optimal wavelengths, (4) developing combined image processing methods to verify the detection performance in the spatial domain, (5) and developing and testifying the whole detection algorithm combined with spectral analysis and image processing.
Materials and Methods Apples Used in the Experiments Fuji apples were purchased from a local fruit market in Beijing, China, in May 2014. A total of 120 apples with various size and shape were selected as the experimental samples in our study. Twenty Fuji apples of the experimental samples were sound apples, and the rest 100 apples were inoculated with Penicillium in the middle area between stem and calyx. The infected apples become rotten after storing for 3 to 4 days in the natural indoor environment condition. The controlled rotten areas with diameters from 5 to 15 mm were obtained, and the early rotten areas present very light yellow substantia
Food Anal. Methods
gelatinosa, and the surface color and texture of the early rotten areas are highly similar to those of the sound areas. All the apples were stored in our inspection laboratory before acquiring the hyperspectral images. Twenty apples with rottenness were used to conduct spectral analysis for optimal wavelength selection and classification model verification in spectral domain, as well as develop image processing algorithm in spatial domain. The other samples, including 20 sound apples and eighty samples with rottenness, were used for testing the performance of whole detection algorithm combined with spectral analysis and image processing.
Hyperspectral Imaging System Figure 1 shows the configuration of the hyperspectral imaging system used in our research. As shown in Fig. 1, the hardware of the imaging system generally consists of the following components: a computer (Dell, Inter(R) Core(TM) i5-2400 CPU @3.10 GHz, RAM 4.0 GB), an Andor monochrome liner EMCCD (Andor Luca DL-604M, Andor Technology plc., N. Ireland) with 1004×1000 pixels, an imaging spectrograph (ImSpector V10E-QE, Spectral Imaging Ltd., Finland) coupled with a standard C-mount zoom lens (V23-f/2.4, Specim Ltd., Finland), two 150-W halogen lamp assemblies (3900-ER, Illumination Technologies, Inc., USA), and a mobile platform. The two halogen lamps were fixed at the both upsides of the mobile platform at an angle of 45° with the height of 40 cm. The imaging spectrograph works in line scanning mode and covers the spectral range from 326.7 to 1098 nm with a spectral resolution of 0.8 nm. The software in our research was provided by Isuzu Optics Corp, Taiwan, China. The hyperspectral image acquisition Fig. 1 The schematic diagram of the developed hyperspectral imaging system
parameters, speed of the motor, and start and end positions of the acquisition could be set by the software. Characters of the Hyperspectral Images A hyperspectral image is a three-dimensional (3-D) block of data containing a stack of two-dimensional images one behind each other at different wavelength (Elmasry et al. 2012). The conceptual view of a hyperspectral image of an apple with rottenness is illustrated in Fig. 2. The hyperspectral images can be viewed either as a spectra I(λ) at each individual pixel (x, y) or as an image I(x, y) at each individual wavelength λ (Liu et al. 2013; Elmasry et al. 2012). Each image acquires spatially distributed spectral information at pixel level and can be used to analyze the rottenness distribution according to the spatial information. Each pixel containing a complete spectrum can be used to characterize the peel condition. Therefore, the processing and analysis of a hyperspectral image can be conducted in the spatial domain (image processing), or conducted in the spectral domain (spectral analysis), or conducted combined with the spatial domain and spectral domain (spectral analysis and image processing). Hyperspectral Image Acquisition and Calibration The rottenness of fruit is always viewed as a typical external defect. Hyperspectral imaging system worked in reflectance mode is considered to be the most suitable approach to detect the early rottenness. Therefore, all samples were scanned line by line by the hyperspectral imaging system with an adjustable motor speed (0.8 mm/s) in the reflectance mode in our research. The distance between samples and lens was set to 65 cm. Each apple of the experimental set was imaged with a
Food Anal. Methods
Fig. 2 The conceptual view of a hyperspectral image with spectral and spatial domains
single acquisition. It is noted that the apples with rottenness were captured in certain position posed manually with the rottenness right up to the lens. A total of 120 hyperspectral images were obtained in our research. A white reference image and a dark reference image can be used to transform the raw hyperspectral images to hyperspectral reflectance images. The hyperspectral reflectance image R for a spatial pixel (i) at a given wavelength was calculated by using the following equation (Lee et al. 2014): Ri ¼
RS i −RDi RW i −RDi
100%
ð1Þ
where RS, RD, and RW are the raw intensity values of identical pixels from the sample image, dark reference image, and white reference image, respectively. Ri is the calibrated hyperspectral reflectance image. The dark reference image RD (with ~0 % reflectance) represents the dark response of the camera. The white reference image RW (with ~99.9 % reflectance) represents the highest intensity values. The dark reference image RD can be acquired by measuring a spectral image with the light source turned off completely and the camera lens covered completely with its nonreflective opaque black cap. The dark reference image RD can be acquired by
measuring a spectral image of the Teflon white board with a 99.9 % reflectance. Spectral Analysis In order to select the optimal wavelengths that carry the most important information for distinguishing the rottenness from the sound tissue, spectral analysis by using chemometric and pattern recognition methods was conducted in the spectral domain of the hyperspectral image. Both the average spectra of the rotten and sound tissues were collected by averaging the spectral values of all the pixels in a rectangular region of interest (ROI). Each ROI contains about 40 to 60 pixels. Four average spectra of both rotten and sound tissues were collected for each apple in the training set; then, a total of 160 average spectra were collected. While the average spectra collected from the hyperspectral image were in the range of 326.7 to 1098 nm, only the spectra between 400 and 1000 nm were taken into account for the spectral analysis. Outside this range, single to noise ratios were too low due to the attenuation of the charge-coupled device (CCD) detector response in these wavelengths. It is noted that the subsequent processing and analysis in spectral domain and spatial domain would be conducted in this selected range. In order to simplify the complexity of computation, improve the efficiency of the rottenness detection, and meet the
Food Anal. Methods
inspection speed required by the industry, variable selection (wavelength selection) is the most necessary and important step to select the optimal variables and remove the highly corrected variables (Liu et al. 2014). The successive projections algorithm (SPA), a forward selection method which uses simple operations in a vector space to minimize variable collinearity, is a novel variable selection strategy in hyperspectral image analysis for multivariate calibration (Wu et al. 2012). SPA employs a simple projection operation in a vector space to select subsets of variables with a minimum of collinearity. Generally, SPA comprises two phases. The first phase consists of projections carried out on the spectral matrix, which generate candidate subsets of variables with minimum collinearity. The second phase consists of evaluating candidate subsets of variables according to the root-mean-square error of validation (RMSEV) value obtained by applying the resulting multilinear regression (MLR) model. Then, variable elimination procedures can be used to remove uninformative variables without significant loss of prediction capability (Galvao et al. 2008; Liu et al. 2014). Many successful applications have proven SPA to be an outstanding variable selection approach (Li et al. 2014; Fan et al. 2014; Wu et al. 2012; Feng and Sun 2012; Kamruzzaman et al. 2013). In order to remove the redundancy of hyperspectral image data, simplify the complexity of computation, and select the candidate optimal wavelength that carry the most important information, SPA was carried out on the full spectral data collected from the training set. More details about SPA method can be found in studies of Galvao et al. (2008) and Liu et al. (2014). In order to evaluate the efficiency of the optimal wavelengths selected by SPA, a binary particle least square discriminant analysis (PLS-DA) model was used to classify the ROI to sound or rottenness ones based on the selected wavelengths in spectral domain.
combination of each monochromatic image by using the following equation (Cho et al. 2013): PC img ¼
n X
ωi S i
ð2Þ
i¼1
where ωi is the weighting coefficient and Si is the image at the selected optimal wavelengths. The weighted values were calculated based on a covariance matrix of the images, and it represented the variance of each PC image. More details about PCA can be found in the studies of Li et al. (2011), Xing et al. (2005), and Zhang et al. (2014). Minimum noise fraction (MNF) rotation transform is also commonly used in the hyperspectral image analysis. MNF is a more complex transformation compared to PCA. MNF rotation transform contains two cascaded PCA transformations of the raw hyperspectral image (Yu et al. 2014a, b; Baranowski et al. 2012). The first PCA transformation of MNF is the PCs of the noise covariance matrix to decorrelate and rescale the noise in the hyperspectral image data. The first rotation results in data in which the noise has unit variance and no band to band correlation. The second PCA transformation of MNF is a standard PC transformation of the noise-whitened data. More details about MNF can be found in the studies of Baranowski et al. (2012), Yu et al. (2014a, b), and Zhang et al. (2014). In this paper, the performance of PCA transform and MNF rotation transform conducted on the selected optimal wavelengths would be comparatively studied. The conventional image processing algorithms, such as thresholding, masking, hole filling, and morphological filtering, would also be conducted to segment the rottenness in spatial domain.
Image Processing Results and Discussion Once the optimal wavelengths were selected, the rottenness detection would be a problem of image processing and classification. The ultimate aim of this study is to develop an efficient image processing and classification algorithm based on the selected wavelengths that could be easily applied toward developing a fast and low-cost multispectral detection system. Principal component analysis (PCA) is the most widely used approach in the hyperspectral imaging system for the external quality inspection of food and agricultural products (Liu et al. 2014; Feng and Sun 2012; ElMasry et al. 2012; Li et al. 2011). PCA transforms the hyperspectral image into sequence of principal component images (PCIs) where the first several component images can be used to represent the overwhelming majority of the information contained in the original hyperspectral image. Each PC image is the linear
Spectral and Spatial Features of Rotten and Sound Apple Tissues In order to discover the efficient features to distinguish the early rottenness from the sound tissue, both the spectral features and spatial features were extracted from the spectral and spatial domains. The spectral and spatial features of the rotten and sound tissues in hyperspectral image are shown in Fig. 3 and would be discussed and analyzed in detail in the following sections. The average spectra of ROIs of different condition peel (green color tissues, light red tissues, red tissues, dark red tissues, and rotten tissues) were extracted. Due to the different reflectance of different peel tissues and different distribution of lightness reflected from the apple sample with irregular
Food Anal. Methods Fig. 3 The spectral and spatial features of the rotten and sound tissues in hyperspectral image
shape, the spectra of the same condition peel were always different. The spectra of the different sound tissues and rotten tissues were studied in the wavelength range between 400 and 1000 nm. The large variable ranges of spectra from different sound tissues were shown in two solid black lines filled with light blue color. The variable ranges of spectra from different rotten tissues were shown in two dotted red lines filled with vertical bar. The spectral features clearly showed that the spectra of the rotten ROIs were completely overlapped by those of sound ones from the wavelength of 400 to 940 nm. The spectra of sound and rotten tissues also shared the similar shapes and trends. This could be explained why it is difficult to distinguish the rottenness from sound tissues only by using spectral features in the spectral domain. In order to attempt to distinguish the rottenness from sound peel, the spatial features were also studied in the separated monochromatic image in the spatial domain. Both the average intensity and textures of the rotten and sound regions were extracted; however, no obvious difference was found. One monochromatic image at 590 nm and its grayscale histogram are also shown in Fig. 3. It is noted that before calculating the statistical histogram, the pixels in the background were
excluded by using a solid mask image which only contained the apple. The average intensity of the rottenness ROI was about 180. The grayscale histogram did not have a deep and sharp valley between two peaks representing sound and rotten tissues, respectively. Therefore, it was also difficult to identify the rotten peel from the sound peel just by using spatial features in the spatial domain. It is also noted that it is not easy to detect the rottenness for the color camera which can only capture three images at 700 nm (red wavelength), 546.1 nm (green wavelength), and 435.8 nm (blue wavelength) due to the overlap in the visible spectrum.
The Spectral Analysis for Rottenness Detection in Spectral Domain The spectral analysis in the spectral domain consists of the following two sub-objectives: candidate optimal wavelength selection by using chemometric methods and verification by using pattern recognition method. All the processing and analysis for the above sub-objectives were conducted in the spectral domain.
Food Anal. Methods
The collected 160 average spectra were labeled as 0 (rottenness tissue class, 80 spectra) or 1 (sound tissue class, 80 spectra) according to their true classes. The 160 spectra were divided into two subsets: one subset containing 120 spectra (60 spectra for rotten tissues and 60 for sound tissues) was used for candidate optimal wavelength selection and verification modeling; the other subset containing 40 spectra (20 spectra for rottenness tissue and 20 for sound tissue) was used for the efficient verification of the selected candidate optimal wavelength by using a binary PLS-DA model. During the process of the SPA, a SPA-multivariable linear regression (MLR) procedure was applied for the calculation of a sequence of root-mean-square error (RMSEP) values using the selected variable subsets (Li et al. 2014). Figure 4 shows the RMSEP scree plot as the number of the selected variables increasing from 0 to 28. As shown in the figure, the RMSEP values sharply declined as SPA selected the first two variables; then, the RMSEP values gradually descended as the number of selected wavelength increasing from 2 to 5. After the number of the selected wavelength greater than 5, the descending trend was not obvious. When four candidate optimal wavelengths were selected for MLR analysis as shown in open square markers, the RMSEP reached its optimal value (0.078 Brix) for prediction of the class of the apple tissues.
Fig. 4 The RMSEP scree plot for the number of selected variables obtained by applying SPA
The selected four candidate optimal wavelengths (labeled as open square marker) and their distribution in the calibration spectra are shown in the Fig. 5. The selected four candidate optimal wavelengths were 563, 611, 816, and 966 nm. The four selected wavelengths were preliminarily recognized as the candidate optimal wavelengths that carried the most important information for distinguishing the rotten tissues from the sound ones. In order to verify the efficiency of the four selected candidate optimal wavelengths, a binary PLS-DA model was designed to evaluate the detection performance of the candidate optimal wavelengths in spectral domain. PLS-DA is a widely used multivariate classification method based on PLS. In our research, a binary PLS-DA classifier was trained based on the training set. The input features were the reflectance values in the four selected candidate optimal wavelengths, and the output was a black (labeled as 0, represented rotten tissues) or white (labeled as 1, represented sound tissues) decision. If the selected candidate optimal wavelengths did carry the important information for distinguishing the rotten tissues from the sound ones, good classification results would be obtained. All the spectra (only the reflectance values in the four selected candidate optimal wavelengths) in the test set would be classified as either the rottenness tissue or the sound tissue. In our research, all the spectra were correctly classified based on the
Food Anal. Methods Fig. 5 The selected four wavelengths and their distribution in the calibration spectra
four selected candidate optimal wavelengths. The results indicated that the four selected wavelengths (563, 611, 816, and 966 nm) did carry the most important information for distinguishing the rotten tissues from the sound ones. Though the selected wavelengths are efficient for the rottenness detection in the spectral domain, however, whether they are still efficient in the spatial domain is needed to be verified. The Image Processing Algorithm for the Rottenness Detection in Spatial Domain It was already known that the selected wavelengths were efficient for the rottenness detection in the spectral domain. In order to verify their detection performance in spatial domain and develop a more suitable image processing method, PCA, MNF, and some conventional image processing methods were conducted on the images at the selected optimal wavelengths. The PCA and MNF transform results are shown in Fig. 6. PC1 and MNF1 score images mainly illustrated the effects of concave shape of the sample and the uneven illumination distribution in the sample and other physical properties. PC2 and MNF2 score images demonstrated the areas where the illumination was saturated. PC3 and MNF3 score images were the suitable images for the rottenness detection in our research due to the best discriminant between the rotten and sound tissues. PC4 and MNF4 score images demonstrated the spots, textures, and other appearance features. Figure 6(a) shows the rottenness region segmented from the PC3 score image, and
Fig. 6(b) shows the final rottenness detection result after a series of image processing (morphological filling, eroding, and dilating) in Fig. 6(a). Figure 6(c) shows the rottenness region segmented from the MNF3 score image, and Fig. 6(d) shows the final rottenness detection results after morphological processing (morphological filling, eroding, and dilating) in Fig. 6(c). It was clear that detection results by using MNF based on selected optimal wavelengths could get a better performance compared to that of PCA. This was not a special case; actually, the other apples in the training set showed the similar detection results by using PCA and MNF transform. MNF-based image processing method was finally used in our research due to its excellent discriminated performance based on the images at the selected optimal wavelengths. The whole detection algorithm combined with spectral analysis and image processing is photographically demonstrated in Fig. 7. The algorithm for rottenness detection mainly comprises two parts: spectral analysis in spectral domain and image processing in spatial domain. Spectral analysis consists of four steps: spectra collection from ROIs of rotten and sound tissues, preliminary optimal wavelength selection by using SPA method, the selected candidate optimal wavelength validation by using a PLS-DA model, and final optimal wavelength determination. Image processing consists of the following steps: the images at the optimal wavelength extraction, MNF transform based on the selected images, and rottenness recognition and segmentation.
Food Anal. Methods
Fig. 6 The whole detection algorithm combined with spectral analysis and image processing. a Mask image obtained by using segmenting and morphological filling in 816-nm image. b Rottenness region segmented
from the MNF3 score image. c Final rottenness detection result after morphological processing
Fig. 7 Rottenness detection results of PCA and MNF based on selected optimal wavelength. a Rottenness region segmented from the PC3 score image. b Final rottenness detection result after a series of image processing (morphological filling, eroding, and dilating) in a. c
Rottenness region segmented from the MNF3 score image. d Final rottenness detection result after morphological processing (morphological filling, eroding, and dilating) in c
Food Anal. Methods
Fig. 8 Examples of rottenness detection results and their main intermediate processed images. a Rottenness region segmented from the MNF3 score image. b Final detection results
Verification Results of the Detection Algorithm
Conclusion
In order to verify the detection performance of the developed spectral and image processing algorithm based on the selected images at the optimal wavelength, all the samples in the testing set (20 sound apples and 80 samples with rottenness) were processed by the algorithm. All the 20 sound apples were recognized as sound apples, and the detection accuracy was 100 %. Two apple samples with rottenness were misclassified as sound apples, and the detection accuracy for rotten apples was 97.5 %. The reasons that causing misclassification were that one samples’ rottenness region in the MNF3 presented black and the other samples’ rottenness region presented white in the MNF2 image; this might be caused by the color variance on apples and the uneven distribution of lightness. Next step, we would be focused on the lightness and spectral correction in hyperspectral images. The results with 98 % overall detection accuracy for the 100 apples in the testing set indicated that the proposed spectral- and image-based algorithm was efficient and suitable for the rottenness apple detection. Figure 8 shows some examples of rottenness detection results and their main intermediate processed images.
In our research, detection of early rottenness on apples was investigated by using hyperspectral imaging system combined with spectral analysis in spectral domain and image processing in spatial domain. In order to select the candidate optimal wavelengths that carried the most important information for distinguishing the rotten tissues from the sound ones, SPA was conducted in the full region of spectra for spectral analysis in the spectral domain. To verify the efficiency of the selected candidate optimal wavelengths, a binary PLS-DA classifier was also used to classify the ROIs as either rotten or sound tissues only based on the selected four candidate optimal wavelengths in the spectral domain. The result with 100 % classification accuracy for the 40 average spectra in the testing set indicated that the selected candidate optimal wavelengths did carry the important information for distinguishing the rottenness from the sound tissues in the spectral domain. In order to testify the efficiency of the images at the selected optimal wavelengths, PCA, MNF, and conventional image processing methods were conducted to identify the rottenness based on the images at the selected wavelengths in the spatial domain.
Food Anal. Methods
Detection results showed that a better performance could be obtained by using MNF transform combined with conventional image processing methods. Finally, the whole detection algorithm combined with spectral analysis and image processing was developed, and the satisfied detection results were obtained. Although the satisfied detection results were obtained, there is something interesting future work to be done to apply the algorithm to the online or real-time inspection systems. One is that the images at the optimal wavelengths were extracted from the hyperspectral images; whether it is still efficient if they were directly obtained from the multispectral imaging system is need to be verified. The other thing is how to realize the whole surface detection required by the industry. The next work will be focused on these two interesting future researches. Acknowledgments This work was supported by the Young Scientist Fund of National Key Technology R&D Program (project No. 2014BAD21B01) and National Natural Science Foundation of China (project No. 31301236). Conflict of Interest Baohua Zhang, Shuxiang Fan, Jiangbo Li, Wenqian Huang, and Chunjiang Zhao declare that they have no a financial relationship with the organization that sponsored the research and also have no conflict of interest. This article does not contain any studies with human or animal subjects.
References Baranowski P, Mazurek W, Wozniak J, Majewska U (2012) Detection of early bruises in apples using hyperspectral data and thermal imaging. J Food Eng 110:345–355 Cho B-K, Kim MS, Baek I-S, Kim D-Y, Lee W-H, Kim J, Bae H, Kim Y-S (2013) Detection of cuticle defects on cherry tomatoes using hyperspectral fluorescence imagery. Postharvest Biol Technol 76:40–49 Cubero S, Aleixos N, Molto E, Gomez-Sanchis J, Blasco J (2011) Advances in machine vision applications for automatic inspection and quality evaluation of fruits and vegetables (vol 4, pg 487, 2011). Food Bioprocess Technol 4:829–830 Ding Z, Zhang R, Kan Z (2014) Quality and safety inspection of food and agricultural products by LabVIEW IMAQ Vision. Food Anal Methods 1–12 ElMasry G, Wang N, Vigneault C, Qiao J, ElSayed A (2008) Early detection of apple bruises on different background colors using hyperspectral imaging. Lwt-Food Sci Technol 41:337–345 Elmasry G, Kamruzzaman M, Sun DW, Allen P (2012) Principles and applications of hyperspectral imaging in quality evaluation of agrofood products: a review. Crit Rev Food Sci Nutr 52:999–1023 Fan S-X, Huang W-Q, Li J-B, Zhao C-J, Zhang B-H (2014) Characteristic wavelengths selection of soluble solids content of pear based on NIR spectral and LS-SVM. Spectrosc Spectr Anal 34(8):2089–2093 Feng YZ, Sun DW (2012) Application of hyperspectral imaging in food safety inspection and control: a review. Crit Rev Food Sci Nutr 52: 1039–1058 Galvao RKH, Araujo MCU, Fragoso WD, Silva EC, Jose GE, Soares SFC, Paiva HM (2008) A variable elimination method to improve
the parsimony of MLR models using the successive projections algorithm. Chemom Intell Lab Syst 92:83–91 Gomes JFS, Leta FR (2012) Applications of computer vision techniques in the agriculture and food industry: a review. Eur Food Res Technol 235:989–1000 Gomez-Sanchis J, Gomez-Chova L, Aleixos N, Camps-Valls G, Montesinos-Herrero C, Molto E, Blasco J (2008) Hyperspectral system for early detection of rottenness caused by Penicillium digitatum in mandarins. J Food Eng 89:80–86 He HJ, Sun DW (2014) Inspection of harmful microbial contamination occurred in edible salmon flesh using imaging technology. J Food Eng. doi:10.1016/j.jfoodeng.2014.10.012 He HJ, Sun DW, Wu D (2014) Rapid and real-time prediction of lactic acid bacteria (LAB) in farmed salmon flesh using near-infrared (NIR) hyperspectral imaging combined with chemometric analysis. Food Res Int 62:476–483 Kamruzzaman M, ElMasry G, Sun D-W, Allen P (2013) Nondestructive assessment of instrumental and sensory tenderness of lamb meat using NIR hyperspectral imaging. Food Chem 141:389–396 Kim MS, Chen Y-R, Cho B-K, Chao K, Yang C-C, Lefcourt AM, Chan D (2007) Hyperspectral reflectance and fluorescence line-scan imaging for online defect and fecal contamination inspection of apples. Sens & Instrumen Food Qual 1:151–159 Lee W-H, Kim MS, Lee H, Delwiche SR, Bae H, Kim D-Y, Cho B-K (2014) Hyperspectral near-infrared imaging for the detection of physical damages of pear. J Food Eng 130:1–7 Li JB, Rao XQ, Ying YB (2011) Detection of common defects on oranges using hyperspectral reflectance imaging. Comput Electron Agric 78: 38–48 Li J, Huang W, Chen L, Fan S, Zhang B, Guo Z, Zhao C (2014) Variable selection in visible and near-infrared spectral analysis for noninvasive determination of soluble solids content of BYa^ pear. Food Anal Methods 1–12 Liu D, Zeng X-A, Sun D-W (2013) Recent developments and applications of hyperspectral imaging for quality evaluation of agricultural products: a review. Crit Rev Food Sci Nutr Liu D, Sun D-W, Zeng X-A (2014) Recent advances in wavelength selection techniques for hyperspectral image processing in the food industry. Food Bioprocess Technol 7:307–323 Lorente D, Aleixos N, Gomez-Sanchis J, Cubero S, Garcia-Navarrete OL, Blasco J (2012) Recent advances and applications of hyperspectral imaging for fruit and vegetable quality assessment. Food Bioprocess Technol 5:1121–1142 Mehl PM, Chen YR, Kim MS, Chan DE (2004) Development of hyperspectral imaging technique for the detection of apple surface defects and contaminations. J Food Eng 61:67–81 Patel KK, Kar A, Jha SN, Khan MA (2012) Machine vision system: a tool for quality inspection of food and agricultural products. J Food Sci Technol Mysore 49:123–141 Qin JW, Chao KL, Kim MS, Lu RF, Burks TF (2013) Hyperspectral and multispectral imaging for evaluating food safety and quality. J Food Eng 118:157–171 Shang L, Guo WC, Nelson SO (2014) Apple variety identification based on dielectric spectra and chemometric methods. Food Anal Methods 1–11 Wang NN, Yang YC, Sun D-W, Pu H, Zhu Z (2014) Shelf-life prediction of ‘Gros Michel’ bananas with different browning levels using hyperspectral reflectance imaging. Food Anal Methods 1–12 Wu D, Sun D-W, He Y (2012) Application of long-wave near infrared hyperspectral imaging for measurement of color distribution in salmon fillet. Innov Food Sci Emerg Technol 16:361– 372 Xing J, Bravo C, Jancsok PT, Ramon H, De Baerdemaeker J (2005) Detecting bruises on ‘Golden Delicious’ apples using hyperspectral imaging with multiple wavebands. Biosyst Eng 90:27–36
Food Anal. Methods Yu K, Zhao Y, Li X, Shao Y, Zhu F, He Y (2014a) Identification of crack features in fresh jujube using Vis/NIR hyperspectral imaging combined with image processing. Comput Electron Agric 103:1–10 Yu K-Q, Zhao Y-R, Liu Z-Y, Li X-L, Liu F, He Y (2014) Application of visible and near-infrared hyperspectral imaging for detection of defective features in loquat. Food Bioprocess Technol 1–11
Zhang B, Huang W, Li J, Zhao C, Fan S, Wu J, Liu C (2014) Principles, developments and applications of computer vision for external quality inspection of fruits and vegetables: a review. Food Res Int 62: 326–343 Zhang B, Huang W, Gong L, Li J, Zhao C, Liu C, Huang D (2015) Computer vision detection of defective apples using automatic lightness correction and weighted RVM classifier. J Food Eng 146:143– 151
