Evaluation of pork color by using computer vision

Meat Science 56 (2000) 57±60

www.elsevier.com/locate/meatsci

Evaluation of pork color by using computer vision

$

J. Lu a, J. Tan a,*, P. Shatadal a, D.E. Gerrard b a

Department of Biological and Agricultural Engineering, University of Missouri,Columbia, MO 65211,USA b Department of Animal Sciences, Purdue University,West Lafayette, IN 47907, USA Received 16 September 1999; received in revised form 27 January 2000; accepted 27 January 2000

Abstract The objective of this study was to determine the potential of computer vision technology for evaluating fresh pork loin color. Software was developed to segment pork loin images into background, muscle and fat. Color image features were then extracted from segmented images. Features used in this study included mean and standard deviation of red, green, and blue bands of the segmented muscle area. Sensory scores were obtained for the color characteristics of the lean meat from a trained panel using a 5-point color scale. The scores were based on visual perception and ranged from 1 to 5. Both statistical and neural network models were employed to predict the color scores by using the image features as inputs. The statistical model used partial least squares technique to derive latent variables. The latent variables were subsequently used in a multiple linear regression. The neural network used a back-propagation learning algorithm. Correlation coecients between predicted and original sensory scores were 0.75 and 0.52 for neural network and statistical models, respectively. Prediction error was the di€erence between average sensory score and the predicted color score. An error of 0.6 or lower was considered negligible from a practical viewpoint. For 93.2% of the 44 pork loin samples, prediction error was lower than 0.6 in neural network modeling. In addition, 84.1% of the samples gave an error lower than 0.6 in the statistical predictions. Results of this study showed that an image processing system in conjunction with a neural network is an e€ective tool for evaluating fresh pork color. # 2000 Elsevier Science Ltd. All rights reserved.

1. Introduction Visual inspection is routinely used to assign grades or quality labels to food commodities such as grain and meat. Rapid advances in hardware and software for digital image processing motivated several studies on the development of computer vision systems to evaluate quality of diverse raw and processed foods (Gerrard, Gao & Tan, 1996; Locht, Thomsen & Mikkelsen, 1997). In a computerized visual inspection system, a camera acquires a sample image and a computer with specialized software is used to conduct pre-de®ned visual tasks. It is anticipated that computer vision inspection of food products will be consistent, ecient and cost-e€ective. Several studies have been conducted for adapting a computer vision system for beef quality evaluation (Cross, Gilliland, Durland & Seideman 1983; Gerrard, $ This paper is a contribution from the Missouri and Purdue Agricultural Experiment Stations. * Corresponding author. Tel.: +1-573-882-7778; fax: +1-573-8845650. E-mail address: [email protected] (J. Tan).

Gao & Tan, 1996; Unklesbay, Unklesbay & Keller, 1986). Pork research, on the other hand, has lagged behind in terms of studies devoted to design and development of methodology for computer vision-based evaluation of quality. One potential obstacle for development of such technology is that meat color can vary greatly over any given cut. Nevertheless, there is a need for an objective and cost-e€ective method to assess fresh meat quality. Monitoring pork quality on-line is important because such information would allow processors to sort product, which would ultimately lead to less variation in quality at the retail counter. Furthermore, quality-based classi®cation would ensure that various components were used in the most suitable production. To reduce product variation and remain competitive in a quality-sensitive meat market, research toward attaining an objective quality measurement for pork is essential. Among the pork quality attributes, color is of special signi®cance because it is critically appraised by consumers and often is their basis for product selection or rejection (Judge, 1989). Unfortunately, pork color is dicult to assess because the color over a meat cut, even

0309-1740/00/$ - see front matter # 2000 Elsevier Science Ltd. All rights reserved. PII: S0309-1740(00)00020-6

58

J. Lu et al. / Meat Science 56 (2000) 57±60

within the same muscle is frequently not uniform (AMSA, 1991). With digital image analysis of a pork chop, it is possible to segment the chop image into exclusively fat, bones, and muscle images. As a result, muscle color can be evaluated solely over a muscle of interest or all muscles in a cut without the in¯uence of fat and bone colorations. Likewise, fat color can be evaluated without the in¯uence of other tissues even though fat is a less important factor in predicting the ultimate pork quality. Collectively, this information will yield an objective means of evaluating pork quality. This type of information is not readily achievable by other means such as a colorimeter. The objective of this study, therefore, was to test whether a neural network or statistical model using color image features of pork as inputs can predict sensory color scoring.

2.3. Image processing Images were ®rst segmented. Photometric and morphological di€erences among background, muscle, and fat were used to develop discriminant functions to remove background, fat, and bones from the images. Algorithms were implemented in Microsoft C/C++. An example of a segmented muscle image is shown in Fig. 2. A boundary tracking-based labeling algorithm was developed to remove irrelevant structures from the image and restore relevant structures. Details of the image analysis algorithm are given in Lu and Tan (1998).

2. Materials and methods 2.1. Samples and sensory evaluation Forty-four randomly picked pork loins were ribbed at the 10th rib. The loins were of bone-in type. Muscle color of each loin was subjected to evaluation by a seven-member sensory panel. The panelists were trained according to procedures described in the AMSA (1991) guidelines. Pre-trials were conducted to determine the range of colors and discoloration patterns of the samples. Lighting conditions were consistent with established guidelines (AMSA). Panelist viewing angle was kept about 45 relative to the light source. Room temperature was maintained at 4 C. Color was evaluated on a scale from 1 to 5 where the major color categories were: pale-purplish gray as 1, grayish pink as 2, reddish pink as 3, purplish red as 4, and dark, purplish red as 5. A ®nal color score for each loin was obtained by averaging the scores of the sensory panel.

Fig. 1. Original pork loin image.

2.2. Image acquisition Image acquisition was performed immediately after scoring by the sensory panel under similar conditions. Loins were removed from carcasses and fabricated into loin chops (2.54 cm). Loin chops including bones were positioned so that the rib bones were vertical in the loin image (Fig. 1). Lighting for image capture was identical to that used for sensory evaluation. The computer vision system used consisted of a Sony XC-711 CCD color camera, a Sony PVM-1342Q color video monitor, a Data Translation (Marlboro, MA) DT2871 color image frame grabber and a DT2878 advanced processor hosted by a Pentium-120 computer. The color images were 512483 pixels in resolution.

Fig. 2. Segmented muscle image.

J. Lu et al. / Meat Science 56 (2000) 57±60

Segmented loin muscle images were represented in both the RGB (red, green and blue) and the HSI (hue, saturation and intensity) color spaces. The mean and standard deviation were computed for each of the six color components. This resulted in 12 color features: R , G , B , H , S , I,  R,  G,  B,  H,  S, and  I; where  is the mean,  is the standard deviation, and the subscripts indicate the color components. The means represent the average color properties of a muscle and the standard deviations provide a measure of color variations over a muscle. 2.4. Color score prediction Statistical and neural-network models were developed for prediction of sensory color scores from the image color features. A cross-validation scheme was used to train and test the data set. Forty observations were employed as the training data set while the remaining four were used for testing. The procedure was repeated 11 times to test all the samples once. Prediction of characteristics of biological objects is prone to complexities, such as collinearity among the predictive variables, non-linearity of the relationship between the predicted and predictive variables, and large number of predictive variables. The partial least squares or PLS method (Wold et al., 1983) can yield orthogonal latent variables that are related to the response variable. Multiple linear regression (MLR) is an e€ective means to explain or predict the responses with a set of easy-to-measure predictors when the predictors are few in number, are not collinear, and have a well-understood relationship with the responses. However, if any of the three conditions are not met, MLR will become inecient or inappropriate. Special care on the above three conditions was taken in the development of the PLS/MLR model. The partial least squares (PLS) method was developed in the 1960s originally as an econometric technique for constructing predictive models when there are several highly collinear factors. When the variables of interest can not be observed directly, PLS is able to capture such unobservable or latent variables (Dijstra, 1983). The partial least squares method is an e€ective modeling tool to extract information from ill-understood data and can yield orthogonal latent variables and result in a reduction of data dimensionality. PLS method was applied to obtain a set of latent variables. Sensory color scores were predicted with a polynomial of the latent variables obtained from PLS. The ®nal model was selected by using the SAS backward elimination procedure at a signi®cance level of 0.05. We refer to this model as the PLS/MLR model. A multi-layer neural net (NN) with back-propagation learning was also used to predict pork color scores. The

59

NN was con®gured with an input layer, a single hidden layer, and an output layer. The hyper-tangent function, tanh(x)=(exÿ eÿx)/(ex+eÿx), was used for the neurons in the hidden layer. The activation function of the output layer neuron was linear. Weights of the NN were initialized with random numbers. The network was fed with the training data set and the stopping criterion for the training process was when either the training goal was met or the number of the epochs reached 10,000. The training goal was reached when the sum of squared error (SSE) became less than or equal to 0.04. The learning rate used was 0.01. Over- or under-®tting can occur if an NN is over- or under-trained. Preliminary trials demonstrated that both situations were avoided if training was stopped when the MSE for the training set reached a value between 1.4 and 1.7. 3. Results and discussion Sensory scores and those predicted by the NN and PLS/MLR methods are given in Fig. 3. The line at 45 represents the perfect prediction. Neural network prediction was better than PLS/MLR prediction. Correlation coecients between predicted and original sensory color scores were 0.75 and 0.52 for NN and PLS/MLR models, respectively. Absolute di€erences between predicted and original sensory scores were considered prediction errors. Errors were compared with the standard deviations of sensory scores among judges (STDJ) for each sample. For NN predictions, 10 samples gave an error greater than the corresponding STDJ. Of the 10 samples, two gave a di€erence greater than two times

Fig. 3. Predicted pork color scores.

60

J. Lu et al. / Meat Science 56 (2000) 57±60

Fig. 4. Prediction error histograms.

and one three times the STDJ. For PLS/MLR predictions, fourteen samples gave prediction error greater than the corresponding STDJ. Of the 14 prediction errors, three were above two times and one above three times the STDJ. This con®rmed that the prediction errors were within the variation shown by the sensory color measurement. Fig. 4 shows the histogram of prediction errors. In the NN predictions, none of the samples gave an error greater than one and for 93.2% or 41 samples the error was between 0 and 0.59 (Fig. 4). Such small errors may not be serious from a practical viewpoint. One sample in the PLS/MLR prediction gave an error greater than one and six samples resulted in errors between 0.60 and 0.99 (Fig. 4). Overall, the results suggest that digital image analysis in conjunction with neural network or statistical modeling can be used to predict sensory color scores of pork loin though neural network may provide better results. No previous study has been reported on predicting sensory scores of pork color by computer vision. Relationship between sensory scores and image features can be complex and that is why non-linear models are necessary for satisfactory results (Tan, Gao & Gerrard, 1999). Modeling of sensory scoring is important because it is one of the methods in frequent use in the pork industry. Also, it attempts to simulate consumer behavior in selecting the pork meat. 4. Conclusions Computer vision can be used for predicting color scores of pork muscle. The sensory color scores of pork loins were non-linearly related to the color features extracted from the loin images. Both statistical and

neural network models resulted in satisfactory prediction though the neural net model was better.

Acknowledgements This study was supported in part by a grant from the National Pork Producers Council. References AMSA (1991). American Meat Science Association Committee on guidelines for meat color evaluation. Contribution No. 91-545A. Cross, H. R., Gilliland, D. A., Durland, P. R., & Seideman, S. (1983). Beef carcass evaluation by use of a video image analysis system. Journal of Animal Science, 57(4), 908±917. Dijstra, T. (1983). Some comments on maximum likelihood and partial least squares method. Journal of Econometries, 22, 67±90. Gerrard, D. E., Gao, X., & Tan, J. (1996). Determining beef marbling and color score by image processing. Journal of Food Science, 61(1), 145±148. Judge, M. D. (1989). Principles of meat science. (2nd ed.). Dubuque, IA: Kendall/Hunt Publishing Company. Locht, P., Thomsen, K., Mikkelsen, P. (1997). Full color image analysis as a tool for quality control and process development in the food industry. Paper No. 973006. St. Joseph, MI: ASAE Lu, J., & Tan, J. (1998). Application of nonlinear transforms to beef image processing. Paper No. 983016. St. Joseph, MI: ASAE. Tan, J., Gao, X., & Gerrard, D. E. (1999). Application of fuzzy sets and neural networks in sensory analysis. Journal of Sensory Studies, 14(2), 119±138 Unklesbay, K., Unklesbay, N., & Keller, J. (1986). Determination of internal color of beef ribeye steaks using digital image analysis. Food Microstructure, 5, 231±277. Wold, S., Albano, C., Dunn, W. J., Esbensen, K., Hellberg, S., Johansson, E., & SjoÈstroÈm, M. (1983). Pattern recognition: ®nding and using regularities in multivariate data.. In H. Martens, & H. Russwurm (eds), Food research and data analysis (pp. 147±188). London: Applied Science Publishers.