Genetic Programming for Prediction of Fat Content in Meat Images Lucia Ballerini1 and Leonardo Bocchi2 1
2
Dept. of Food Science, Swedish University of Agricultural Sciences P.O. Box 7051, 75007, Uppsala, Sweden
[email protected] Dept. of Electronics and Telecommunications, University of Florence Via S.Marta 3, 50139 Firenze, Italy
[email protected]
Abstract. In this paper we investigate the feasibility of genetic programming for combining features extracted from meat images to predict fat content. Images of beef meat were acquired and analyzed. Feature sets including fat percentage, fat fleck size distribution and homogeneity measurements were extracted from these images. Using genetic programming we built individual programs which give a prediction of fat content based on these features. Results shows that the proposed method is effective in achieving a very good approximation of the fat percentage measured by chemical analysis.
1
Introduction
Intramuscular fat content in meat influences some important meat quality parameters. For example, the quantitative intramuscular fat content has been shown to influence the palatability characteristics of meat. In addition, the visual appearance of the fat does influence the consumers overall acceptability of meat and therefore the choice when selecting meat before buying. There are several methods to analyze quantitative intramuscular fat content and its visual appearance in meat. However, few of them are satisfactory enough in terms of fat quantification in a cross section of a consumer size meat slice, without using large amounts of organic solvents, being too time consuming, or invasive. Computer video systems are being used increasingly in the food industry for quality assurance purposes. These systems offer the potential to automate manual grading practices thus standardizing techniques and eliminating tedious human inspection tasks. Review papers that describe the progress of computer vision in the agricultural and food industry attest the growing interest in these methods. [1] Image processing and image analysis are recognized as being the core of computer vision. [2] As concern the meat industry, visual inspection is used extensively for the quality assessment of meat products applied to processes from the initial grading through to consumer purchases. [3] Applications to beef, lamb and pig meat can be found in literature [4]. Gerrard et al. [5] have developed an image processing
system to evaluate beef quality, in particular to determine beef marbling and color scores. In the work of Li et al. [6] color, marbling, and image texture features were used to develop tenderness prediction models by using statistical and neural network techniques. Texture features have also been used by Shiranita et al. [7, 8] They studied a method to detect the meat quality using the concept of “marbling score” and texture analysis. They proposed the implementation of a meat grading system using neural network techniques and multiple regression analysis. Basset et al. [9] use texture analysis to classify photographic images of meat slices according three biological factors: muscle type, age and breed. Yoshikawa et al. [10] investigated an automatic grading system for beef marbling which is based on discriminant threshold selection method and run length processing. Other image modalities, like computer tomography (CT) or magnetic resonance (MR), widely applied in medical sciences, has been recently used also in food science [11, 12]. The aim of the present study was to develop and apply image processing techniques to quantify intramuscular fat content in beef. The reference method, which is commonly used to assess the percentage of fat, is based on chemical analysis, and this makes it quite expensive and time consuming. Therefore, we evaluated the correlation of this measure with the results obtained by chemical visual and sensory analysis and assumed that this correlation can be used to evaluate the reliability of the estimation obtained by means of the image processing system, The study required to analyze a set of meat samples both with the traditional method and with the proposed system, and compare results. In the following we will focus only on the image processing task, using results from the chemical analysis only to tune the system and to evaluate the results. The processing starts from the acquisition phase, where color images of meat samples were captured. Image analysis software was specifically developed for the interpretation of these images. In particular, a segmentation algorithm is used for classification of different substances: fat, muscle and connective tissue. Methods to measure size and spatial distribution of fat flecks are proposed and used to extract features. These features are combined by genetic programming [13] in order to predict fat content.
2
Image Processing
Beef meat image were acquired and processed to collect data used by the genetic programming. To segment fat regions from the acquired images three stages of image processing were developed, i.e., meat extraction, connective tissue removal, fat segmentation. After segmentation measurements has been performed and features extracted. A block diagram of the procedure used for image processing is described in Figure 1.
Meat Extraction
Connective Tissue Removal
Fat Segmentation
Fat Percentage
Fleck Size Distribution
Fat Homogeneity
Fig. 1. Schematic description of the overall image analysis procedure
2.1
Image Acquisition
Color images of beef meat were captured by a Sony DCS-D700 camera. The same exposure and focal distance were used for all images. Digital color photography was carried out with a Hama repro equipment (Germany). Green color was used as background and photographs were taken on different part of the muscle, as shown in Figure 2. Chemical analysis has been performed on the same piece of meat, as shown in the Figure. Three parts were photographed: both sides (N2 and N3) of the meat slice that undergoes also chemical analysis and an external side (N1). The meat pieces were enlighten with two lamps, with two fluorescent tubes each (15 W). Polaroid filters were used on the lamps and on the camera. Images were 1344 × 1024 pixel matrices with a resolution of 0.13 × 0.13 mm (see Figure 3, as an example). N3 N2
N1
chemical analysis
Fig. 2. Schematic illustration of loin and relative slice used for chemical analysis and image analysis (N1, N2, N3)
2.2
Image Segmentation
The choice of green color as background was very helpful for the first stage of the segmentation (meat extraction). This step was performed through image
Fig. 3. Digital camera image of a representative beef meat slice
thresholding based on histogram analysis techniques [14]. Thus, meat extraction was performed by detecting the main valley between the two peaks and using the corresponding value as threshold. All pixels below the threshold were assigned as the region of meat and all pixels above the threshold were set as background. To remove noises and gaps, morphological operators were applied to the binary image obtained by the threshold-based segmentation method. A dilation operation was implemented to fill the gaps within the region of meat. Then an erosion operation was applied to remove noises around the border area. This operation was used also to remove connective tissue areas, that are not of interest for our studies. Considering that connective tissue is very often near the border of the meat area, this operation, beside its simplicity, has been powerful to remove connective tissue from most of our images. Other studies have been carried out to remove connective tissue. In particular we presented an algorithm that uses active contours [15], with a modified version of their energy in order to consider the area enclosed in contour. This algorithm works quite well, but unfortunately the energy minimization by genetic algorithms is slow. A second segmentation algorithm has been used to segment fat in the obtained meat image. We experimented several methods, that provided accurate results. They are described in details in previous works [16, 17] and will not reported here. Figure 4 shows the results after fat segmentation: the fat contour is outlined in blue on the original image. 2.3
Feature Extraction
The percentage η of fat in the meat area was computed as: R ff at (x, y)dA × 100 η= Ω R dA Ω
(1)
Fig. 4. Fat segmentation from meat image shown in Figure 3
where Ω is the meat area and ff at (x, y) is the binary segmentation obtained as follows: 1 if f (x, y) ∈ “fat ” ff at (x, y) = (2) 0 otherwise. To measure fat fleck size, each fleck is uniquely labeled (identified) by extracting the connected components in the image. Connected components are used frequently for automatic inspection in food [18]. A minimum component size has been set in order to avoid getting excessive numbers of very small regions, that are probably noise and not fat flecks. The size distribution histogram is shown in Figure 5. 200
number of fat spots
150
100
50
0
0-40
41-80
81-120 fat spot size
121-160
161-200
Fig. 5. Size distribution of fat regions on meat piece shown in Figure 3
We also propose a new method to describe and quantify the distribution of fat at different resolutions [19]. This method is based on scale-based region
homogeneity [20]. We considered a pyramid image representation, where the image was divided into four quadrants at each hierarchical level (see Figure 6). The homogeneity of fat distribution could easily be related to the levels l of the pyramid representation, where: l = 0, ..., L, where 2L is the smaller of the two image dimensions.
Level 0
Level 1
Level 2
Fig. 6. Pyramid image representation
Fat percentage η was computed for each quadrant, according Equation (1). Mean and standard deviation of fat percentage was computed for each level. Low values of standard deviation at low levels mean uniform distribution, while high values mean non-uniform distribution. We could say that the slope is an indicator of homogeneity. These measurements are computed for all the beef meat images, and correlations with chemical analysis are studied. The correlation obtained between fat percentage measured by image analysis on side N2 and chemical analysis is 0.83, between side N3 and chemical analysis is 0.82. Not surprising, the best correlation (0.88) is obtained by averaging fat percentage obtained by image analysis on both sides of the meat slice used for chemical analysis. The regression plot of the these measurements is shown in Figure 7.
3
Genetic Programming
A visual analysis of the features described in the previous section indicate that they are somehow related to the fat content evaluated by chemical methods. However, the exact relationship between features and fat content cannot be easily assessed. Therefore, the proposed method uses a genetic program to evolve toward a satisfactory relation among the feature set and the fat content. Once the evolution has completed, the resulting relationship can be used to provide an estimation of the fat content on meat samples not belonging to the original dataset.
10
FAT % (two side average)
8
6
4
2
0
0
2
4 6 FAT % (chemical analysis)
8
10
Fig. 7. Comparison of fat percentage measured by chemical analysis and by image analysis.
3.1
Simulation Kernel
The genetic programing application has been implemented by means of an existing kernel, which is publicly available, called lil-gp [21]. This program is capable of evolving function trees, which may contain any set of user-defined function nodes and terminals. In order to work with the simulation kernel it is necessary to define, using C language, the set of function nodes, the set of terminals, and the method to evaluate the fitness of the individual. The kernel itself takes care of the simulation aspect, from initialization of the population, individual selection, offspring generation and so on. 3.2
Experimental Setup
To better predict the fat content the features described in the previous section are used as terminals of a genetic program. The following procedure is adopted. The fitness is based on the error between the measured and the predicted fat content. The error has been evaluated as the total squared error between the prediction and the measured data, evaluated over the training samples. The best fitness, therefore, corresponds to a null error, which means a perfect estimate over the whole dataset. The fitness of the best individual, therefore, give a direct indication of the estimation error. However, the total error does not give an indication of the system to predict the differences of fat content among different samples. Therefore, we analyzed also the correlation existing between the real values and the estimated ones. The terminal set is composed of the six features (three from each side) extracted from the meat images, namely, fat percentage, number of fat flecks, fat homogeneity measure. An ephemeral random constant is also used as defined
in lil-gp 1.0 User’s Manual [21]. The allowed function nodes include the four arithmetic operations (add, subtract, multiply, and protected divide). We performed 30 independent runs using a population of 5000 individuals and 200 generations and the following parameter settings: Table 1. Parameter settings function set selection initialization method initialization depth maximum depth reproduction rate crossover rate mutation rate
ADD, SUB, MUL, DIV fitness half-and-half 1-6 10 10% 90% 10%
We noted that in one of the run the genetic programming gave the best-of-run individual after 23 generation: (* (+ (* X0 0.81117) (* X3 0.75751)) 0.50500) This is in agreement with our previous results, in fact X0 and X3 are terminals containing the values of fat percentages measured on the two sides of the meat slice. This results was obtained in so few generation by limiting the depth of the tree size to 6. The correlation between the fat percentage values estimated by GP and the measured ones was comparable with the one obtained by simply averaging the two fat percentages. Better results were obtained by letting the GP creating more elaborated trees that use also other features, i.e. fleck size and homogeneity measurements. In these cases, in most of the runs the solution was found almost when reaching the maximum number of the generations. The best correlation value we obtained is 0.92. The regression plot of the this prediction is shown in Figure 8. This confirms our hypothesis that inclusion of more features improves the overall estimation of fat percentage.
4
Conclusions and future work
In this paper we described an image analysis procedure combined with genetic programming for prediction of fat in beef meat images. A segmentation algorithm, together with methods for measuring fat size distribution and homogeneity have been presented. A genetic programming approach to combine these feature is evaluated. The very high correlation obtained confirm the validity of the method. The high correlations between the two methods for determination of fat percentage
10
FAT % (estimated by GP)
8
6
4
2
0
0
2
4 6 FAT % (chemical analysis)
8
10
Fig. 8. Comparison of fat percentage measured by chemical analysis and predicted by genetic programming.
confirm that image analysis is a good tool in this type of investigation. Moreover image analysis allows a more accurate quantification of the visual appearance of fat in meat, by measuring other features as number and size distribution of fat flecks that are impossible to measure by chemical analysis. The new method proposed in this paper for fat homogeneity quantification can lead to a even more accurate description of meat quality and its expected palatability. The combination of all these features by genetic programming lead to a more accurate prediction of fat percentage. Future work could be the inclusion of more features, like texture features and fractal dimension. Another possibility could be the use of features extracted from the external side image (N1) to predict fat content in the internal part of the loin. Sensory and visual scoring analysis are planned on the same pieces of meat. We believe that good correlation will be obtained also between image analysis and the results of these methods. Acknowledgment We would like to thank Maria Lundesj¨o Ahnstr¨om and Prof. Kerstin Lundstr¨om, Department of Food Science, Swedish University of Agricultural Sciences, Uppsala, Sweden for providing the pictures and for general discussions.
References 1. Brosnan, T., Sun, D.W.: Inspection and grading of agricultural and food products by computer vision systems- a review. Computers and Electronics in Agriculture 36 (2002) 193–213
2. Brosnan, T., Sun, D.W.: Improving quality inspection of food products by computer vision - a review. Journal of Food Engineering 61 (2004) 3–16 3. Ballerini, L., H¨ ogberg, A.: How do people choose meat? In: Proc. Int. Conf. on Visualization, Imaging, and Image Processing, Marbella, Spain (2001) 501–505 4. Chandraratne, M.R., Kulasiri, D., Frampton, C., S., S., Bickerstaffe, R.: Prediction of lamb carcass grades using features extracted from lamp chop images. Journal of Food Engineering 74 (2006) 116–124 5. Gerrard, D.E., Gao, X., Tan, J.: Beef marbling and color score determination by image processing. Journal of Food Science 61 (1996) 145–148 6. Li, J., Tan, J., Martz, F.A., Heymann, H.: Image texture features as indicator of beef tenderness. Meat Science 53 (1999) 17–22 7. Shiranita, K., Miyajima, T., Takiyama, R.: Determination of meat quality by texture analysis. Pattern Recognition Letters 19 (1998) 1319–1324 8. Shiranita, K., Hayashi, K., Otsubo, A., Miyajima, T., Takiyama, R.: Grading meat quality by image processing. Pattern Recognition 33 (2000) 97–104 9. Basset, O., Buquet, B., Abouelkaram, S., Delachartre, P., Culioli, J.: Application of texture image analysis for the classification of bovine meat. Food Chemistry 69 (2000) 437–445 10. Yoshikawa, F., Toraichi, K., Wada, K., Ostu, N., Nakai, H., Mitsumoto, M., Katagishi, K.: On a grading system for beef marbling. Pattern Recognition Letters 21 (2000) 1037–1050 11. Cernadas, E., Carrion, P., Rodrigues, P.G., Muriel, E., T., A.: Analysing magnetic resonance images of iberian pork loin to predict its sesorial characteristics. Computer Vision and Image Understanding 98 (2005) 345–361 12. Vestergaard, C., Erbour, S.G., Thauland, T., Adler-Nissen, J., Berg, P.: Salt distribution in dry-cured ham measured by computed tomography and image analysis. Meat Science 69 (2005) 9–15 13. Banzhaf, W., Nordin, P., Keller, R.E., Francone, F.D.: Genetic Programming: An Introduction. Morgan Kaufmann (1998) 14. Ballard, D., Brown, C.: Computer Vision. Prentice–Hall, Englewood Cliffs, NJ (1982) 15. Ballerini, L.: Genetic snakes for color images segmentation. In: Application of Evolutionary Computation. Volume 2037 of Lectures Notes in Computer Science., Milan, Italy (2001) 268–277 16. Ballerini, L., H¨ ogberg, A., Lundstr¨ om, K., Borgefors, G.: Colour image analysis technique for measuring of fat in meat: An application for the meat industry. In Hunt, ed.: Machine Vision Applications in Industrial Inspection IX. Volume 4301 of Proc. SPIE, Electronic Imaging., San Jose’ (CA) (2001) 113–124 17. Bocchi, L., Ballerini, L.: Image space colonization algorithm. In: Application of Evolutionary Computation. Number 3907 in Lectures Notes in Computer Science, Budapest,Hungary (2006) 356–367 18. Gonzalez, R., Woods, R.: Digital image processing. 2nd edn. Prentice hall, Upper Saddle River, New Jersey (2002) 19. Ballerini, L.: A simple method to measure homogeneity of fat distribution in meat. In: Proc. 12th Scandinavian Conference on Image Analysis, Bergen, Norway (2001) 343–349 20. Saha, P.K., Udupa, J.K.: A new optimum thresholding method using region homogeneity and class uncertainty. In Loew, ed.: Medical Imaging 2000: Image Processing. Volume 3979 of Proc. SPIE., San Diego (CA) (2000) 21. Zongker, D., Punch, B.: lil-gp 1.0 User’s Manual. Michigan State University. (1995)
