Including 3D-textures in a Computer Vision System to ...

Including 3D-textures in a Computer Vision System to Analyze Quality Traits of Loin 1 ´ M. Mar Avila , Daniel Caballero2(B) , M. Luisa Dur´ an1 , Andrés Caro1 , 2 Trinidad Pérez-Palacios , and Teresa Antequera2 1

Computer Science Department, Polytechnic School, University of Extremadura, Av/Universidad S/n, 10071 Caceres, Spain [email protected] http://gim.unex.es/mmavila 2 Food Technology, Research Institute of Meat and Meat Product (IproCar), University of Extremadura, 10.003, Caceres, Spain [email protected]

Abstract. Texture analysis by co-occurrences on magnetic resonance imaging (MRI) involves a non-invasive nor destructive method for studying the distribution of several texture features inside meat products. Traditional methods are based on 2D image sequences, which limit the distribution of texture to a single plane. That implies a loss of information when texture features are studied from different orientations. In this paper a new 3D algorithm is proposed and included in a computer vision system to study the distribution of textures in 3D images of Iberian loin from different orientations. The semantic interpretation of textural composition in each orientation is also reached. Keywords: Co-occurrence

1

· 3D Textures · Iberian loin · MRI

Introduction

Computer vision systems are a subject of research and application for several industrial processes. These systems have been successfully applied in many engineering fields such as robotics, industrial image processing, food processing and other fields [1]. Quickness, possibilities for non-destructive evaluation, easy procedures for application, and quantum of output per unit time are some advantages that promote the application of computer vision systems to food engineering [2]. During the last years food industry have been among the top five industries in 3D computer vision applications. Researches into the development of appropriate techniques for evaluating 3D objects and scenes are being pursued [3]. The Iberian dry-cured meat products, mainly hams and loins, constitute an important industry in the South-western part of the Iberian Peninsula. These are usually targeted to the dry-cured product market, reaching a high sensory quality and first rate in consumer acceptance. Our research group has been using methods of texture analysis on images of meat products, especially from Iberian pigs [4]. c Springer International Publishing Switzerland 2015 L. Nalpantidis et al. (Eds.): ICVS 2015, LNCS 9163, pp. 456–465, 2015. DOI: 10.1007/978-3-319-20904-3 41

Including 3D-textures in a Computer Vision System

457

Texture analysis methods have been applied to images obtained by means of MRI which allow seeing inside the product, without destroying the sample [5]. Relationships between computational texture features and physico-chemical characteristics such as fat, salt level, and moisture [6] have been established. And the prediction of sensory features such as acceptable color, tenderness on the palate, and acidity has been achieved [7]. MRI provides sequences of two-dimensional images showing transversal slices from the meat product. This enables us to make three-dimensional reconstructions of the bodies. The physical properties of the pieces are distributed inside the product, but not always in a uniform way. This idea leads to search for methods to analyse the distribution of different texture features inside the piece, exploring it as a volume, a three-dimensional space, instead of a sequence of images of slices. The classical Gray-Level Co-Occurrence Matrix (GLCM) algorithm obtains texture features based on co-occurrence of gray levels found in each of the four orientations in the plane ( 0◦ , 45◦ , 90◦ , and 135◦ ), for 2D images. These co-occurrences are accumulated into a matrix, on which the texture features proposed by Haralick are calculated (energy, entropy, correlation, Haralick correlation,inverse difference moment, inertia, cluster prominence, cluster shade, contrast, and dissimilarity) [8]. The same method can be applied in all orientations in a three-dimensional space. This would be the natural evolution from 2D to 3D texture algorithm [9]. However, instead of accumulating all co-occurrences in a matrix, these can be analysed at different levels obtaining several points of view on feature textures inside the product. This is the key of the 3D texture algorithm proposed for our computer vision system. The most recent advances in researches on three-dimensional images are focused on visual systems such as real scenes reconstruction and true textures recreation [10] even for face recognition applications [11]. There are other developed methods in the field of the to analyze internal tissues or tumor [12,13]. Only few attempts have been applied to analyse food products [9,14]. The objective of this work is to develop computer vision system to determine quality characteristics of loin, including a new 3D texture algorithm. The semantic content of the textures features is also aimed to be explained.

2

Material

Magnetic resonance images (MRI) from ten Iberian loins were generated at the Animal Source Foodstuffs Innovation Services (SiPA, C´ aceres, Spain). A MRI scanner (ESAOTE VET-MR E-SCAN XQ 0.18 T) was used with nine different configurations differing in echo time (TE) and repetition time (TR). Sequences of Spin Echo (SE) T1 were applied with a field of view (FOV) of 150 × 150 mm2 , slice thickness 4 mm, i.e., a voxel resolution 0.23 × 0.23 × 4 mm3 . Twenty nine slices per loin piece were obtained. The total number of 2D images is 2610 images (29 images × 10 loins × 9 configurations). Physico-chemical analysis on loins were carried out in order to obtain values for moisture, color coordinates (L, a, b), salt content and lipid content. Those values

458

´ M.M. Avila et al.

Fig. 1. 3D synthetic image

are related to the quality of the loin and were applied to correlate them with texture features calculated on 3D figures. A set of 15 synthetic images (512 × 512 × 5) were used to evaluate the feasibility of the proposal. Figure 1 shows one of these images.

3

Methods

Figure 2 illustrates our computer vision system. This system obtains quality parameters of loins based on MRI and texture algorithms. Several texture algorithms have been tested to evaluate the performance of the system: two-dimensional texture algorithm (classical GLCM method) proposed by Haralick [8], threedimensional texture algorithm (a readapted version of the 2D approach) [9], and our own 3D texture algorithm (called 3DTextFED).

Fig. 2. Applied methodology

Figure 3 shows the flow chart explaining 3DTextFED, organized in three steps. First, in step (a) a 3D figure is constructed applying a linear interpolation function between each pair of slices. The linear interpolation function between two points a and b is calculated as: f (p) = (1 − x) ∗ f (a) + x ∗ f (b)


459

where f (a) and f (b) are the values of the function for the a and b points, x is the distance between one of the points and p, the point where the value of f is interpolated. The image acquisition process obtains sets of 2D images representing 3D spaces, by means of MRI. High resolution images are usually obtained (pixel resolution 0.23 × 0.23 in our case) nevertheless, the distance between consecutive images is not small enough (4 mm slice thickness in our case). However, it is better to have reasonable dimensions when trying to obtain textural information from volumetric structures. Otherwise, the voxel sizes would certainly be inconsistent in the Z dimension (0.23 × 0.23 × 4 mm3 ). In 3DTextFED, this is achieved by interpolating four new images between each pair of consecutive MRI images, so that the voxel size becomes 0.23 × 0.23 × 0.8 mm3 . In step (b) textures are computed based on co-occurrences for each of the thirteen orientations. The classical GLCM algorithm [8] obtains texture features based on co-occurrence of gray levels found in each of the four orientations in the plane, for 2D images. These co-occurrences are accumulated into a single array, from which all the textural features are extracted. That implies a lack in the characterization of the objects from which textures are extracted, since an object can presents characteristics of high roughness in a plane (for example, in the XY plane), and characteristics of high uniformity in the perpendicular plane (for example, the YZ plane). If the calculations of all possible planes are accumulated in a single array from which the texture features are extracted, all this information would be lost. Classical 3D GLCM algorithms also compute all the co-ocurrences in the same matrix. Our 3D texture algorithm (3DTextFED) solves this problem. It generates a independent co-occurrence matrix for each one of the thirteen orientations: 0◦ –180◦ , 90◦ –270◦ , 135◦ –315◦ , 45◦ –225◦ in the XY plane, 0◦ –180◦ , 135◦ –315◦ , 45◦ –225◦ in YZ plane, 135◦ –315◦ , 45◦ –225◦ in the XZ plane and 135◦ , 315◦ , 45◦ , 225◦ in the XYZ plane (Fig. 3b). Thus, instead of accumulating all the cooccurrences of the thirteen orientations in just one array, a separate matrix for each of these orientations are obtained. This allows studying independently the behavior of the textures in each orientation. Finally in step (c) seven texture features are computed in each of the thirteen matrices: energy, entropy, inverse difference moment (IDM), Haralick’s correlation (HC), inertia, cluster shade (CS), and cluster prominence (CP), as Fig. 3(c) shows. Thus, considering the thirteen orientations and the seven textural features in each of the orientations, a total amount of 91 features are extracted for each of the Iberian loins (7 × 13). Semantic content such as coarseness, homogeneity, symmetry, contrast are described based on the obtained values for calculated texture features. Therefore, different behaviors can be observed in different planes and orientations. For example, texture features could present a very rugged behavior in some of the planes, whereas in others the aspect could be completely uniform. This is illustrated in Figs. 4 and 5, showing different views from the synthetic image shown in Fig. 1.

460


Fig. 3. The proposed 3D texture algorithm (3DTextFED).


461

Figure 4 shows an image from the viewpoint of the XY plane. The image is filled with vertical stripes. However, from the point of view of the YZ plane (Fig. 5) the image is formed by completely homogeneous planes, which clarifying the color according to the advance in the X coordinate. If all information will be stored in a single array, all that valuable knowledge would be lost.

Fig. 4. XY plane Fig. 5. YZ plane of a 3D synthetic image of a 3D synthetic image

4

Results and Discussion

Table 1 shows the values of the texture features after applying the proposed methodology on the 3D synthetic image (Fig. 1). The values of each feature are equal in the YZ planes, because they are completely homogenous planes. However, these values change in the other planes. Thus, it can be confirmed that this method can analyze 3D images from different angles, appreciating the different textures that they may present. Table 1. Texture features obtained from a 3D synthetic image in each one of thirteen orientations Energy Entropy HC ◦

◦

IDM

Inertia CS

CP

0 –180 xy

0.1586

0.7475

0.9513 0.0368 1.0000 1.0000 0.9515

90◦ –270◦ xy

1.0000

0.7475

1.0000 1.0000 0.0000 0.9141 1.0000

135 –315 xy 0.1586

1.0000

0.9513 0.0368 1.0000 1.0000 0.9515

45◦ –225◦ xy

0.1586

1.0000

0.9513 0.0368 1.0000 1.0000 0.9515

0◦ –180◦ yz

1.0000

0.7475

1.0000 1.0000 0.0000 0.9141 1.0000

135 –315 yz 1.0000

0.7475

1.0000 1.0000 0.0000 0.9141 1.0000

45◦ –225◦ yz

1.0000

0.7475

1.0000 1.0000 0.0000 0.9141 1.0000

135 –315 xz 0.1586

1.0000

0.9513 0.0368 1.0000 1.0000 0.9515

45◦ –225◦ xz

0.1586

1.0000

0.9513 0.0368 1.0000 1.0000 0.9515

135 xyz

0.1586

1.0000

0.9513 0.0368 1.0000 1.0000 0.9515

315◦ xyz

0.1586

1.0000

0.9513 0.0368 1.0000 1.0000 0.9515

135 xyz

0.1586

1.0000

0.9513 0.0368 1.0000 1.0000 0.9515

135◦ xyz

0.1586

1.0000

0.9513 0.0368 1.0000 1.0000 0.9515

◦

◦

◦

◦

◦

◦

◦

◦

462


Table 2. Texture features obtained from a fresh Iberian loin in each one of thirteen orientations Energy Entropy HC ◦

◦

IDM

Inertia CS

CP

0 –180 xy

0.8439

0.0220

0.7696 0.9943 0.0003 0.2672 0.0001

90◦ –270◦ xy

0.6762

0.0426

0.7694 0.9980 0.0002 0.2943 0.0001

135 –315 xy 0.6205

0.0673

0.7701 1.0000 0.0010 0.2917 0.0001

45◦ –225◦ xy

0.8904

0.0191

0.7703 0.9858 0.0000 0.2997 0.0001

0.6358

0.0570

0.7696 0.9294 0.0004 0.2668 0.0001

135◦ –315◦ yz 0.7531

0.0380

0.7705 0.9534 0.0007 0.2717 0.0001

0.7720

0.0327

0.7703 0.9833 0.0001 0.2642 0.0001

135◦ –315◦ xz 0.6543

0.0372

0.7701 0.9718 0.0000 0.2946 0.0001

◦

◦

◦

◦

0 –180 yz ◦

◦

45 –225 yz ◦

◦

45 –225 xz

0.6684

0.0422

0.7702 0.9578 0.0002 0.2944 0.0001

135◦ xyz

0.6137

0.0635

0.7707 0.9991 0.0006 0.2860 0.0001

315◦ xyz

0.6220

0.0616

0.7710 0.9664 0.0008 0.2932 0.0001

135 xyz

0.8222

0.0281

0.7711 0.9528 0.0002 0.3046 0.0001

135◦ xyz

0.8817

0.0207

0.7709 0.9785 0.0001 0.2974 0.0001

◦

Table 3. Texture features obtained from a dry-cured Iberian loin in each one of thirteen orientations Energy Entropy HC

IDM

Inertia CS

CP

0◦ –180◦ xy

0.0280

0.7404

0.7159 0.2140 0.1237 0.4531 0.2516

90◦ –270◦ xy

0.0042

0.9050

0.7077 0.0180 0.4608 0.4804 0.2500

135◦ –315◦ xy 0.0079

0.8651

0.7119 0.0687 0.2768 0.4824 0.2599

◦

◦

45 -225 xy

0.0069

0.8811

0.7031 0.0286 0.4236 0.4783 0.2390

0◦ -180◦ yz

0.0112

0.8064

0.7147 0.1155 0.1527 0.4528 0.2605

135 –315 yz 0.0161

0.8054

0.7142 0.1239 0.1802 0.4595 0.2661

45◦ –225◦ yz

0.0171

0.7834

0.7140 0.1242 0.1519 0.4476 0.2536

135◦ –315◦ xz 0.0056

0.8834

0.7061 0.0272 0.3596 0.4798 0.2670

◦

◦

◦

◦

45 –225 xz

0.0057

0.8803

0.7066 0.0333 0.3808 0.4802 0.2617

135◦ xyz

0.0083

0.8607

0.7100 0.0556 0.2726 0.4733 0.2691

315◦ xyz

0.0086

0.8524

0.7107 0.0770 0.2513 0.4853 0.2686

135◦ xyz

0.0073

0.8809

0.7018 0.0314 0.4011 0.4865 0.2482

0.0078

0.8712

0.7022 0.0349 0.3583 0.4765 0.2465

◦

135 xyz

The set of synthetic images were used to prove the feasibility of the algorithm and also to determine the semantic content of the texture features. The meaning of the texture features is a key aspect to understand them. Applying this methodology to each 3D Iberian loin piece, texture features in thirteen orientations are obtained. Below, two examples are shown, one of them referent to fresh loin (Table 2) and the other one referent to dry-cured loin (Table 3).


463

Table 4. Correlation coefficients R between physico-chemical parameters and texture features from GLCM. 2D Moisture

3D

3DTextFED

0.948

0.931 0.956

Water activity 0.950

0.956 0.956

Lipid

0.791 0.638 0.773

Salt

0.949

0.946 0.963

Color L

0.902

0.906 0.913

Color a

0.851 0.820 0.819

Color b

0.786 0.733 0.774

The homogeneity of the fresh Iberian loin can explain the low variation into the values in Table 2. Slight variations can be observed when obtained values for the fresh and cured loin are compared, because of the types of meat (fresh and cured meat, respectively). Analyzing the meaning of the values of the texture features (Table 2), it can be determined that the fresh loin is uniform (high energy values, close to (1). Therefore, it does not have a messy texture (low entropy), being very homogeneous (very high IDM), with a quite low contrast, i.e. there is not large clusters of pixels with the same or similar gray level (low inertia, as fresh meat is very uniform), and it is not symmetrical in its structure (very low CS), nor gray levels (very low CP). Regarding the values of features in the cured loin (Table 3), it can be seen that the energy decreases, so the cured loin is more rough than the fresh loin. And therefore, the entropy is high, presenting quite low IDM values, since it is not as homogeneous. The contrast increases and also the degree of symmetry, because the meat is cured. Texture features are used as explanatory variables in a system of equations applying multiple linear regression (MLR). The free software WEKA (Waikato Environment for Knowledge Analysis) (http://www.cs.waikato.ac.nz/ml/weka/) was used for carrying out multiple linear regression [15]. The obtained equations provide values which have been correlated with real data obtained by physicochemical analysis. Table 4 shows these correlations. Correlation of over 0.75 has been achieved in almost all cases, being in some of them of over 0.9. Better correlations were obtained for 2D GLCM texture algorithm when compared to 3D GLCM texture algorithm. Five of the seven quality parameters (moisture, lipid, salt, color a, and color b) obtained better correlation for the 2D than for the 3D approach. That could imply that 3D textures, despite providing extra information, are not a satisfactory solution. However, our 3D algorithm provides the ability of considering multiple points of view (different angles), by computing the co-occurrences in different matrix. This allows better correlation ratios. The 3DTextFED algorithm obtained the best correlations for four of the seven quality parameters, and then was chosen as texture algorithm for the computer vision system.

464


Nevertheless, there is a high correlation between the parameters obtained by traditional destructive techniques and data obtained by means of textures analysis based on volumetric information. Quality parameters can be obtained in a non destructive nor invasive way, by using 3D textures. This is an important development for the meat industries, since it determines that it is not necessary to destroy any pieces to obtain parameters related to the quality of them.

5

Conclusion

A computer vision system to obtain quality parameters of loin based on MRI and texture algorithms has been tested and validated. The 3DTextFED algorithm is suitable for studying volumetric texture features from MRI loin, and the texture distribution has been analyzed in different orientations. Texture features in 3DTextFED reach high correlations with physico-chemical parameters of Iberian loin. Analyzing volumetric textures in different planes is a better option than applying simple adaptations of the classical 2D texture methods. The semantic content for the texture features of loins has been explained. Acknowledgments. The authors wish to acknowledge the funding received for this research from Ministerio de Ciencia e Innovacion and FEDER-MICCIN-Infrastructure Research Project (UNEX10-1E-402), Gobierno de Extremadura - Consejeria de Empleo, Empresa e Innovacion and funds by FEDER (European Regional Development Funds). We also wish to thank Animal Source Foodstuffs Innovation Services (SiPA) from Faculty of Veterinary of University of Extremadura.

References 1. Mahendran, R., Jayashree, G.C., Alagusundaram, K.: Application of Computer Vision Technique on Sorting and Grading of Fruits and Vegetables, J. Food Process. Technol. S1–001 (2012) 2. Brosnan, T., Sun, D.-W.: Improving quality inspection of food products by computer vision - a review. J. Food Eng. 61, 3–16 (2004) 3. Gunasekaran, S.: Computer vision technology for food quality assurance. Trends Food Sci. Technol. 7, 245–256 (1996) 4. Cernadas, E., Dur´ an, M.L., Antequera, T.: Recognizing marbling in dry-cured iberian ham by multiscale analysis. Pattern Recogn. Lett. 23, 1311–1321 (2002) 5. Caro, A., Dur´ an, M., Rodr´ıguez, P.G., Antequera, T., Palacios, R.: Mathematical morphology on mri for the determination of iberian ham fat content. In: Sanfeliu, A., Ruiz-Shulcloper, J. (eds.) CIARP 2003. LNCS, vol. 2905, pp. 359–366. Springer, Heidelberg (2003) ´ 6. Petr´ on, M.J., Dur´ an, M.L., Avila, M., Cernadas, E., Antequera, T.: A computer vision system to discriminate iberian pigs from ham images. Electron. J. Environ. Agric. Food Chem. 2(5), 549–557 (2003) 7. Antequera, T., Muriel, E., Rodr´ıguez, P.G., Cernadas, E., Ruiz, J.: Magnetic resonance imaging as a predictive tool for sensory characteristics and intramuscular fat content of dry-cured loin. J. Sci. Food Agric. 83, 268–274 (2003)


465

8. Haralick, R.M., Shapiro, L.G.: Computer and Robot Vision. Addison-Wesley, Reading (1993) ´ 9. Avila, M.M., Dur´ an, M.L., Antequera, T., Palacios, R., Luquero, M.: 3D reconstruction on MRI to analyse marbling and fat level in iberian loin. In: Mart´ı, J., Bened´ı, J.M., Mendon¸ca, A.M., Serrat, J. (eds.) IbPRIA 2007. LNCS, vol. 4477, pp. 145–152. Springer, Heidelberg (2007) 10. Lopez, A., Pla, F., Ribelles, J.: 3D modeling of structured scenes through binocular stereo vision. In: Scandinavian Conference on Image Analysis (2001) 11. Traver, V.J., Latorre-Carmona, P., Salvador-Balaguer, E., Pla, F., Javidi, B.: Human gesture recognition using three-dimensional integral imaging. J. Opt. Soc. Am. Opt. Image. Sci. Vis. 31(10), 2312–2320 (2014) 12. Mahmoud-Ghoneim, D., Toussaint, G., Constans, J.M., de Certaines, J.D.: Three dimensional texture analysis in MRI: a preliminary evaluation in gliomas. Magn. Resonnance Imaging 21(9), 983–987 (2003) 13. El-Baz, A., Casanova, M.F., Gimel’farb, G., Mott, M., Switala, A.E.: Autism diagnostics by 3D texture analysis of cerebral white matter gyrifications. Med. Image Comput. Assist. Interv. 10, 882–890 (2007) 14. Brosnan, T., Sun, D.-W.: Improving quality inspection of food products by computer vision, a review. Comput. Electron. Agric. 36, 193–213 (2002) 15. Perez-Palacios, T., Caballero, D., Caro, A., Rodrguez, P.G., Antequera, T.: Applying data mining and computer vision techniques to MRI to estimate quality traits in Iberian hams. J. Food Eng. 131, 82–88 (2014)