7RDQ$XWRPDWLF9LVXDO,QVSHFWLRQRI(JJ6KHOO'HIHFWV M.C.Garcia-Alegre, J.Enciso, A.Ribeiro, D. Guinea Instituto de Automática Industrial Consejo Superior de Investigaciones Científicas 28500. Arganda. Madrid. Spain. e-mail:
[email protected] $EVWUDFW This paper addresses the automation of the visual inspection process for eggs grading purposes at the farm packers. Some preliminary tests have been performed looking for processing algorithms that hold for a trade-off between the accuracy and speed restrictions. First experimental work is conducted to an image analysis based on the enhancement of the discontinuities with respect to the image background before any quantification is accomplished. Finally sensor integration is suggested as a way to improve the reliability and to enlarge the repertoire of defects suitable to be detected. The final goal of the research aims at the development of an on line system for eggs-shell defects detection able to adapt to quality control and to market requirements. .H\ :RUGV 9LVLRQ V\VWHPV DJURIRRG SURFHVVLQJ IHDWXUH H[WUDFWLRQ DXWRPDWHG PDFKLQHU\
,QWURGXFWLRQ The possibility of automatic separation of defective eggs from qualified ones is a fundamental issue at the poultry industry for both economical and health reasons. The main defects that are actually looked at are: blood spots, dirt stains and cracks. Other defects include irregular shells, meat spots, watery whites, low level color and defects of homogeneity in brown eggs and large air cells. Human visual inspection is actually performed at a rate up to 12 eggs/s for high speed inspection at the graders/packers factories. Thus, any automation on the inspection process would lead to a great reduction on the graders visual stress as well as to an improvement on the quality control process. Some research has been performed in eggs [1] and wood [2] based on image color planes analysis when the pursued defect had a unique dominant color. Thus, to differentiate blood spots, stain and cracks some studies have been undertaken by training neural nets with color histograms [3] with a good performance on one defect eggs but unable to accurately work with eggs showing more than one defects. Last point being irrelevant at the USA poultry industry as most leakers are removed prior to eggs-grading by means of a washing process that is not allowed at the E.U. Thus as the frequency of isolated defects ranged from 1% for blood spots and for dirt stain to 5% for cracks, the probability of an egg with more than one defect was very small. This situation obviously change at the European poultry market, were all kind of defects appear in multiple combinations, including the white egg leakages. Eggs with cracks and leakers, are a major issue as they degrade all the mechanical parts while progressing on the graders/packers machinery. This is way the automation of the visual inspection is suggested, not at the large packers [4], but at the small ones which are present in most poultry farms, to deviate highly defective eggs prior to its packing and shipment to the scarce and high-cost graders/packers which deal with poultry distribution over large regions. A conventional visual system is proposed to account for the detection of all defects related to changes on the eggs-shell standard geometry, texture and color, as are: irregular feather stuck on the shells, meat and blood spots, and dirt stains, leakers. The algorithmic process is devoted to the detection and quantification of discontinuities from an homogeneous background pattern on eggshells color images.
,PDJHVDFTXLVLWLRQDQGFRORUKLVWRJUDPV First experimental work has been performed with a conventional Video Camera/Recorder HitachiVM-H80E (8mm) having a resolution of 470.000 pixels, to obtain images of single stationary eggs. Video signal has been digitized with a FPS-60 board at a rate of 15fps, and 384x288 pixels resolution. Finally individual images have been selected and resized to 160x120 pixels, Figure 1, by using the Adobe Premiere LE Video Editor package. The off-line image analysis has been accomplished on a sample of sixty four eggs’s images, forty of them corresponding to defected eggs, on a PC Pentium/200Mhz. using an image processing environment [5].
Figure 1. Three image color planes of a blood spots egg with their histograms
A color segmentation has been faced by means of a histograms based representation of the three image color planes (red, green, blue) that displays the frequency distribution of the image intensity for the more representative sample' s images. Due to the ellipsoidal egg shape a high amplitude peak appears at zero brightness that account for the black image cones and which has to be subtracted to increase the resolution in the relevant amplitudes range. In spite of all color information is embedded in the histograms, its extraction is difficult due to several factors: • • •
Non-homogeneous illumination conditions High variety of eggs's color tones Large diversity of defects present
All these premises unwise a color processing segmentation at the current egg-shell sample and points to the analysis of the discontinuities pattern from the egg-shell regular background, to get a bivaluated classification.
,PDJHHQKDQFHPHQWLQWKHVSDWLDOGRPDLQ Image processing for enhancement purposes can be performed either in the spatial domain or other domain such as the Fourier domain [6]. In the spatial domain pixel values are modified according to transformations depending on its original value. Alternatively, they can be combined with or compared to others in their immediate neighborhood in a variety of ways, either to replace brightness values for image contrast expansion or noise smoothing [7]. In order to enhance image discontinuities from the egg-shell background, several high-pass filter have been tested. The Laplacian operator, which is an approximation to the linear second derivative of brightness B in direction x and y, was initially selected due to its invariance to rotation and hence insensitiveness to the direction in which the discontinuity runs. It highlights points, lines, and edges suppressing uniform and smoothly varying regions, but Laplacian image is difficult to interpret.
∇2 % ≡ ∂2% ∂2 [2 + ∂2 % ∂2 \2 The kernel that has been used to perform the image convolution, Laplacian, is a symmetrical 3x3
− 1 − 1 − 1 − 1 + 8 − 1 − 1 − 1 − 1 matrix that subtract the brightness values of each of the neighboring pixels from the central pixel. Consequently, regions with either uniform brightness or uniform gradient of brightness reduce their gray level to zero, but discontinuities result in a non-zero value. The Laplacian is a good high-pass filter, but nor specially good for demarcating edges. In most cases edges of defects or regions appear at least locally as a step in brightness, sometimes spread over several pixels. In noisy images Laplacian shows such points more strongly than the edges or boundaries of interest, Figure 2, thus leading to a disapproval of such a filter in this domain.
Figure 2. Original image red color plane, and negative images of corresponding Laplacian 3x3 and 5x5
Derivatives in one or more directions constitutes another transformation useful for edges detection. In an image with digitized finite pixels, a finite derivative is calculated by the difference value between adjacent pixels. The result is somewhat noisy but it can be smoothed averaging in the direction perpendicular to the derivative. Derivative images, as the Gradient North West 3x3 are obtained applying an asymmetric kernel, Figure3.
− 2 − 1 + 0 − 1 + 0 + 1 + 0 + 1 + 2
Some effects underlying this transformation are: • • •
Noise reduction by averaging together the adjacent pixels in each vertical column before taking the difference Due to the kernel asymmetry, central pixel is reinforced for variations in one direction Faster operation as it only requires a single pass through the image
Figure 3. Gradient_North 3x3 and Gradient_North West 3x3 negative images from the original red color plane
This kernel allows for eight gradient transformations, however in present application the directionality is not a fundamental issue as the eggs defects are random in either size, direction or location on the shell, thus being irrespective to work with one or the other among the eight possible directions. Kernel dimensionality has been reduced to the minimum , being defects usually small, to keep within the paradigm of low computational cost. For the same reason the combination of twodirectional derivatives (Roberts or Sobel filters), used to obtain a magnitude value insensitive to the edges, have also been disregarded.
'HIHFWVVHJPHQWDWLRQDQGIHDWXUHVH[WUDFWLRQSURFHVV Defects segmentation is one of the most widely used processing to extract relevant information from images (usually preprocessed), aiming at the division of the image into regions of interest that correspond to object or structural units in a scene, thus reducing the amount of data from about 0.5 million bytes to a few dozen and then to an affirmative or negative answer-action, as it is in current quality control application Segmentation can be described by analogy to visual processes as a foreground/background separation, implying that attention is fixed to a single kind of features, disregarding the rest of the image/scene. In this respect computer systems have the advantage of dealing with more than one type of feature of interest at a time. Features selection within a scene or image is a prerequisite for most measurements or interpretation of a scene. Usually a range of brightness was selected and associated to foreground pixels, rejecting the rest to the background. This operation is called thresholding, and brightness histograms of the image are useful to select such a value, that otherwise can be set up interactively. Noise that has not been eliminated due to small size kernel and lack of smoothing processes at the filtering step, can be conveniently reduced at the thresholding / binarisation phase by an appropriate fitting of the brightness threshold. One of the shortcoming of thresholding is that the pixels are selected primarily by brightness, and only secondarily by location, meaning that there is no requirement for regions to be continuous. Thus, instead of defining a region as a collection of pixels with similar brightness values in one (spatial relationship) or more images (temporal relationship) an alternate definition can be based on a boundary. In current images sample several intensity thresholds have been tested to fit the expert operator knowledge on the eggs-shell grading with the experimental results. Finally, the intensity cut-off is fixed at an intensity level of forty. This level allows for the splitting of the image pixels in two classes
(defects/foreground and regular shell/background) in a binary representation. On the resulting binarised image some structures and the egg-shell boundary in the gradient direction, appears as black pixels, Figure 4.
Figure 4. Negative binary images of Gradient North 3x3 and Gradient NorthWest 3x3 images
Object #1 #2 #3 #4 #5 #6 #7 #8
Area 15,00 26,00 17,00 113,00 19,00 39,00 16,00 27,00
Perimeter 19,83 32,90 19,07 105,84 19,31 41,63 19,66 27,49
Object #1 #2 #3 #4 #5 #6 #7 #8 #9 #10 #11 #12 #13 #14
Area 25,00 36,00 21,00 26,00 17,00 22,00 35,00 16,00 17,00 35,00 52,00 18,00 25,00 59,00
Perimeter 28,14 48,73 22,90 28,73 16,49 24,07 29,24 21,73 20,49 32,80 56,21 20,49 35,31 68,46
Figure 5. Label, area and perimeter of the selected objects/defects for the two Gradient images
Different representations are possible [8] on this binary image. By using a run-length encoding algorithm (chord encoding) that treats the binary image as series of scan lines, the image relevant information is stored in much less space than the original array of pixels. Thus, for each sequential line across each region or feature, the line and column locations and the line-length is recorded and used for area and position measurements, with low arithmetic processing involved. Objects segmentation is performed based on a four neighbour connectiveness. Then objects with more than fifteen pixels are labeled and some geometric attributes calculated, Figure 5. The location being meaningless in this application has been purposively suppressed. However, the perimeter that would not be computed in real time processing, is displayed to show its correlation with the area for small size defects and helps also to check for egg-shell boundary exclusion. Finally the overall defective area, expressed in pixels, is used to classify the eggs into two classes: "suitable" and "defective". Those eggs marked as defective would be rejected. The notion of defective egg is also context dependent and this is way a quality function must be, further on, formulated to account for both quantitative values of the egg-shell defects and human qualitative weighting.
&RPPHQWDQGIXWXUHWUHQGV First comment points to the statement that all processing must be addressed to the selection of simple, efficient and low computational cost classification algorithms for real time processing (about 500ms. per egg). In present images classification process, only two transformations are proposed: a directional derivative and a binarisation. At the same time that binary image is performed foreground defective pixels can be summed up and its final value will or will not activate the egg rejection. This preliminary results points also to the need of refinement on the lighting conditions in order to assure that changes in brightness are due to the presence of a structure or defect on the shell and not due to external light variations. Therefore, as there are not restrictions concerning lighting nor respect the eggs flow topology or length on the mechanical parts, an "ad hoc" fully suitable visual system environment can be designed to be integrated at the farm packers. All processes on point or neighborhood slow down defects detection speed accordingly, therefore fast and dedicated processors [9], [10] high memory size and high speed access to it, would be essential requirements for a final system that envisage real time automatic processing [11]. Due to the variable nature of the defects as well as to their random appearance on the eggshell, other sensor systems (acoustic) and light analysis (polarized) would be investigated to get a more complete and reliable defects classification.
$FNQRZOHGJPHQWV Present work is supported by a grant Comunidad Autónoma de Madrid-CAM/06G/038/96 and a grant of the CICYT-TIC96-1392 wherein the Instituto de Automática Industrial-CSIC and Instituto de Optica in Madrid, and Centro Nacional de Microelectrónica of Sevilla are jointly developing minimum hardware architectures for visual inspection applications. Authors wish to thanks past and present cooperation of Felipe Fadón from New Data Real Time Co. in Madrid and Jose Luis Ayuso from Avicú Co. in Guadalajara.
5HIHUHQFHV [1]
Patel, V.C., McClendon, W., Goodrum, J.W., “Detection of blood spots and dirt stains in eggs using computer vision and neural networks”, $SSOLHG(QJ,Q$JULFXOWXUH, 12, 2, 1996, pp. 253-258. [2] Conners, R.W., McMillin, C.W., Ng, Chong N., “The utility of color information in the location and identification of defects in surfaced hardwood lumber´3URF)LUVW,QWHUQ&RQI2Q6FDQQLQJ7HFKQRORJ\ LQ6DZPLOOLQJ, Miller Freeman, San Francisco, 1995, pp.253-258. [3] Patel, V.C., McClendon, R.W., Goodrum, J.W., "Crack detection in eggs using computer vision and neural network", $,$SSOLFDWLRQV, Vol.8, Nº 2, 1994, pp.21-30. [4] Moba Newsletters, “2PQLD 7KHEROGDGYDQFH”, Barneveld, Holland, 1996. [5] 87+6&$,PDJH7RRO", Univ. of Texas Health Science Center, San Antonio, USA. [6] Serrano Sánchez, J., "Procesado morfológico de imágenes (y II)", (XURIDFK(OHFWURQLFD, 1995, pp.34-40. [7] Russ, J.C., "7KHLPDJHSURFHVVLQJKDQGERRN", CRC Press, 1995. [8] Enciso, J., García-Alegre, M.C., Guinea, D., "Análisis morfológico de imágenes de superficies metálicas con fisuras tratadas con líquidos penetrantes", 7HFK5HSRUW,$,&6,&, 12/96. [9] Enciso, J., García-Alegre, M.C., "Procesamiento paralelo de imágenes con DSP-C407HFK5HSRUW,$, &6,&, 06/96. [10] Enciso, J., García-Alegre, M.C., Guinea, D., "Estrategias de visión activa en detección y análisis de fisuras con arquitecturas especializadas DSP)RUR,EHURDPHULFDQRGH7UDWDPLHQWR'LJLWDOGHLPiJHQHV 9LVLyQ ,QGXVWULDOH,PDJHQ0pGLFD, AIDO, Valencia, Oct.1996. [11] Guinea, D., Rodríguez, A., Cristobal, G., "Sistema Integrado de Visión Activa", )RUR,EHURDPHULFDQRGH 7UDWDPLHQWR'LJLWDOGHLPiJHQHV 9LVLyQ,QGXVWULDOH,PDJHQ0pGLFD, AIDO, Valencia, Oct.1996.
