Mitosis extraction with a new color object detector Audrey Ledoux, No¨el Richard, et Anne-Sophie Capelle-Laiz´e
University of Poitiers, XLIM-SIC JUR CNRS 7252, Bd Marie et Pierre Curie, T´el´eport 2, 86962 Futuroscope Cedex, France. Tel : +33 (0)5 49 49 74 92, Fax : +33 (0)5 49 49 65 70
[email protected]
Abstract The hit-or-miss transform is a mathematical morphological process designed to find objects in images. Its extension to grayscale domain is not unique but Barat’s method appears as the most appropriate to find specific objects with a spatio-colorimetric template. This template, constructed with non-flat structuring elements, allows a color and shape selectivity. The color extension is constructed with a new color mathematical morphology based on the concept of “convergence”. In this paper, we show the ability to parameterize this tool from the shape and the color of objects. We illustrate the use of this color objects detector in the mitosis detection context. Key words Hit-or-miss transform, mathematical morphology, perceptual color, mitosis detection.
1
Introduction
The pattern detection is one of the major domain of the image processing science. In this context, the hit-or-miss transform was one of the first approaches. Originally defined in binary domain, some of its grayscale extensions allow to find objects upon their shape and contrast relating to the image background. However, only one method allows to extract objects with variations in shape and simultaneously in contrast. This method is called MOMP and was developed by Barat [2]. It requires non-flat structuring element, so the MOMP extension to color, called “Color MOMP”, needs a dedicated morphological framework. Actually only the color morphological approaches based on distances function and convergence coordinates allows this possibility [5]. In this article we focus on the detection of mitoses in areas of breast tumor to show the ability to construct spatio-colorimetric templates and to tune them. Currently this type of cancer is a major cause of death worldwide. The counting of these cells is an important feature to assess the severity of the tumor. We show how the “Color MOMP” transform can be used on to detect complex colour objects.
A. Ledoux, N. Richard, A-S. Capelle-Laiz´e
2
Context
The screening for breast cancer is very common nowadays but its severity measure is essential to complete the analysis after the tumor detection. Then specialists analyze tissues extracted from the tumor region and produce histological slides to assess the severity. The number of mitotic cells in the tumor region is the most important information for the estimation of the grade. Normally the specialists segment and count mitotic cells manually. The goal is to automatically process the detection with a computer analysis to help the specialists [11,3]. However, the mitotic cells detection is a challenging task in images because cells differ in appearance (see figure 1). In this paper we extract the mitotic cells in the images of the ICPR2012 contest entitled “Mitosis Detection in Breast Cancer Histological Images (MITOS dataset)” [12].
(a) Prophase
(b) Metaphase
(c) Anaphase
(d) Telophase
Figure 1. Four mitosis phases
3
“Color MOMP”
The algorithm Hit-or-Miss is a very powerful shape detection belonging to the mathematical morphology. It allows to retrieve an object taking into account its shape and its support. From its binary writing [9], various methods are developed in grayscale [6,10]. Some authors differ from others in their method using a nonflat structuring element [8,4,1,7]. This type of structuring elements gives the ability to detect objects with a variation in shape or in contrast (relative to the image background). The Barat method [2] appears to be the most suitable in the search for objects and is called MOMP (Multiple Objects Matching using Probing). The method uses a probing of the image with two non-flat structuring elements (see figure 2). The used template is composed with two structuring elements, the inferior g 0 and the superior g 00 . All objects fully included in this template are considered to correspond to the desired object. In other words, an object is detected when the distance between both structuring element is inferior to the δ value which is the difference between both structuring elements at centering point placed in their
Mitosis extraction
reference position. With g = {g 0 , g 00 } the set of used structuring elements, the MOMP transformed is written as: M OM P (f, g)(x) = δg (f, (−g 00 )r )(x) − εg (f, g 0 )(x)
(1)
where δg and εg are respectively the dilation and erosion for grayscale images, and g r is the reflectivity function of g (g r = g(−y)). The MOMP allows variation in shape and simultaneously in contrast. In [5], we shown that the structuring elements magnitudes allows to select the desired contrast range. We also defined the parameter δ as the selectivity parameter, it is the contrasts range size. With a new color mathematical morphology writing which is based on the “convergence” concept and which allows non-flat structuring elements writing, we extended the MOMP algorithm to color images. This new algorithm is named “Color MOMP”. “ColorM OM P 00 (f, g)(x) = δc (f, −(g 00 )r )(x) − εc (f, (g 0 )r )(x) _ ^ = f (x − y) − g 00 (y) − x∈Df ,y∈Dg
c
(2) f (x − y) − g 0 (y)
x∈Df ,y∈Dg
c
where + and − are the addition and subtraction for color coordinates. c
c
(a) Template constructed with two structuring elements
(b) Probing of the function with the template
Figure 2. Principle of the MOMP transformed
4
Spatio-colorimetric template specification
With this writing form, “Color MOMP” allows to detect complex objects in a natural way. The structuring elements form a template which determines the desired shape. The convergence coordinates for the morphological operators are defined from the objects and background colors. Finally, the selectivity parameter called δ determines the distance between the hyper-surfaces of the two structuring elements. The selectivity parameter value adjusts the desired variability in the color contrast. The “Color MOMP” interest lies in the ability to specify the search template from spatio-color models. The next step is an example of a user’s reasoning to define the
A. Ledoux, N. Richard, A-S. Capelle-Laiz´e
parameters. With the analyse of some mitotic cells extracted from several images of the database (Figure 3(a)), we define all the parameters. In first, to define both color convergence, we manually select a median color from colors in the mitotic cells and a median color among those present in the neighborhood of the cells (Figure 3(b)). As the mitotic cells can take many shapes, we chose a spherical template with a different diameter for the two structuring elements. The lower structuring element diameter is selected according to the minimum width found and equal to 9 pixels. The diameter of the upper structuring element is chosen depending on the maximum size of the cells found in the images and is equal to 41 pixels. The figure 3(d) shows the used template. Finally the selectivity parameter is fixed to tolerate the differences between strongly or slightly textured cells and light or dark cells, and is equal to 25.
(a) Different mitotic (b) Convergence cell example colors
(c) Cell surface view example (luminance)
(d) Template
Figure 3. Example of mitotic cells extraction to the “MITOS dataset”
5
Results
The MITOS database is composed of 50 images from 5 breast cancer biopsy slides with more than 300 mitosis. In this part, to detect mitosis, we use the parameters described in the section 4. One of the detection complexity lies in the texture variation inside the mitotic cells. When the cell is in the prophase stage, the chromosomes are diffused in the nuclei while in the others stages, the chromosomes are concentrated in one or two areas. The selectivity parameter must be large enough to allow cells with higher or lower texture complexity. Unfortunately, with a high selectivity parameter, more distant objects are accepted. But the most similar cells enter fully into the template and have often some leeway, then the result is a group of pixels. Our objective in this low-level processing is to reduce at least the number of undetected mitosis. In figure 4(b), one result example could be observed. To reduce the count of false detection, some easy post-processing steps are used. First one is linked to the mitotic cell shape. As some cells have a bigger spatial shape
Mitosis extraction
(a) Initial image
(b) “Color MOMP” result
(c) Filtered result
Figure 4. Mitosis detection result
than those specified by the morphological templates, they induces several detection inside the same cell. So a morphological binary closing on the “Color MOMP” result agglomerates the nearest pixels, then an opening deletes small groups of pixels that could not be a mitotic cell. In table 1, we show the gain obtained by this simple post-processing step. The initial objective to find all the mitotic cells is reached, but now we need to optimize the selectivity of the “Color MOMP” transform to reduce the false alarm count. The problem is induced by the coloring product. To highlight mitotic cells Hematoxylin and Eosin stain (H&E) are used, but such coloring product gives a particular color for the mitotic cells but also for young cells, that compose the false alarm in our colour pattern detection. Table 1. Number of detected objects with the “Color MOMP”and after filtering
“Color MOMP” result
false positive true positive false negative “Color MOMP” false positive result true positive + filtering false negative
6
Object count 9463 47 1 402 47 1
Figure 5. Undetected cell with spatial template supports
Conclusion and Perspectives
Thanks to a new color mathematical morphology based on the concept of convergence, that allows the non-flat structuring element writing, we extend the MOMP transform of Barat [2]. This morphological process called the “Color MOMP” allows to extract objects specified trough a spatio-colorimetric templates. This template allows to find objects include between the spatial limits defined by both structuring
A. Ledoux, N. Richard, A-S. Capelle-Laiz´e
elements and with colorimetric variation included between the two hyper-surfaces associated. The major interest of this framework is to allow an easy design of templates from images examples or theoretical models. The second interest is to embed directly the notion of spatial and colorimetric selectivity by the distance between the two non-flat structuring elements defining the template. Recently, a contest was posted to compare pattern detection system for mitotic cells detection. This problem is complex due to the cells evolution and coloring product used. Due to the medical domain, we chosen to focus the severity parameter on the detection of all mitotic cells. Upon this objective, we success to find all the expected cells. But due to these choices, we obtain an important number of false alarm. Some post-processing approaches are needed to reduce this number, and we are working to do it. The publication of the contest has not yet occurred so we can not be compared our results. Last element of interest for us in this context, lies in the fact that our approach is directly extensible to multispectral images, thanks to using the right multispectral distance. Our first results gives some nice perspectives generalizing the “Color MOMP” approach.
References 1. G.J.F. Banon and S.D. Faria. Morphological approach for template matching. In Proceedings of X Symposium on Computer Graphics and Image Processing (CGIP), pages 171–178. IEEE, 1997. 2. C. Barat, C. Ducottet, et al. Pattern matching using morphological probing. In International Conference on Image Processing (ICIP), volume 1, pages 369–372. IEEE, 2003. 3. C.H. Huang and H.K. Lee. Automated mitosis detection based on exclusive independent component analysis. In 21st International Conference on Pattern Recognition (ICPR), pages 1856–1859. IEEE, 2012. 4. M. Khosravi and R.W. Schafer. Template matching based on a grayscale hit-or-miss transform. Transactions on Image Processing, 5(6):1060–1066, 1996. 5. A. Ledoux, N. Richard, A.S. Capelle-Laiz´e, and C. Fernandez-Maloigne. Color hit-or-miss transform on dermatological images. In Twentieth Color and Imaging Conference, pages 164–169, 2012. 6. F. Odone, E. Trucco, and A. Verri. General purpose matching of grey level arbitrary images. Visual Form, pages 573–582, 2001. 7. B. Raducanu and M. Grana. A grayscale hit-or-miss transform based on level sets. In International Conference on Image Processing, volume 2, pages 931–933. IEEE, 2000. 8. C. Ronse. A lattice-theoretical morphological view on template extraction in images. Journal of Visual Communication and Image Representation, 7(3):273–295, 1996. 9. J. Serra. Image Analysis and Mathematical Morphology, volume I. Academic Press, 1982. 10. P. Soille. Advances in the analysis of topographic features on discrete images. In Discrete Geometry for Computer Imagery, pages 271–296. Springer, 2002. 11. C. Sommer, L. Fiaschi, F.A. Hamprecht, and D.W. Gerlich. Learning-based mitotic cell detection in histopathological images. In 21st International Conference on Pattern Recognition (ICPR), pages 2306–2309, 2012. 12. IPAL UMI CNRS TRIBVN Piti´e-Salpˆetri`ere Hospital The Ohio State University. “mitosis detection in breast cancer histological images (mitos dataset)”. http://ipal.cnrs.fr/ICPR2012/.
