Skin Melanoma Segmentation by Morphological

Skin Melanoma Segmentation by Morphological Approach Arlete Teresinha Beuren Uniguaçu - Faesi Faculdade de Ensino Superior de São Miguel do Iguaçu Rua Valentim Celeste Palavro, 1501 Jardim Panorama, São Miguel do Iguaçu, Paraná, Brazil

Rodrigo Janasieivicz Gomes Pinheiro

Neusa Grando FACET UTP Universidade Tuiuti do Paraná Rua Sydnei A Rangel Santos, 238 Curitiba-Pr, Brazil

Free Lance Profissional Photograph Rua Alberto Folloni, 634 Curitiba,Paraná, Brazil

[email protected]

[email protected]

arlete [email protected]

Jacques Facon PPGIA PUCPR Pontifícia Universidade Católica do Paraná Rua Imaculada Conceição 1155, Curitiba, Paraná, Brazil

[email protected]

ABSTRACT A new approach to extract the lesion region of melanoma is presented. This approach is based on color morphological operators which are defined from a lexicographic order on the HSI color space. The morphological filtering allows highlighting the region of melanoma that is then segmented by binarization. No a priori knowledge about the process of image acquisition and the type of melanoma is employed and a few heuristics are used. Tests were performed for two sets of benign and malignant melanoma images and compared with the ground-truth lesion segmentation by applying twelve metrics. The results prove the efficiency of this approach with regard to the automatic segmentation of both benign and malignant melanoma.

Categories and Subject Descriptors I.4.3 [IMAGE PROCESSING AND COMPUTER VISION]: Enhancement—Filtering ; I.4.6 [IMAGE PROCESSING AND COMPUTER VISION]: Segmentation—Pixel classification,partitioning

General Terms Algorithms

can appear and develop. A benign tumor is a cluster of cells that grows slowly and remains separated from the surrounding tissue and can be removed surgically. If malignant, skin tumor can invade nearby structures and acquire the ability to spread to other parts of the body through the blood for example. As the skin is formed by more than one cell type, different types of skin cancer exist, for example, cell carcinoma, basal cell carcinoma and melanoma. Although melanoma is the deadliest skin cancer, if early detected and identified by a dermatologist, the cure and treatment probability will increase significantly. From medical and dermatological points of view, the dermatologist usually uses four features known as ABCD rules to identify and recognize if a melanoma is benign or malignant: Asymmetry, irregular Border, varied Color and Diameter (greater than 6mm). Figure 1-(a) shows a typical benign nevus. Malignant transformation is a large lesion with color changes, increased diameter or irregular edges, as shown in Figure 1-(b). Non-melanoma skin cancers are the most common skin cancers. Malignant melanoma is a less common tumor of unknown etiology, and is the most serious one due to its ability in invading other parts of the body, even if the lesion is still small. Therefore if detected in early stages, the results of treatment grow up.

1. INTRODUCTION Skin cancer is a disease whose main characteristic is autonomous and uncontrolled growth of cells. When the immune system can not destroy the abnormal cells, a tumor

(a)

(b)

Figure 1: Examples of skin lesions: (a) Benign nevus, (b) Malignant A benign lesion usually is rounded or oval with regular edges

and uniform color.A malignant tumor is a large lesion with color mixture, increased diameter or irregular edges. From the computational point of view, the skin tumor recognition also uses ABCD rules. That requires in advance locating and extracting the skin lesion to allow quantifying the above features. We propose a new approach to automatic locate and extract both benign and malignant skin tumors. The melanoma images are filtered by color morphological operators based on a lexicographic order applied to the HSI color space. After a conversion to grayscale, the images are segmented by a binarization technique. A numerical evaluation of image segmentation using ground-truth segmentation and twelve metric allows us to measure the efficiency of the new proposed approach. Another comparison by Fuzzy C-Means skin tumor segmentation is also performed. The rest of the paper is organized as follows. Section 2 describes some researches about melanoma segmentation. Section 3 describes the HSI lexicographic order, the morphological operators and the threshold-based technique. Experimental results onto benign and malignant databases are discussed in Section 4.

2. STATE OF ART Various techniques have been proposed for automatic segmentation and recognition of benign and malignant melanoma. In case of melanoma segmentation, most of approaches are based on color comprehensive methods (with or without classification process) and filtering. To segment the area of melanoma, in [7] firstly a Hair Removal step based on Luv color space is applied. The hair removal algorithm is an exhaustive task and the authors have used it in few cases only. The lesion segmentation is based on the clustering of the two-dimensional Luv color space. A modified fuzzy Cmeans technique is used. A small database composed of 25 images, selected for the smoothness of their border, is used to compare the segmentation obtained by the method versus segmentation carried out by five expert dermatologists. No numerical evaluation is provided. In [12] the authors present an exemplar-based matching for lesion segmentation. Pre-segmented exemplars are chosen and for each one a color histogram is associated. The histograms are used to define probabilities in labeling all pixels in the query images as lesion or skin. These exemplar probabilities are combined using weights derived from the histogram distances to obtain the predicted lesion probabilities. Three dermoscopy datasets with ground truth segmentations have been used and the experiments have provided error segmentation rates between 13.70% and 26.76% Six different skin lesion segmentation methods based on thresholding, edge-based, and region-based techniques are compared in [9]. The segmentation methods were applied to 100 dermoscopic images and evaluated with four different metrics, using the segmentation ground truth images. The best results were obtained from the adaptive snake and expectationmaximization level set methods with error rates of 8% and 11%, respectively.

Figure 2: Flowchart of the melanoma segmentation

3.

METHODOLOGY

Human skin can present various types (oily or dry and with or without hair) and colors (light or dark) that may influence the mode of acquisition of tumors. Therefore computationally deal with human skin and skin tumors is a complex task. It is proposed to locate and extract the melanoma region without using prior knowledge about the process of acquisition and without using any specific technique of hair removal. The strategy adopted is based on color morphological filtering approach followed by binary morphological post-processing and concluded by a binarization step. Figure 2 depicts the strategy to segment melanoma lesions.

3.1

Color Mathematical Morphology

To treat images of different human skins (dry, oily, light, dark, hairy) showing variations in lighting, tumor format and color, one needs to develop flexible and fast processing techniques. By providing a wide range of filtering operators who have demonstrated their efficiency in many grayscale image applications, we have decided to employ mathematical morphology to achieve such goal. The innovative aspect of this approach will be to apply the color mathematical morphology which today represents an evolution in color image processing. The color mathematical morphology will be used in our approach to enhance the lesion contrast and color without changing neither its geometry nor its format and thus make easier its segmentation. Whatever the type of mathematical morphology, the basic principle is to extract information on geometry and topology of image structures based on the notion of order and more specifically based on the notion of minimum and maximum. Unlike grayscale morphology where the concepts of minimum and maximum between grayscale values do not

(a)

(d)

(c)

(d)

Figure 3: Filtering by Opening: (a) Original image, (b) 3×3 structuring element , (c) 5×5 structuring element , (d) 7 × 7 structuring element present theoretical difficulty, sorting color values is not an easy task. Special care should be taken with regard to ordering and to color space where the ordering is applied. We have chosen the lexicographic order proposed by [4] onto HSI color space. This ordering avoids the generation of false colors during the process and represents a complete order like dictionary ordination. The minimum value between two three-component vectors P1 (H1 , S1 , I1 ) and P2 (H2 , S2 , I2 ), also named as infimum ∧, is defined as follows:

∧ {P1 (H1 , S1 , I1 )

P2 (H2 , S2 , I2 )} (1) 8 I < I 1 2 > > > < or = I1 = I2 and S1 < S2 > > > : or I1 = I2 and S1 = S2 and H1 < H2 ,

The maximum value, also named as supremum ∨, can be defined in a similar way. Then, from this lexicographic order, morphological erosion ε and dilation δ of image f by using the structuring element B at the pixel x with respect to structuring element support set E ⊂ ℜ are : εB (f (x))

= ∧{f (y) − B(x − y) : y ∈ E}

(2)

δ B (f (x))

= ∨{f (y) + B(x − y) : y ∈ E}

(3)

It is noticed that the lexicographic ordering starts the comparison from the channel I. If the values of I between two pixels are equal, the second channel S is used. In case of equality, the third channel H is also used. Then, from this lexicographic order, erosion ε and dilation δ of image f by structuring element B at the pixel x with respect to support set E ⊂ ℜ are: εB (f (x)) = ∧{f (y) − B(x − y) : y ∈ E}

(4)

δ B (f (x)) = ∨{f (y) + B(x − y) : y ∈ E}

(5)

3.2 Morphological Filtering Through the color morphological filtering, the objective is to enhance the colors of the lesion in order to highlight their geometry, respecting its format without deforming it. No specific hair removal technique has been applied. Both hair and noise elimination is implicitly embedded in this step.

To have interesting properties with regard to highlight and noise removal, the process of morphological opening (erosion followed by dilation defined by the equation 6) was adopted.

γ B (f ) = δ B (εB (f ))

(6)

Figure 3 illustrates opening results by HSI lexicographic ordering with 3 × 3, 5 × 5 and 7 × 7 square structuring elements. Tests have allowed defining the opening with 7 × 7 square structuring element as the best sequence of filtering. We can check that the chosen lexicographical ordering removes excessive hair and highlights, homogenizes the color of the lesion without modifying neither the edge nor the geometry.

3.3

Threshold-based segmentation

The melanoma region extraction is carried out by global thresholding. First filtered colored images are converted to grayscale ones. Some binarization methods hve been tested and the binarization by Renyi ’s entropy [6] has proved to be the most efficient lesion thresholding process avoiding to affect the geometry and shape of lesions and to add or removing parts. Through the results of segmentation by thresholding shown in Figure 4, we can see some of the benefits of color morphological filtering. Without filtering, the segmented regions are imperfect. The color morphological opening led to the removal of colored noise and highlights while preserving the geometry and shape of the lesions. Figure 5 depicts other binarization results and other advantages and results of morphological filtering.

3.4

Binary Filtering

This step consists in filling holes and removing noise. The hole filling is based on reconstruction ρ given by equation 7, where δg (f ), f and g are the geodesic dilation, marker image and mask one, respectively. ρ(f ) = lim δg (δg (...δg (f ))) with δg (f ) = δ(f ) ∧ g n→+∞ | {z }

(7)

n

Noise filtering aims to suppress local minima or maxima less than a give size, to remove noise whithout affecting structures of interest. Noise filtering is performed from a binary opening with 7 × 7 square structuring element. Figure 5 depicts some results of hole filling and noise filtering.

(a)

(d)

(c)

(d)

Figure 4: Benefits of Lexicographic filtering: (a) Original image, (b) Binarization of original image, (c) Lexicographic filtration result, (d) Binarization of filtered image

(a)

(d)

(c)

(d)

Figure 5: Results: (a) Original image with segmented lesion border, (b) Binarization result, (c) Filtration Result, (d) Ground-truth image

4. EXPERIMENTS AND EVALUATION To measure the efficiency of the proposed approach, two procedures were performed. The first one consists of a numerical evaluation from two benign and malignant melanoma databases, each each one composed of 100 images. Most of images have been downloaded from site [5] where every human being can learn and test his knowledge on melanoma. The ground-truth segmentation for each image was manually carried out by an image processing specialist. Figure 5 shows examples of ground-truth images and depicts some final results where the segmented contour of melanoma is added to original one. We have compared, in the second procedure, the proposed methodology with the modified melanoma segmentation approach of Umbaugh and Cheng [1]. The RGB color space was replaced by T SL color space considered by [10] as more suitable color space for human skin processing. Four features (mean, standard deviation, skewness and kurtosis) have been used for each T, S, L channel resulting in a twelve feature vector. Finally, the multi-layer perceptron classifier (M LP ) was replaced by Fuzzy C-Means one [11]. Both proposed color morphological and Fuzzy C-Means classifier approaches have been evaluated and compared from twelve metrics which are defined as follows: • M E (Misclassification Error [8]) returns 0 in case of perfect segmentation: ME = 1 −

|Bg ∩ Bs | + |Fg ∩ Fs | |Bg | + |Fg |

(8)

where (Bg , Fg ) represent the background and foreground of ground-truth segmentation, (Bs ,Fs ) represent the background and foreground obtained by segmentation approach, and | | is the cardinality. • RAE (Relative Foreground Area Error [8]) returns 0 in case of perfect segmentation: 8 |F |−|F | g s > if |Fg | > |Fs | < |Fg | (9) RAE = or > : |Fs |−|Fg | if |F | < |F | g s |Fs |

where Fg is the foreground of ground-truth segmentation, Fs the foreground of segmentation approach, and | | the cardinality.

• P (Precision [2]) returns 1 in case of perfect segmentation: P =

T+ T+ + F+

(10)

• A (Accuracy [2]) returns 1 in case of perfect segmentation: A=

T+ + T− T+ + T− + F+ + F−

(11)

• R (Recall [2]) returns 1 in case of perfect segmentation: R=

T+

T+ + F−

(12)

• ER (Error [2]) returns 0 in case of perfect segmentation: ER =

F+ + F−

T+

(13)

• F M (F-Measure [3]) returns 1 in case of perfect segmentation: 2∗P ∗R (14) FM = P +R • N RM (Negative Rate Metric [3]) returns 0 in case of perfect segmentation: N RM

= where and

N RF N + N RF P 2 F− N RF N = + T + F− F+ N RF P = − T + F+

with • T + the true lesion pixels (lesion pixels in the groundtruth detected by the segmentation approach as lesion pixels); • T − the true background pixels (background pixels in the ground-truth detected by the segmentation approach as background pixels); • F + the false positive lesion pixels (background pixels in the ground-truth erroneously labeled as lesion pixels by the segmentation approach); • F − the false negative lesion pixels (lesion pixels in the ground-truth erroneously labeled as background pixels by the segmentation approach). The color morphological and Fuzzy C-Means approach efficiency evaluation is depicted in Tables 1 and 2, respectively. While the original approach of [1] has obtained 78% of melanoma segmentation to a database of 160 color tumor images, the modified approach achieved higher segmentation rates (± standard deviation) of true positive T + = (86, 69 ± 18, 80)% and T + = (89, 33±18, 14)% to benign and malignant melanoma databases, respectively. We can observe that color morphological approach is more efficient than modified Fuzzy C-Means one with higher segmentation rates (± standard deviation) T + = (95, 67 ± 06, 17)% and T + = (97, 22 ± 04, 96)% to benign and malignant melanoma databases, respectively. We can observe this higher efficiency not only from true positives T + rates, but also from the values of the metrics M E, RAE, T − , F + , F − , P , A, R, ER.. And also to standard deviation rates . We can also emphasize that, the average F M and N RM metrics from the color morphological approach are 95, 22%, 04, 79% and 94, 65%, 05, 56% to benign and malignant melanoma databases, respectively, with standard deviations lower than 03, 50%.

Table 1: Evaluation of ME RAE BenignAverage 04,35 12,37 ± ± ± Deviation 04,00 13,87 Malignant Average 05,84 12,30 ± ± ± Deviation 04,46 09,10

Table 2: Evaluation of ME RAE Benign Average 08,00 17,25 ± ± ± Deviation 08,00 17,37 Malignant Average 10,40 18,45 ± ± ± Deviation 07,04 16,24

Benign and Malignant Morphological segmentation (%) T+ T− F+ F− P A R ER FM 95,67 94,75 05,25 04,33 95,26 95,21 95,67 04,79 95,22 ± ± ± ± ± ± ± ± ± 06,17 06,67 06,67 06,17 05,57 03,79 06,17 03,79 03,88 97,22 91,66 08,34 02,78 92,62 94,44 97,22 05,56 94,65 ± ± ± ± ± ± ± ± ± 04,96 07,75 07,75 04,96 06,24 03,96 04,96 03,96 03,78

Benign T+ 86,69 ± 18,80 89,33 ± 18,14

and Malignant Fuzzy C-Means segmentation (%) T− F+ F− P A R ER FM 92,63 07,15 13,26 93,41 89,78 86,73 10,22 88,33 ± ± ± ± ± ± ± ± 09,22 09,15 18,81 07,27 09,12 18,81 09,12 12,52 88,43 11,57 10,67 89,79 88,88 89,33 11,12 87,82 ± ± ± ± ± ± ± ± 10,73 10,73 18,14 07,86 08,84 18,14 08,84 13,33

We can also observe that the standard deviation values are lower than the standard deviation values of these metrics for the modified Fuzzy C-Means approach. This demonstrates the efficiency of the proposed methodology. A Figura 6 depicts a comparison between color morphological approach (red contour) and Fuzzy C-Means one (blue contour) for images showing large variations in color, edge and shape. The average F M and N RM values are detailed.

5. CONCLUSION A morphological approach to segment the lesion of melanoma has been proposed. This study permits to conclude that the use of color, grayscale and binary morphological operators can efficiently detect melanoma lesions. The fact of chosing color mathematical operators based on HSI lexicographic order was determinant to filter and enhance melanoma colors and to make easier the segmentation by binarization. The choice of binarization process by Renyi ’s entropy has also been important to better detect the region of interest of lesion. The segmentation approach was tested on two 100 benign and malignant image databases and quantitatively evaluated by comparing the results to ground-truth segmentation. The values of the twelve metrics used to evaluate the efficiency of the proposed methodology show that, although the approach is relatively simple, the results are quite satisfactory. More particularly the high value of F M metric and the low value of N RM one for the two databases show that the approach is promising.

6. REFERENCES [1] Y. Cheng and S. Umbaugh. Color-based diagnosis: Clinical images. Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE, https://www.ee.siue.edu/CVIPtools, 2005. [2] A. Del Bimbo. Visual Information Retrieval. Morgan Kaufmann, 1999.

NRM 04,79 ± 03,79 05,56 ± 03,96

NRM 10,20 ± 09,12 11,12 ± 08,84

[3] B. Gatos, K. Ntirogiannis, and I. Pratikakis. Icdar 2009 document image binarization contest (dibco 2009). International Conference on Document Analysis and Recognition, pages 1375–1382, 2009. [4] G. Louverdis, M. Vardavoulia, I. Andreadis, and P. Tsalides. A new approach to morphological color image processing. Pattern Recognition, 35:1733–1741, 2001. [5] Oncopeau. Testez-vous en images. oncopeau. http://info-melanome.net/pub/en savoir plus/testez vous en images/ , 2012. [6] P. Sahoo, C. Wilkins, and J. Yeager. Threshold selection using renyi´s entropy. Pattern Recognition, 30(1):71–84, 1997. [7] P. Schmid-Saugeon, J. Guillod, and J. Thiran. Towards a computer-aided diagnosis system for pigmented skin lesions. Computerized Medical Imaging and Graphics, (27):65–78, 2003. [8] M. Sezgin and B. Sankur. Selection of thresholding methods for non destructive testing application. In International Conference On Image Processing, ICIP2001, pages 764–767, 2001. [9] M. Silveira, J. Nascimento, J. Marques, M. R.S., M. T., Y. S., M. J., and R. J. Comparison of segmentation methods for melanoma diagnosis in dermoscopy images. IEEE Journal of Selected Topics In Signal Processing, 3(1):35–46, 2009. [10] V. Vezhnevets, V. Sazonov, and A. Andreeva. A survey on pixel-based skin color detection techniques. In Proceeding of Graphicon, pages 85–92, 2003. [11] Y. Yong, Z. Chongxun, and L. Pan. A novel fuzzy c-means clustering algorithm for image thresholding. Measurement Science Review, 4(Section 1):11–19, 2004. [12] H. Zhou, J. Rehg, and M. Chen. Exemplar-based segmentation of pigmented skin lesions from dermoscopy images. IEEE ISBI International Symposium on Biomedical Imaging, pages 225–228, 2010.

(a)

F M = 71, 78% N RM = 22, 56%

F M = 85, 24% N RM = 13, 09%

F M = 75, 76% N RM = 19, 54%

F M = 85, 29% N RM = 13, 51%

(b) Fuzzy C-Means

F M = 95, 87% N RM = 3, 98%

F M = 98, 53% N RM = 1, 48%

F M = 98, 99% N RM = 1, 00%

F M = 95, 68% N RM = 4, 49%

(c) Morphological approach

(d) Figure 6: Comparison: (a) Original Image, (b) Fuzzy C-Means segmentation, (c) Morphological segmentation, (d) Original Image with two contours.