Tu34.5
2016 14th International Conference on Control, Automation, Robotics & Vision Phuket, Thailand, 13-15th November 2016 (ICARCV 2016)
Reaction-diffusion based Level Set Method with Local Entropy Thresholding for Melasma Image Segmentation Xu Zhang, Yunfeng Liang, Dongyun Lin, Zhiping Lin School of Electrical & Electronic Engineering Nanyang Technological University Singapore E-mail:
[email protected]
Abstract—This paper proposes a new method for melasma pigmentary area segmentation utilizing reaction-diffusion based level set model (RDLS M) together with local entropy thresholding. In the adopted level set model, a diffusion term is used to regularize the level set function while a reaction te rm with anticipated sign property is used to force the zero level set towards desired locations. Then local entropy thresholding is applied to address the over-segmentation issue of RD LS M and to extract desired boundaries with higher overall local entropy. As a result, the melasma pigmentary areas and the normal skin areas can be better identified. Experimental results show that the proposed method performs well for melasma image segmentation, especially for cases with severe non-uniform illumination distribution. Keywords—Melasma image segmentation; local entropy thresholding; reaction-diffusion based level set model
I. INT RODUCT ION Melasma is one of the most common skin discolorations with the symptoms of irregular brown patches normally found on sun-exposed areas [1]. For treatment of melas ma, an accurate assessment of the severity is a prime requisite. Current diagnosis is conducted by visually observing melasma pigmentary area and extent of pigmentation by clinicians, which is biased since different clinicians may subjectively make inconsistent assessments for the same cases based on their own experience. In contrast, computerized methods are able to automatically provide standardized scores, which can be a good reference for medical diagnosis. Image segmentation is a fundamental step to determine the area of involvement for melasma severity assessment. In the few related research works on the topic of computerized methods for melas ma image segmentation, Extreme Learn ing Machines (ELM) based segmentation method was proposed in [2], where 20 skin texture features are extracted from patients ’ images as inputs to the ELM classifier for segmentation. This method is effective for cases with clear boundaries but not robust for some other cases. More recently, a hybrid threshold optimization method was proposed for melasma lesion segmentation [3]. The optimal threshold is acquired by minimizing the deviation between the given image and the threshold surface. This segmentation method is shown to achieve superior performance to some other thresholding
978-1-5090-3549-6/16/$31.00 ©2016 IEEE
Steven Tien Guan Thng, Emily Yiping Gan, Evelyn Yuxin Tay National Skin Center 1 Mandalay Road, Singapore 308205 Singapore
methods for melas ma severity assessment [3]. However, for cases where illu mination of the image is severely nonuniformly distributed, the results are still not satisfactory. Besides the segmentation methods mentioned above, level set is another image segmentation method which has been extensively studied in computer vision and image processing [4-7]. Several works that apply level set methods on medical image segmentation achieve promising results. In [7], an adaptive multi-grid level set method is developed for medical image segmentation and is especially efficient for images with non-sharp segment boundaries. In [8], a level set method which adaptively integrating local and global intensity information is proposed to segment images with intensity inhomogeneity. Distance regularized level set evolution model is proposed in [4] and it is able to greatly reduce the computational cost with sufficient numerical accuracy. In [6], an image segmentation model derived from level set methods together with reactiondiffusion equations is proposed. This method is effective for images with boundary concavity and blurred boundaries. Inspired by the promising performance of the above level set methods, in this paper we apply one of them to melasma lesion segmentation problems, namely the reaction-diffusion based level set model (RDLSM) [6]. However, the algorithm is sensitive to illu mination and hence a local entropy thresholding method is incorporated. The proposed method can produce reasonable image segmentation results even for forehead melasma images with highly biased illumination. The remainder of this paper is organized as follows. Section II presents the methodology of the proposed method. Section III gives experimental results and discussions. Finally, the paper is concluded in Section IV. II. M ET HODOLOGY A. Pre-processing Before segmentation for normal skin and melas ma lesion area, several image preprocessing techniques are applied to suppress noise and artifacts. At first, color difference method [2, 9] is applied to construct a mask that excludes non-skin area on a face image. Successively, the image is cut in line with MASI scoring method [10] and is labeled as malar case or
forehead case according to the face region contained in the resulting image. For malar cases, in order to enhance the contrast of the image, a weighting scheme is proposed for channels ܽ , ܾ in Lab and channel saturation ( )ݏin HSV co lor space. Since ܽ and ܾ contain equal chrominance features of the image, they are combined to produce a representative value of chrominance
as in (1). ଵ
ൌ ට ሺܽଶ ܾଶ ሻ
(1)
ଶ
Saturation and c are combined with equal weight in the weighting scheme as in (2): ଵ ൌ ሺ ݏ ܿሻ
where ܯis the average intensity of the heterogeneous region that consists of both background and melas ma p ixels. Because the value ܯ
ǡ݂ሺݔǡ ݕሻ is made to have opposite signs inside and outside of the objects. Therefore, making ݂ ሺݔǡ ݕሻ equal to zero will force the zero level set function to move towards the boundary of the objects. In our application, the melasma reg ions are firstly segmented out by RDLSM. However, the algorith m tends to produce over-segmented results, i.e., there are many normal skin areas that are misclassified as melas ma lesion shown in Fig. 2. To allev iate the over-segmentation problem, a local entropy thresholding method is introduced as detailed below.
(2)
ଶ
Subsequently, contrast-limited adaptive histogram equalization (CLAHE) [11] is applied on the image that is constructed by the proposed weighting scheme. As for forehead cases, frontal bone structures of the patients normally cause a strongly biased spread of illu mination over the entire forehead region. The illu mination effect could result in misclassification for some shaded areas. To suppress illu mination effects, we conduct single scale retinex theory [12] for illu mination normalization. One example is illustrated in Fig. 1.
Fig. 2 RDLSM result for forehead case. The outer heavy boundaries correspond to the boundaries of masks.
2) Local Entropy Thresholding
(a) (b) Fig. 1 (a) Input image and (b) output of single scale retinex processing
B. Segmentation 1) Reaction-diffusion based Level Set Model (RDLSM) Reaction-diffusion based level set model (RDLSM) [6] defined in (3) is adopted as our segmentation method: డఝ డ௧
ൌ ݀݅ݒሺ݃ሺ ȁ ࣌ ݑȁሻ߮ሻ ݂ ሺݔǡ ݕሻ , inȳ ൈ ሺͲǡ λሻ(3)
with given image ݑ, in itial level set function ߮ሺݔǡ ݕǡ Ͳሻ ൌ డఝ ቚ ൌ Ͳ. ߮ ሺݔǡ ݕሻ in Ω and Neumann boundary condition డ డஐ
݃ is diffusion coefficient and ݊ is the exterio r normal to the boundary ߲ȳ. In (3), the first term is the diffusion term, which is designed to regularize the level set function and controls the smoothness of the boundary. ݃ሺݔሻ ൌ ߤ݁ ି௫Ȁସ is a monotonically decreasing function that aims to control the smoothing process. ݑఙ ൌ ܩఙ ݑ כis the convolution of given image ݑwith the Gaussian function ܩఙ with standard deviation σ. The second term ݂ (ݔ, )ݕis the reaction term defined as follows: ݂ ሺݔǡ ݕሻ ൌ ܯȂ ݑሺݔǡ ݕሻ (4)
Fro m the segmentation results by RDLSM , it is observed that the over-segmentation issue arises from both oversensitivity of the algorith m and the weak local lu minance of normal skin area. Since the lu minance of false positively classified regions is relatively lower than that of normal skin areas, the RDLSM algorith m tends to group them as the melasma region. Hence, it would be effective to pick out extreme dark reg ions from segmentation results since they are clearly the melasma lesion areas. Fo r these regions, the intensity difference between two sides of the lesion boundary should be greater than that of the false positively classified regions. Therefore, entropy could be an effective index for boundary detection as the local entropy along the boundary of melasma lesion is higher. Based on the above analysis and the observation that melasma areas are normally large patterns, local entropy thresholding is introduced to help extract the boundaries between melasma and normal skin regions. The detailed steps of local entropy thresholding is presented as following: a) Extract the foreground boundaries by applying RDLSM; b) Construct a local entropy [13] image fro m the orig inal gray level melasma image as defined in (5) ሺȳ ሻ ൌ െ σିଵ ୀ ݈݃
(5)
where ሺ ȳ ሻ is the local entropy for neighborhood ȳ . ܮ is the number of gray levels of the image, and is the probability of gray level j occurring in ȳ that is
estimated by the normalized h istogram. Record the largest value of ሺ ȳ ሻ as ܧ௫ .
To summarize, a flowchart of the proposed melis ma image segmentation method is shown in Fig. 5.
In the local entropy calculat ion, the non-skin region is filled in by inward interpolation [14] as the original values are noises for the process. Fig. 3 shows an example of an image after interpolation.
Input Image
M alar Case
Forehead Case
Weighting Scheme Single Retinex Theory CLAHE Fig. 3 Inward interpolation of masked image
c)
Grayscale Image
Calculate the average local entropy ܧ௩ for each boundary and record the largest value ܧ௩ ; the first filtering condition to extract melasma lesion is: ܧ௩ ͳ݈݄݀ݏ݁ݎ݄ݐൈ ܧ௩
RDLSM
(6)
where ͳ݈݄݀ݏ݁ݎ݄ݐis a constant between 0 and 1. This condition aims to filter out darker regions that are more likely to be melasma lesion areas.
Primary Output
d) Another term ܧ௧௧ that measure the length of region boundary is computed in (7): (7)
Local Entropy Thresholding
where the numerator is the summation of local entropy along a boundary, and ܾܽ ݁ݏhas the scale approximately equals to the number of pixels of the longest boundary. The second filtering condition to extract melas ma lesion is:
Final Output Fig. 5 Flowchart of the proposed melisma image segmentation method
ܧ௧௧ ʹ݈݄݀ݏ݁ݎ݄ݐൈ ܧ௫
III. EXPERIMENT
ܧ௧௧ ൌ
σ ாሺǡሻ ௦
(8)
where ʹ݈݄݀ݏ݁ݎ݄ݐis another constant between 0 and 1. This constraint is based on the observation that melasma areas are normally grouped as large patterns . Thus the calculation of ܧ௧௧ is biased towards longer boundaries. The final segmentation result for a forehead case after the local entropy thresholding filtering is presented in Fig. 4.
A. Experiment Data The same image set adopted in the oHybrid segmentation method [3] is utilized, wh ich contains images fro m 29 patients and recorded by National Skin Center of Singapore. B. Parameter Setup Initial level set function ߮ ሺݔǡ ݕሻ is set to be 1 for malar case melasma image segmentation and 0 for the forehead case. ͳ݈݄݀ݏ݁ݎ݄ݐis normally chosen between 0.7 and 0.9 for severe cases. For mild cases, it is between 0.5 and 0.7. ʹ݈݄݀ݏ݁ݎ݄ݐ should be below 0.5 for cases with small melasma lesion patches and above 0.5 otherwise. C. Result and Discussion Some co mparisons of our proposed method with oHybrid segmentation method are presented below. Fig. 6 shows the segmentation results of a malar case by using the proposed method and the oHybrid segmentation method [3].
Fig. 4 Final segmentation result of forehead case
(a) (b) Fig. 7 Result comparison for forehead region between (a) proposed method and (b) oHybrid method (a) (b) Fig. 6 Result comparison for malar region between (a) proposed method and (b) oHybrid method
By co mparing the two results, it is noted that the proposed method achieves comparable accuracy with oHybrid segmentation method. For the result of proposed method shown in Fig. 6(a), the false positively classified region circled by green ellipse comes fro m the bias towards large patches in local entropy thresholding and the false negatively classified region in blue rectangular contains relatively mild melas ma lesion which is not detected. The other regions are segmented out accurately. In comparison, in Fig. 6(b), oHybrid segmentation method has a bit over-segmentation problem which is reflected by false positively classified areas circled by green ellipses. The malar cases are normally less influenced by the illu mination problem and both the methods can achieve high accuracy of segmentation. While the forehead cases are more complicated because of the more severe illu mination reflect ion problem. The proposed method can achieve superior performance to the oHybrid method and results are discussed as following. Fig. 7 shows the results fro m the proposed method and oHybrid segmentation method which are applied to a forehead image. It is observed that the illu mination reflect ion in the middle area of the image is quite severe and this makes an accurate segmentation a difficult task. Ho wever, the result fro m the proposed algorith m produces highly pro mising results as shown in Fig. 7(a). Most of the melasma reg ions are segmented out without any false positive classifications. Only a small melasma reg ion is misclassified as normal skin, which is highlighted by the blue rectangular. In contrast, the performance of the oHybrid method is much worse as shown in Fig. 7(b) as the misclassificat ion rate is very high and most parts of melasma reg ions are not identified within the red ellipse. There are also some false positive errors at some wrinkle areas due to shading effect. The above e xperimental results demonstrate that for malar case, our proposed method can achieve comparable results with the recent o Hybrid seg mentation method in [3]. In particular, for the forehead case, our proposed method is superior to the oHybrid method.
Another difference between the proposed method and the oHybrid method is that a patch of normal skin region needs to be manually selected as prior informat ion in oHybrid method. Different selections could result in highly different segmentation results. In contrast, the proposed method does not require any input. Therefore, the proposed method is more robust. IV. Conclusion In this paper, a new image segmentation method utilizing reaction-diffusion based level set model (RDLSM) and local entropy thresholding is proposed for melas ma p ig mentary area segmentation. Contrast limited adaptive histogram equalization and single scale ret inex theory are emp loyed to enhance image contrast and normalize illu mination based on suitable color spaces. After applying RDLSM for segmentation, local entropy thresholding is proposed to address the over-segmentation issue thus filter out desired boundaries and eliminate misclassified areas. Co mparable results are achieved using the proposed method with the recent oHybrid segmentation method [3]. Moreover, it successfully solves the heavily biased illu mination problem for the forehead case, which remains an obstacle for segmentation methods proposed in other works. The p roposed method showed that it is promising to apply level set method for melasma image segmentation. Our future work is to achieve fully automatic to exclude subjectivity as well as to balance toward small patches in local entropy thresholding. A CKNOWLEDGMENT We wish to acknowledge the funding for th is project fro m Nanyang Technological Un iversity under the Undergraduate Research Experience on CAmpus (URECA) programme. REFERENCES [1]
[2]
H. H. Ghorbel, F. Boukari, E. Fontas, H. Montaudié, P. Bahadoran, J.-P. Lacour, et al., "Copper Bro mide Laser vs Triple -Co mbination Cream for the Treat ment of Melasma: A Randomized Clinical Trial," JAMA dermatology, vol. 151, pp. 791-792, 2015. Y. Liang, Z. Lin, J. Gu, W. Ser, F. Lin, E. Y. Tay, et al., "Melasma image segmentation using extreme learning machine," in Proceedings of ELM-2014 Volume 2, ed: Springer, 2015, pp. 369-377.
[3]
[4]
[5]
[6]
[7]
[8]
Y. Liang, L. Sun, W. Ser, F. Lin, E. Y. Tay, E. Y. Gan, et al., "Hybrid threshold optimizat ion between global image and local reg ions in image segmentation for melasma severity assessment," Multidimensional Systems and Signal Processing, pp. 1-18, 2015. C. Li, C. Xu, C. Gu i, and M. D. Fo x, "Distance regularized level set evolution and its application to image segmentation," Image Processing, IEEE Transactions on, vol. 19, pp. 3243-3254, 2010. T. F. Chen, "Medical image segmentation using level sets," Technical Report. Canada, University of Waterloo, 2008. W. Wen, C. He, Y. Zhang, and Z. Fang, "A novel method for image segmentation using reaction– diffusion model," Multidimensional Systems and Signal Processing, pp. 1-21, 2015. M. Droske, B. Meyer, M. Ru mpf, and C. Schaller, "An adaptive level set method for medical image segmentation," chapter in Information Processing in Medical Imaging, Springer Lecture Notes in Computer Sciences, 2001, pp. 416-422. F. Dong, Z. Chen, and J. Wang, "A new level set method for inho mogeneous image segmentation," Image and Vision Computing, vol. 31, pp. 809-822, 2013.
[9]
[10]
[11]
[12]
[13]
[14]
A. R. Robertson, "The CIE 1976 Colo rϋDifference Formulae," Color Research & Application, vol. 2, pp. 7-11, 1977. C. K. Kimb rough-Green, C. E. Griffiths, L. J. Finkel, T. A. Hamilton, S. M . Bu lengo-Ransby, C. N. Ellis, et al., "Topical retinoic acid (tretino in) for melas ma in black patients: a vehicle -controlled clin ical trial," Archives of Dermatology, vol. 130, p. 727, 1994. K. Zuiderveld, "Contrast limited adaptive histogram equalization," in Graphics gems IV, 1994, pp. 474485. D. J. Jobson, Z.-u. Rah man, and G. A. Woodell, "Properties and performance of a center/surround retinex," Image Processing, IEEE Transactions on, vol. 6, pp. 451-462, 1997. C. Yan, N. Sang, and T. Zhang, " Local entropy-based transition region extract ion and thresholding," Pattern Recognition Letters, vol. 24, pp. 2935-2941, 2003. H. J. Kim, S. Lim, S. Jang, B. Kim, and E. S. Jung, "Interpolation method identification fro m resampled images," in Proc. of International Workshop on Image Analysis for Multimedia Interactive Services, Delft, Netherlands, 2011.
