Skin Color Segmentation Using Adaptive PCA and ... - IEEE Xplore

ICACSIS 2011

ISBN: 978-979-1421-11-9

Skin Color Segmentation Using Adaptive PCA and Modified Elliptic Boundary Model Noha A.Hikal1, Roumen Kountchev2 FCIS, Information Technology Department Mansoura University, Mansoura, Egypt 2 FCTTm Radiocom Department Technical University, K1.Ohridsk 8, Sofia, Bulgaria 1

E-mail: [email protected], [email protected]

among people and especially among people from different countries. Moreover, researchers claim that skin chromaticity is roughly invariant among different color formats [3]. Regarding skin color degradation, luminance introduces many problems, whereas chromaticity includes useful information. Therefore, skin color detection is possible and successful by using proper color spaces that separate luminance from chromaticity components. The previous skin detection methods are pixel-based classifiers, they aim to introduce a tool to measure the distance of each pixel color to skin color tones. These techniques are vulnerable; hence the color itself can be represented in many different formats. Also, these models show weak performances in the presence of changing lighting or imaging conditions [4]. A survey of skincolor modeling and detection methods is introduced in [3]. Different algorithms of skin color segmentation in different color spaces were discussed. Also, different supervised and unsupervised classifiers were introduced and discussed. The survey reported the more critical issues about skin color segmentation which are:

Abstract—Skin color segmentation is a promising research field in computer vision. It has a wide variety of applications such as: development of more effective interfaces for human-computer interaction (HCI), face detection, contentbased image retrieval (CBIR) systems, gesture analysis, and medical applications. However, the skin color detection is a challenging task as the skin color in an image is sensitive to various factors like illumination, camera characteristics, and ethnicity. This paper introduces a newly developed method of human skin color detecting and segmentation under different challenges. The skin colors are first fully decorellated using the adaptive color PCA (ACPCA) and then segmented using the modified elliptic boundary model (MEBM). It can be observed from the experimental results that the proposed method successes in skin color detecting and segmentation for different ethnic groups and imaging conditions. In comparison with other famous color models: the dedicated detection rate is about 98% and the false positive rate is 4% on an average. The advantages of the new method for skin color segmentation are the higher accuracy, relatively low computational complexity. Also, it does not require a priori information about the number of objects in the image or applying any post-processing morphological operations for the segmented areas.

 The choice of color space should depend on the available image format.

INTRODUCTION

 Dropping the intensity component so as to obtain robust parameters against illumination conditions as approached in many of the works actually reduces the skin detection performance.

color segmentation is a promising research field SKIN in computer vision. Its most appealing applications are: development of more effective and friendly interfaces for human-computer interaction (HCI), face detection, faces tracking, content-based image retrieval (CBIR) systems and gesture analysis. Moreover, it can be used for medical applications. Last but not least, it can be used as a biometric feature in mobile security problems [1, 2]. Many researchers investigated skin color segmentation through many different ways concluding that it is a complex problem. All of them tend to classify the pixels of the input image into skin color and non-skin color clusters by using a thresholding technique that exploits the information of a skin color distribution map in the different color spaces. However, skin color varies quite dramatically. It is vulnerable to changing lighting conditions and it differs

 To improve the accuracy of the classifier, various other features such as shape, spatial and motion information can be used along with skin-color information  When classifier methods are used in real-time, huge computational and storage requirements are required, since skin detection could be a first stage in face detection machine. Concluding that; obtaining robust color representations against varied illumination conditions is still a major problem. Moreover, to build an accurate classifier which can detect all the skin types under

407

ICACSIS 2011

ISBN: 978-979-1421-11-9

different illuminations, ethnicity, shadows, complex backgrounds and makeup is still an unsolved problem. In [4], an adaptive skin color model based on the Gaussian Mixture Model (GMM) to handle the changing conditions was introduced. Initial estimation of the number and weights of skin color clusters are obtained using a modified form of the general expectation maximization changes in imaging algorithm. The model adapts to conditions and refines the model parameters dynamically using spatial and temporal constraints. The initial classification is based on large data sets of the ethnic characteristics of the skin color values while the adaptations are due to changing imaging conditions. Although GMMs have been a popular choice for skincolor segmentation, the initialization and iterative training of the GMM are computationally expensive especially with large data sets. Also, the mixture model is slower to use during classification since all the Gaussians must be evaluated in computing the probability of a single color value [3]. This paper has focused on the problem of developing an accurate and robust model for the human skin detection and segmentation. The new adaptive skin color detection and segmentation method is based on employing adaptive color Principal Component Analysis (PCA), or KarhunenLoeve Transform (KLT) [6], and a modified version of the elliptic boundary model (EBM) [7], respectively. The adaptive color PCA (ACPCA) fully decorrelates the color vectors of an image. Moreover, the adaptation property of the transform gives the transformed coefficients the flexibility to fully adapt to the local statistics of the specified color vector. The elliptic boundary model is a possible way to detect an object with preset dominant color. This model comprises vectors with Gaussian distribution of the probability density, which define the colors of the pixels placed on the visible surface of the object. Unlike previous methods, the proposed method provides the following advantages: (i) it doesn't associate with any special color spaces or lighting conditions; (ii) higher accuracy, because the ellipsoid approximates the cluster of vectors more precisely than other famous color models (Single Gaussian models and Gaussian mixture models [3]) (iii) it has a relatively low computational complexity. The new approach permits reliable object detection and identification in various positions, lighting conditions and viewpoints. This paper is organized as follows: Section 2 describes the ACPCA. Section 3 describes the methodology of detecting and segmenting an object with preset dominant color based on the modified elliptic boundary model with the ACPCA. Experimental results are introduced and discussed in Section 4. Finally, conclusion is given in Section 5.

new modification in the PCA, in which the transformed matrix coefficients are not fixed; they are fully adapted to the local statistics of the specified pixel colors. This, in turn, lowers the computation complexity of the PCA. 

Let C  C1 , C2 , C3  t is the color vector in the 3D coordinate system C1 , C2 , C3 . In every color RGB image of size MN pixels, the color vector of the sth pixel is  Cs  Rs , Gs , Bs  t for s = 1,2,..,S (S = МN – number of pixels). Unlike deterministic color space transforms C1s , C2s , C3s (for example X,Y,Z, YCrCb, Lab, HSI,



HSV, etc.) in the case when vectors Cs are processed with the PCA [6] their components C1s , C2s , C3s are fully decorrelated. Besides, the basic part of the transformed  vectors energy Ls  L1s , L2s , L3s  t is concentrated mainly in the first

L1s , and less – in the second component ( L2s

and L3s respectively). Here [] is a PCA matrix of size 3×3, whose elements ij are defined by the eigenvectors  of the covariance matrix [KC] of the vectors Cs . The basic disadvantage of PCA, which restricts its practical application, is the higher computational complexity in comparison with the renowned deterministic color transforms. In the approach, offered by Ionita and Corcoran [8], for the color segmentation of human faces is used PCA approximation with a matrix [] of fixed coefficients. In this case the KLT is of relatively low computational complexity, but it is not fully adapted to the local statistics of the face colors. In order to make the computational complexity of the color PCA lower is used the so-called Adaptive Color PCA (ACPCA) [6], in which the transform matrix coefficients are not fixed. In this case, the direct ACKLT



of the vectors Cs  R s , Gs , Bs  is defined by the t

relation:

 L1s   11 12  L     22  2s   21  L 3s   31  32

13   23    33 

R s  R    G s  G   Bs  B 

(1)

where R  E(Rs ), G  E(Gs ), B  E(Bs ),

( x  E( x s ) 

1 S  x s - mean value of xs); S s1

m1 Am/Pm; m2 Bm /Pm; m3 Dm /Pm for m = 1,2,3 The parameters of (2) can be calculated as:

Am (k3  m )[k5 (k2  m )  k4k6 ],

Bm (k3  m )[k6 (k1  m )  k4k5 ],

II. ADAPTIVE COLOR PRINCIPAL COMPONENT ANALYSIS (ACPCA)

Dm k6[2k4k5  k6 (k1  m )]  k52 (k2  m ),

Unlike different color spaces transforms, the adaptive color PCA transform the color skin vectors into concentrated energy clusters which are then used to train the elliptic model. The word "adaptive" comes from the

Pm  A2m  B2m  D2m ;

408

(2)

ICACSIS 2011

ISBN: 978-979-1421-11-9

k1 E( R s2 ) ( R )2 , k 4  E( R sG s )  ( R )( G ), k 2  E(Gs2 )  (G )2, k 5 E( R s Bs )  ( R )( B),

 1 T  C   Cs , T s1

 1 T   C   f sCs , S i 1

k 3 E( Bs2 ) - ( B)2; k 6  E(G s Bs )  ( G )( B); p

1 2

3

3 2

  a cos  ; 2 2 3 3 p 3

p 3

   cos  3

 a  ;  3

q  2(a/ 3) 3 (ab)/3  c, p  - (a 2 /3)  b;



 arccos  q / 2



a   (k1 k 2  k 3); b  k1k 2  k1k 3 k 2 k 3 (k24  k52  k 62); The inverse ACKLT for s = 1,2,…,S is:

 R s  11  21  31   L1s   R    G     22  32   L2s   G ,  s   12    Bs  13  23  33   L3s   B 

are classified in two classes, as belonging to the object with the chosen color, and all others. In order to eliminate the influence of the object illumination is necessary to

(4)



normalize the color vectors Cs , regarding their module 0    for s = 1,2,..,T are the Cs . Here, Cs  Cs / Cs

Where the matrix of the Inverse ACKLT

11  21  31      1  t   ,  ,  . 1 2 3  12 22 32  13  23  33  



normalized color vectors in the training set. In this work is apply the direct ACPCA (4) on the color  vectors Cs   R s , Gs , Bs  t for

(5)

s=1,2,..,T (T T), fs is the number of samples of the color  vector Cs , S is the total number of samples in the training   set and  C is the mean of the color vectors Cs in the

(3)

    a cos  ;  3  3

3

[ C] 

(11)

1 ,  2 ,  3 , are dеfined in accordance with Eq. 3. Taking into account Eq. 11, the EBM (Eq. 9) is then transformed as follows:

where the eigenvalues

(6)

  where C is the color vector, C  [C1, C2 , C3 ]t and

     ( Ls )  [ L s L ]t [ L ]1[ L s  L ] 

[C] are the model parameters defined as:

( Ls1L1 ) 2 1( Ls 2 L 2 ) 2 2 ( Ls 3 L3 ) 2 3     D 2 ( Ls , L ).

409

(12)

ICACSIS 2011

ISBN: 978-979-1421-11-9

 The function  ( L s ) defines the quadratic Mahalanobis  





distance D2 (Ls ,L ) between vectors Ls and L . If  ( Ls )  1 the relation (12) is the canonic equation of a triaxial ellipsoid in Cartesian coordinates ( L1 , L2 , L3 ) and the semi-axes are of lengths 1 ,  2 ,  3 , into which center with coordinates (L1, L2, L3) is placed  the vertex of the mean vector L . In accordance with: 2

 L   0    si L i   1   i  i 1  3

(13)

each pixel s is classified to certain area in the color space in or out of the ellipsoid, defined by (13).

Fig. 2. Skin color samples in Fig. 1 plotted in RGB space.

IV. EXPERIMENTAL PART

TABLE 2 RESPECTIVE ACPCA-EBM OF SKIN COLORS GIVEN IN TABLE 1.

Regarding skin colors degradation, it typically includes whitish, brownish, yellowish and darkish skins, indoor and outdoor images with varying lighting conditions. Representative skin colors are shown in Fig. 1, respective skin colors in RGB are shown in Table 1. The skin color samples plotted in RGB cube as blue points in 3-D space as seen in Fig. 2. It is to be noted that these are few representative skin pixels from the skin color space which is approximately 0.25% of the entire RGB space [11]. Table 2 represents the model parameters of the skin color samples of Table 1 respectively. Moreover, Fig. 3 represents their plot in 3-D space cube as blue points. It can be noticed from Fig. 3 that the proposed techniques is able to concentrate the energy of the different skin color grades into very small and close coefficient.

[C1, C 2 , C3 ]t

[C1, C 2 , C3 ]t

[C1, C 2 , C3 ]t

[0.0268,

[0.0265,

[0.0275,

-0.0379,-0.037]

-0.0392,-0.036]

-0.0377,-0.038]

[0.0242,

[0.0261,

[0.0275,

-0.0382,-0.033]

-0.0367,-0.036]

-0.0377,-0.038]

[0.0268,

[0.0299,

-0.0381.-0.037]

[0.0293,-0.0402,0.040]

[0.0252,

[0.0280,

[0.0291,

-0.0388,-0.035]

-0.0387,-0.038]

-0.0398,-0.039]

[0.0205,

[0.0286,-0.0392,0.039]

[0.0291,

[0.0304,-0.0415,0.042]

[0.0294,

[0.0281,-0.0388,0.039]

[0.0279,-0.0382,0.038]

[0.0276,-0.0383,0.038]

[0.0280,

-0.0355,-0.035] [0.0235,

[0.0284,

[0.0278,

-0.0351,-0.033]

-0.0389,-0.039]

-0.0381,-0.038]

-0.0322,-0.032] [0.0269, -0.0389,-0.037] [0.0309, -0.0422,-0.042] [0.0254,

-0.0408,-0.041]

-0.0398,-0.040] -0.0402,-0.040]

-0.0383,-0.038]

 1

Fig. 1. Typical skin colors from different ethnic groups and skin.

TABLE 1 RESPECTIVE SKIN COLORS IN RGB SPACE FOR THE COLORS IN FIG. 1 RGB RGB RGB (119, 79, 53) (218, 160, 172) (254, 203, 178) (88, 59, 55) (166, 104, 89) (249, 207, 183) (82, 73, 68) (131, 106, 86) (197, 173, 149) (204, 150, 103) (252, 218, 191) (253, 220, 213) (127, 41, 44) (235, 185, 152) (246, 215, 197) (119, 78, 58) (226, 165, 137) (243, 214, 200) (173, 98, 67) (240, 188, 151) (248, 237, 235) (150, 91, 57) (195, 136, 106) (238, 209, 91) (124, 48, 16) (211, 150, 95) (220, 194, 159)

 2

 3

Fig. 3. Plot of skin color samples, Fig. 1, in

   1, 2 , 3

space of the

ACPCA.

The data used for testing in our experiments are web images of people collected from different ethnic groups as

410

ICACSIS 2011

ISBN: 978-979-1421-11-9

shown in Fig.4. For training, skin pixels were manually segmented. The training data have been selected to cover almost all ethnic groups and imaging conditions. For testing, the test images were applied to the proposed method. The software tool developed in MATLAB 2009, no post processing or preprocessing was needed. The skin color segmentation results for Fig.4 are depicted in Fig. 5 respectively. For example, the image "Girl", Fig.4-f, of size 315×237 pixels; the parameters of modified ЕВМ are: 1. Eigenvalues and eigenvectors:

c.

d.

1 =0.007;  2 =1.57.10-5; 3 =7.86.10-7,  1 = [0.5734, 0.5749, 0.5836]t;   2 = [-0.7344, 0.0450, 0.6772]t;  3 = [-0.3631, 0.8170, -0.4480]t. 2. Vector-cetroid of color cluster:

 L = [0.0100, - 0.0139, - 0.0138].

3. Тhresholds are within

0 = 0.0149 and

e.

1 = 0.0160. Corresponding classification rule: рixel(s)skin-color, if

f.

Fig. 4. Images used for training and testing.

3

0 [(Lsi Li )2 /  i ]  1 i 1

for s = 1, 2, . . , S (S = 74655). Fig.6 shows a sequence of frames under variant illumination, the parts marked with red circles in the video sequence to show the effect of changing illumination. The result of skin segmentation of Fig.6 is shown in Fig. 7. The skin segmented image here shows the success of this approach under variant illumination conditions. The whiter background images, Fig. 4. b, c and Fig. 6, highlight one problem with color based skin model, that the colors which appears similar to skin colors palettes shown in Fig.1, (here whitish skin like pixels on the background) can't be segmented efficiently. But, in comparison with the literatures [4, 10], it is still a future work problem in most of the researches.

a.

b.

c.

e. a.

d.

f.

b. Fig.5. The results of skin color segmentation for Fig.4 based on the proposed method.

411

ICACSIS 2011

ISBN: 978-979-1421-11-9 the cluster of vectors more precisely than other famous color models and it has relatively low computational complexity. The new approach permits reliable object detection and identification in various positions, lighting conditions and viewpoints. Also, it does not require a priori information about the number of objects in the image or any post processing morphological operations for object segmentation. The method is tested and evaluated for several images and its performance was found satisfactory in terms of processing time, since the proposed method doesn't need learning phase. The CPU execution time needed for skin detection and segmentation based on the proposed method is function of image size. Generally, the processing time is measured in seconds. Some results highlight one problem with the proposed model, that the colors which appears similar to skin colors palette, whitish skin or reddish like pixels, can't be successfully detected. Moreover, it increases the FPR value. For future study, the problem of segmenting non-skin objects that have colors appear similar to skin colors palette will be considered. Moreover, the proper selection of optimum threshold that varies from image to image could be included in future study which will enable us to perform fully automatic skin segmentation with reduced FPR.

Fig. 6. A video sequence with variant illuminations.

Fig. 7. Skin color segmentation using the proposed method.

For quantitative analysis of the proposed method, introduced in Table 3, we use False Positive Rate (FPR) and Detection Rate (DR) [11]. These are calculated using pixel by pixel comparison of perfectly skin color pixels, Table 1, with the segmented images based on the proposed method. The mathematical expression as follows: 𝐹𝑃𝑅 = 𝑁𝑜. 𝑜𝑓 𝑛𝑜𝑛 − 𝑠𝑘𝑖𝑛 𝑝𝑖𝑥𝑒𝑙𝑠 𝑐𝑙𝑎𝑠𝑠𝑖𝑓𝑖𝑒𝑑 𝑎𝑠 𝑠𝑘𝑖𝑛 = 𝑇𝑜𝑡𝑎𝑙 𝑁𝑜. 𝑜𝑓 𝑛𝑜𝑛 − 𝑠𝑘𝑖𝑛 𝑝𝑖𝑥𝑒𝑙𝑠 𝐷𝑅 ==

REFERENCES [1] G. Schaefer, M. Rajab, M. Emre, H. Iyatomi, "Colour and Contrast Enhancement for Improved Skin Lesion Segmentation" ELSEVIER, Journal of Computerized Medical Imaging and Graphics 35(2011). pp.99-104 [2] D. O’Mara, "Automated facial metrology" Ph. D. Thesis, Department of Computer Science and Software Engineering, University of Western Australia, 2002. [3] P. Kakumanu, S. Makrogiannis, N. Bourbakis, "A survey of skincolor modeling and detection methods". ELSEVIER Journal of the Pattern Recognition Society, 40, 2007, pp. 1106 – 1122. [4] R. Hassanpour, A. Shahbahrami, and St. Wong. " Adaptive Gaussian Mixture Model for Skin Color Segmentation". World Academy of Science, Engineering and Technology, 41, 2008, pp. 1-6. [5] R. Dony, Karhunen-Loève Transform, Book chapter in "The transform and data compression handbook", K. Rao, P. Yip, CRC Press LLC, 2001. [6] R. Kountchev, R. Kountcheva, Image color space transform with enhanced KLT, Book chapter in: "New Advances in Intelligent Decision Technologies", K. Nakamatsu, G. Wren, L. Jain, R. Howlett . Springer-Verlag, Berlin, Heidelberg, 2009, pp. 171-182. [7] J. Lee, S. Yoo, "An elliptical boundary model for skin color detection" Proc. of the International Conference on Imaging Science, Systems and Technology, 2002. [8] M. Ionita, P. Corcoran, "Benefits of using decorrelated color information for face segmentation/tracking" Advances in optical technologies, Hindawi Publishing Corporation, 2008, ID 583687. [9] K. S. Deshmukh, G. N. Shinde, "An Adaptive Color Image Segmentation". Electronic Letters on Computer Vision and Image Analysis. 5(4) 2005, pp.12-23. [10] S. Phung, A. Bouzerdoum, D. Chai, "Skin Segmentation Using Color Pixel Classification: Analysis and Comparison". IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 27, No. 1, Jan. 2005. pp.148-154. [11] K. K. Bhoyar, O. G. Kakade, "Skin Color Detection Model Using Neural Networks and its performance evaluation" Journal of computer science 6 (9). 2010. pp. 963-968

(14)

𝑁𝑜. 𝑜𝑓 𝑠𝑘𝑖𝑛 𝑝𝑖𝑥𝑒𝑙𝑠 𝑐𝑜𝑟𝑟𝑒𝑐𝑡𝑙𝑦 𝑐𝑙𝑎𝑠𝑠𝑖𝑓𝑖𝑒𝑑 𝑇𝑜𝑡𝑎𝑙 𝑁𝑜. 𝑜𝑓 𝑠𝑘𝑖𝑛 𝑝𝑖𝑥𝑒𝑙𝑠

(15)

TABLE 3 QUALITATIVE ANALYSIS OF THE PROPOSED METHOD. Image

DR%

FPR%

Fig. 1.a Fig. 1.b Fig. 1.c Fig. 1.d Fig. 1.e Fig. 1.f

99.50 97.00 99.70 96.30 99.86 98.50

1.3 6.5 2.2 7.3 3.2 5.4

Regarding the previous experimental results, it can be noted that; the proposed method successes in skin color detecting and segmentation for different ethnic groups and imaging conditions. In comparison with other famous color models: Single Gaussian models (SGM), Gaussian mixture models (GMM) and other classifiers based on histogram analysis or ANN for human skin segmentation [4, 9, 10, 11], the advantages of the new method for skin color segmentation are the higher accuracy, because the hyperboloid approximates

412