A Review of Methods for Face Verification under ... - CiteSeerX

A Review of Methods for Face Verification under Illumination Variation Mehran Emadi 1, Farhad Navabifar2, Marzuki Khalid 3, Rubiyah Yusof 4 Faculty of Computer Engineering of Islamic Azad University Mobarakeh Branch, Easfahan , Iran 1,2,3,4 Center for Artificial Intelligence and Robotics University Technology Malaysia, International Campus, Jalan Semarak, 54100, Kuala Lumpur, Malaysia [email protected] 1,2

Abstract - Face identification has many successful applications in image processing. Several problems such as varying illumination, pose, expression and occlusion exist in practical face identification that researchers should be solved. The purpose of this paper is to present a review of face verification under illumination variation methods and to investigate advantages and disadvantages of them. Keywords-face illumination

1

identification;

face

verification;

INTRODUCTION

Face recognition is one of the most popularly used biometric techniques. One of the main reasons is due to the non invasive nature of the recognition system. The ability to enroll static images, ability to leverage existing surveillance hardware such as CCTV , the passiveness in the usage such as no direct user contact or cooperation are required, are some of the factors that contributes to the popularity of the face recognition system. However, while this technology has many merits, it also has a number of limitation and challenges that must be addressed before it can be used widely. To highlight a few are: the potential for abuse of privacy rights due to its ability to enroll without active user involvement, the requirement of a controlled environment with suitable lighting or background environment, and very much dependent on the camera angle. Moreover, disguises and changes in facial characteristic can affect accuracy [28]. Various approaches have been developed to cater for automatic face recognition problems in a way of improving face recognition accuracy. In 1990s, automatic face recognition technology moved from laboratory to the commercial world largely because of the rapid development of the technology, and now many applications use this technology [3]. These applications include everything from controlling access to secure areas to verifying the identity on a passport. Face Recognition System can operate in two modes: face verification (authentication) and face identification (recognition). In face verification, the matching technique is one-to-one match that compares a query of faces images against a template face image whose identity is being claim. In face identification or recognition, the matching technique is one-to-many matches that compare a query of face image against all the template images in the database to determine the identity of the query faces.

The variation in two face images of the same person may originate due to changes in photometric and geometric characteristics of the environs affect image appearance. Geometric characteristics relate to the geometry of the capturing device with respect to the face being captured including distance and orientation and pose. Photometric characteristics report the illumination conditions like number, size, intensity, color, placement, etc. of light sources. Clearly, changing lighting can produce very dark or too bright images, which can disturb the recognition and verification process. It is well known that the performance of face identification systems drops highly when the illumination is present within the input images. Furthermore, the task of face verification or recognition is very difficult when pose and illumination variations happen together [1,2]. Illumination is the most important factor that is efficient to change the appearance of face. Adjacent lighting extremely changes within and between days and among indoor and outdoor environments. Due to the 3D structure of the face, strong shadows that accentuate or diminish certain facial features can be cast by a direct lighting source. It has been proving experimentally [10] and theoretically those differences in appearance caused by illumination are bigger than the difference between individuals. In other words, the difference between two face images of the same individual captured under illumination variation is larger than the difference between any two face images captured under same lighting conditions. Because dealing with variation lighting is a central topic in computer vision, several approaches for face identification under illumination variation have been proposed. In a face identification system, there are three different stages for compensation illumination variation: during the pre-processing, the feature extraction and the classification. However, the feature extraction phase does not seem to overcome with variation lighting. In fact, Adini et al [10] experimentally, presented that classical image representation such as edge maps, conjugates of the gray level as well as the image filtered with 2D Gabor-like functions are not enough for recognition under wide range of illumination variation. In the recent years, researches have been studying two main different approaches in order to circumvent the problems induced by illumination variations: the first approach attempts to change the input image, to turn it into in a more appropriate representation for recognition purpose. Therefore, this approach deals with pre-processing only, and any kind of classifier is not used. The second

approach tries to model the object of interest across all possible lighting conditions. Therefore, classification is necessary in this approach [31]. This paper provides a review on face verification and recognition under illumination. The existing methods are comprehensively reviewed and explained. Their methodologies, advantages/ disadvantages are explained.

1.1

Face identification

Nowadays, extension usage of biometrics is caused that researchers attend to improvement of these systems specially face recognition and verification systems. Although face identification systems do not have accuracy as same as another system, for example, finger print and iris detection but face identification systems for some reasons such as friendly, acceptability, nonintrusive, universality, and easy to use, have a high position in biometric systems. Generally, a face identification system as it is shown in Figure1.It consists of three main modules that are: 1- Image acquisition and face alignment, 2- Feature extraction, 3- Decision making. Image acquisition (Detection and Alignment)

Feature extraction

conclusion and Future direction for methods to overcome illumination problems is given in Section 3.

2

Illumination variation review

Researchers in image processing area have been proposed many face recognition methods and gotten satisfactory performance in the frontal position of a face and normal illumination conditions since now but the task of robust face identification is still hard in real condition. Dealing with changing lighting is very difficult because sometimes the changes which are influenced by illumination are bigger than the varieties between individuals. In comparing pose and illumination variation, illumination makes a bigger variation in face images than pose.

Decision making

Figure1. Face identification flow As we can understand from Figure1 firstly, system detects to face from input image. In face detection, position of face in an image without statues of face will be specified. In stage of image acquisition some registration and image preprocessing will be done. In image preprocessing, geometric and photometric normalization will be processed and faces in different pose and illumination will change in frontal and normal lighting condition. Geometric normalization replaces situation of some important factors of face such as mouth, nose, and eyes. This stage is one of the most difficult phases of geometric normalization because in several images, some part of face perhaps occulted. In different researches linear and nonlinear conversion is used for geometric normalization. These methods are founded on position of special point of face such as eyes. Photometric normalization reduces effects of illumination variation. In illumination variation, intensity or size or orientation of lighting or all of them will be changed. Illumination variation has valid influence on face identification[4]. Produce a database that simulated all illumination variations and doing some preprocessing toward delete lighting variation and normalization of input images, are two methods, which can use for illumination normalization and lighting compensation. Feature extraction is a special form of dimensionality reduction. The amount of resources required to describe a large number of data accurately will be involved in feature extraction. The remainder of this paper is organized as follows. In Section 2, we will review different methods of face identification under illumination variation. Finally, a

Figure2. Examples of the illumination variations from BANCA database As it can be understood from Figure.2, the appearance of a face in the image plane will be altered far right picture is similar to besides the image and is variant with left first image. Light will be reflected from a surface because of Lambertian reflectance and specular reflectance. Lambertian reflectance is due to reflectance of light dropped on a surface in all directions. When the surface moves similar to a mirror and reflects the light from the surface in a single direction, specular reflectance is occurred. Shading and specular reflections are two affects of illumination which due to those types of reflection. When light descends on a surface with changing gradient, shading will be accrued. Regions of brightness on the surface are due to specular reflections. Shadows happen when a light source is occluded. In a face picture, a shadow is an area of reduced brightness with a sharp edge. Along with the development of human face identification techniques, a relatively large number of face databases have been gathered. However, many of these databases are created for particular needs of the algorithm under development or desire [28]. Table 1gives a review of the recording conditions of some databases. The task of illumination invariant face verification is the effect of brightness change in image capture cannot be easily separated from the important sensitive information between face images captured under equivalent lightly conditions [4]. Different approaches exist for solving

illumination variation that can be divided into four categories which in the following these are summarized. These categories are: heuristic method, image comparison methods, class based methods, and 3D model based approaches.

2.1 Heuristic methods Some investigators have been proposed that illumination variation can be diminished by throwing away the three most significant components. Belhumeur et al [5-6] suggested the 3D linear subspace technique for compensation illumination variation. To start, two simplifying assumptions were made by them: firstly, they presumed that the objects have Lambertian reflectance functions; secondly, they assumed that the shape of an object’s surface in convex. They used Fisher’s Linear Discriminant (FLD) to maximize the ratio of betweenclass scatter to that of within-class scatter. In their method, the first few components were discarded, and they got satisfactory results under different lighting conditions. The base of their algorithms was on this reality which these first principal components entrap only variation due to lighting, but sometimes very important information was wasted. They showed that Fisherface method attends to be best at interpolating and extrapolating over variation in lighting. Furthermore, the performance of the Eigenface method in the attendance of illumination variation was improved but the error rates did not achieve as low as some of the other methods. They used 30 images from Harvard Robotics Laboratory database and 36 images from Yale database. The performance of their method was %47.4 [5-6]. Bartlettand et al [7] improved an algorithm for dividing the statistically components of a data set through unsupervised learning. That algorithm has turned out successful in separating randomly mixed auditory signals. Two architectures used Independent component analysis (ICA) for representing face images for face recognition. The first architecture was used to execute a blind separation of a mixture auditory signal and FMRI data also they applied this architecture to find a set of statistically independent source images for a set of face images. The second architecture was applied to find image filters that produced statistically independent outputs from natural scenes. Furthermore, they employed ICA under this architecture to get a factorial code for the face images. FERET database was used to test performance of face recognition. Images of 425 individuals in four frontal views were used for face recognition. Nearest neighbor method was applied for evaluating. The performance of their method was %48 [7]. Zhao [8] used symmetry method in heuristic technique to get better performance under illumination variation. He used symmetry shape-from-shading (SFS) algorithm as a tool to get a prototype image which is illumination- normalized. It is seen that, symmetric SFS algorithm has a unique point- wise solution. In that time, symmetry had been more accurate than existing approach to develop a new model-base source-from-shading algorithm. The prototype images were used to construct an illumination-invariant measure and in existing PCA

and subspace LDA face identification systems. Three experiments have done by him. In first experiment, in order to improve recognition performance, he used a new illumination-invariant measure and compared this measure with previous works. In second experiment, he showed that using the rendered prototype images instead of original images can significantly improve face recognition method such as PCA and LDA. Finally, in third experiment, he presented that the generalized recognition rate of subspace LDA can be greatly enhanced. 1038 images from FERET and 495 images from Yale and Weizmann databases were applied for his experiments. The performance of his approach was %57 for Yale and %38.5 for Weizmann. The advantages of this method are: 1) this method has a unique solution for albedo. 2) In that time, compared to other photometric stereo algorithms, the registration problem of multiple images had been relieved. The disadvantages of this method are: 1) Using symmetric light-source estimation for each image, got very time. 2) The accuracy of this method relies on light-source estimation. 3) This method is only efficient for exact frontal face images[8]. Clearly, low-frequency band is the place for lying illumination variation. For this reason, Chen et al [9] used Discrete Cosine Transform (DCT) coefficients to reduce an influence of illumination variation. The key idea of using this method was that DCT can truncate lowfrequency band. They tried to identify faces using reflectance characteristic, lighting variations can be decreased by removing low-frequency components of face images. Human’s hair is a kind low-frequency feature. Therefore, for this reason, human’s hair is not regarded as a kind of important facial feature. Image from a spatial domain to a frequency domain was translated by using DCT. They got low-frequency DCT coefficients to zero to remove the low-frequency components of a face image simply. They used the Yale B and CMU PIE face databases to evaluate their proposed approach. These two databases include face images with large lighting variations. They rescaled all the face images to the size of 120*105. In their experiments, they applied the nearest neighbor classifier based on the Euclidean distance for classification. Furthermore, they normalized all the face images so all face images had zero mean and unit variance. 50 principal components were used for the Eigenface method. Clearly, the error rate meaningfully decreased after a few DCT coefficients were discarded. As a matter of fact, illumination variations and facial features are not exactly separated with respect to frequency components. Shadows and specularities lie in the same frequency bands like facial features do. As a result, mainly low- frequency intensity variations of facial feature components have to be offered in order pay back for some illumination variations. The performance of their approach in CMU PIE database was %57 and in Yale B was %48.6. The advantages of their method are: 1) any bootstrap sets and modeling step were needed; 2) Their approach was very fast, and it could be easily implemented in a real-time face identification system. The disadvantage of their technique as mentioned before was that the shadowing and specularity problems were not

perfectly solved [9]. In table 2 the summary of mentioned approaches and their performances are shown.

 Discussion In this section, some representative techniques of human face identification by using Heuristic method have been reviewed with attentions on their performances and accuracies under illumination variation. We summarized the techniques and their performances in Table [1]. The base of these methods is on discarding some first components of face image. In methodology level, Discrete Cosine Transform (DCT) coefficient is more robust to illumination variation. This is because DCT can truncate low-frequency band then lighting variations can be decreased by removing low-frequency components of face images. For these face identification techniques, the problem of these methods is that perhaps some important information of image will be lost when first components of face image are discarded. Their recognition rates are below %60.

2.2 Image comparison method In this category, researchers have been using some filters in order to reduce the influence of the illumination. Adini et al [10] carried out a comparative evaluation of various methods by using different image representations and distances measures. They have been evaluated some image representations such as edge maps, derivatives of the gray level image, 2D Gabor filter, and a representation that combines a log function of the intensity in these representations. Furthermore, they used point-wise distance, log point-wise distance, regional distance and local affine gray level distance for distance measures. They took five frontal photos from 25 persons to make database. The observational results permit to reason out that, these representations are not able to overcome illumination variation a lonely[10]. Nanni and Maio [11] suggested a local base method that attempted to overcome ineffectiveness the previous methods in illumination variation by using Gabor filter + PCA instead of Gray Values +PCA for feature extraction. In order to calculate the scores, they applied Parzen Window Classifier (PWC) in place of the nearest neighbor whom the weight of each local classifier was acquired by a Genetic algorithm. PWC is probabilistic and it is a one-class classifier. They used 1196 individual images from FERET database. The performance of their approach was better than previous approaches. The Equal Error Rate of their approach was %5.81[11]. In a lot of methods, researchers attempt to get a representation of the face as the quotient of the image itself with reference images. Jacobs, Belhumeur and Barsi [12] suggested a method to reduce lighting effects based on the Lambertian reflectance. They proposed a number of manners of attempting to measure whether the distinction between two images is due to a variation in lighting or in object structure. They used a simple measure of the complexity of the ratio image by using the integral of the magnitude of its squared gradient. A publicly available database of faces made by Hallinan

[25] was applied for their experiment. 450 images of 10 individuals were used. They presented that the squared magnitude of the ratio image operates dramatically better than simple correlation, or correlation after projecting onto the twenty principal components of the training images. Clearly, this method did not attempt to compound information from a number of training images to make up as representation of a face, as do techniques as Fisherfaces, the linear subspace method, or the illumination cones’ method. The error rate of their experiment for correlation was %74.7 and for PCA was %76.5 and for Gradint of ratio was %24.7 and for illumination cones was %10[12]. Liu et al [13] suggested a method to overcome the illumination problem. They applied a ratio-image between the face image and reference face image, both of which were blurred by a Gaussian filter to restore a face image captured under arbitrary lighting conditions. Furthermore, they used an iterative algorithm to update the reference image, which was recreated from the restored image by PCA to be visually better restored image. The performance of their method was evaluated by applying the Yale B database and Yale database. Under normal lighting in these two databases, the number of distinct subjects was small, so they applied the face images from the FERET face database to train the eigenspace for image identification. The advantages of their method are: 1) In the training, only a single face under frontal lighting is needed; 2) In this method, the normal’s face surface and light source directions did not need to estimate; 3) In their technique, image warping, was not required to perform. The recognition rate was %50.8 for Yale B database and was %55 when Yale database combined by 24 eigenvectors trained from these databases [13]. Qing, et al [14] suggested a method based on harmonic images. In their technique, canonical lighting was predefined. Firstly, the nine low-frequency components of the illumination from the input images were estimated. Thus these facial images were normalized to the standard illumination by re-rendering them to apply the illumination ratio image method. They approximated the lighting of the original face image and then executed relighting with calculating the ratio of image between the canonical image and the primary image. They used CMUPIE and Yale B databases for their experiment. The error rate comparisons between different canonical illumination and various alignments on the CMU-PIE database were %47.1 and on the Yale B database were %60[14]. Many researchers used different filters to the input image to reduce the influence of illumination variation. Arandjelovic and Cipolla [15] recommended a method based on application of filters in order to reduce illumination variation effects. They proposed an adaptive framework to learn absolutely how similar the new and training illumination condition is, and to emphasize suitably the use of either the raw input image or filtered version. In their work several automatic face recognition algorithms were applied on a large database using (i) raw grayscale input, (ii) a high- pass (HP) filter and (iii) the self-Quotient Image. They used FaceDB100 and FaceDB60 databases, which totally contain 160 individual face images. Their recognition rate for FaceDB100 was%58.3 and for FaceDB60 was %46.6[15].

Savvides, Kumar, and khosla [16] recommended a hybrid approach based on using correlation filters and PCA. They used PCA for gaining the variance in a set of training images and applied correlation filters, which have attractive features to find illumination tolerance. Clearly, this method uses the ability of the correlation filters against illumination and robustness of PCA to capture the variability in the data. One of the best properties of correlation filters is built-in shift invariance, i.e., the correlation output will be shifted when the test input image is shifted. They used Yale B and CMU PIE databases for their experiment. A large improvement was gained in compared with the minimum average correlation energy method (MACE). The recognition rate of their technique was %98.9. [16]. Du and Ward [17] used wavelet transform for illumination normalization. Their method raises the contrast as well as the edges of face images at the same time, in the frequency domain using the wavelet transform, to facilitate face recognition tasks. They divided the input image into its high and low frequency components. Then they retouched various band coefficients separately: low frequency coefficients are multiplied by histogram equalization and a scalar is used for retouching high frequency coefficients then they adapted coefficients by using inverse wavelet transform. They used Yale B database for their experiment. On average, the recognition rate applying the Euclidean distance nearest-neighbor classifier was improved to %95.65 from %80 and the performance of their method for raw images was %56.2[17]. Nanni and Lumini [18] explained an improved algorithm by name RegionBoost. They developed a fast and robust invariant Local Binary Pattern histogram in a face recognition system. They suggested to apply a multiclassifier which each classifier, an AdaBoost of feedforward back-propagation network, was trained applying a single Sub-Window of the whole image; the classifiers were finally mixed using the”Sum Rule”. Their method had four steps: Pre-Processing, Feature Extraction, Feature Selection, Classification and Fusion. They normalized the image in pre-processing method. The normalized image was calculated by using the operation below:

(1) Where P(x,y) shows the pixel value at the coordinate (x,y),m and v be the image mean and variance, respectively. They changed the size of face images of data to be 100*90 because they considered sub-windows of dimension 25*25 and 15*15 taken at steps of 11 pixels then they had a total of 98 SWs for each representation. Furthermore, they made their feature vector by concatenating the features extracted from the invariant LBP histograms of 10 bins, 18 bins and 26 bins of a given SW. They applied Sequential Forward Floating Selection (SFFS) to find the most useful sub-windows of the whole face. An AdaBoost, with 50 iterations of feed-forward back-

propagation network with 140 hidden nodes was used for classification and sum rule was performed for fusion. They used the Notre-Dame database for their tests. This database consists of a total 275 different persons participated in one or more sessions. They applied 198 individuals’ images from this database for their test. The Equal Error Rate (EER) of their experiment was %25 [18]. Tan and Triggs [19] recommended Local Ternary Pattern (LTP) as an extension to Local Binary Pattern (LBP). The goal of their technique was definition descriptors more robust to noise. The main different between local ternary pattern and local binary pattern is that LTP uses 3-value for coding (1,0,-1) but LBP uses only 2-value for coding also LTP compares the pixels in the neighborhood by value of the central pixel and uses a tolerance threshold which reduces the sensitivity to noise. They showed that using local histogramming with a local distance transform improves the performance of face recognition. They used Grand Challenge version 1 database and Yale B and CMU-PIE for their experiment. The average recognition rate of their implementation on FRGC-104 was %45 but improved this rate to %80.4 by replacing LBP with LTP and increased to %86.3 by adding Distance Transform. The overall recognition rate of their experiment on Yale-B was %97.2. The recognition rate of their method on CMU-PIE was %100[19]. Franco and Nanni [20] used a method based on fusion of various classifiers to overcome illumination variation in face identification. They understood that a various feature represents the different information and fusion role can use them complimentarily. In their study, they combined three matches. The main idea of their approach was to use the good properties of different image based image to gain comparably results with 3Dbased systems. Their results approved that matchers gained applying various features, were complementary and let to achieve a noticeable performance improvement with respect to the single matchers. Furthermore, they understood that lowpass information on the wavelet transformed was the most useful feature set for the identification problem in an absence of strong illumination. They used low-pass information of the first level and the horizontal details of the third level of decomposition gained by Biorthogonal wavelet and Reverse Biorthogonal wavelet. They projected features by using PCA onto a k-dimensional subspace classified the faces by using neighbor or kernel Laplacian Eigenfaces. After that they used “Sum Rule” for combining the classifiers trained on the transformed features. They used ORL and Yale-B and BioLab databases. The overall error rate of their approach on ORL database was %5.5 and on Yale B database was %9 and on BioLab was %22.7[20]. Gaoyan et al [21] proposed an illumination normalization model (INM) for the preprocessing of face recognition under illumination. The advantage of their method is that it could compensate all kinds of change lighting effects in face recognition such as diffuse reflection, specular reflection, attached shadow and cast shadow. They decomposed the face sample in two parts low and high frequencies, and they generated a smallscale part image such as nose, eyes and skin textures,

which contain detailed features. These features are illumination invariant. After that they produced a noiseless large-scale part and then compensated illumination effects by using a region based Histogram Equalization. At the last, they fused two scale parts of normalized illumination facial features. High order statistical methods like KPCA and ICA were used for feature extraction. They used CAS-PEAL and FERET and Yale-B databases for their implementation. The performance of their method on CAS-PEAL was %55 and the recognition rate of their model was %61 on FERET database and %40 for Yale-B database [21]. Hsieh and Tung [22] suggested a novel shadow compensation approach for face identification under varying illumination based on facial symmetry and image average, which can solve the drawbacks of the conventional algorithm such as region-based histogram equalization (RHE) and histogram equalization (HE). The proposed approach called a mirror-region based histogram equalization and image average approach (MR-HEIA). Their new technique not only has properties of previous mentioned models such as simplicity, general propose and no training images required, but also it solves the problems of those techniques. In order to enhancing the local contrast, HE in pre-defined mirror-regions is carried out. After that, 2k enhanced mirror regions are obtained. In next step, they produced k enhanced mirror-regions then k modified average images were obtained. They compensated to face image and used PCA and 2DPCA for feature extraction respectively and applied nearest neighbor classifier based on the Euclidean distance for classification in their face recognition procedures. Yale B and Weizmann databases were used for evaluation the performance of their approach. 600 face images of 10 subjects from Yale B database were applied. They showed that the performance of MR-HEIA achieved up to a %18.57. In Weizmann database, they selected 150 face images of 15 different illumination conditions. They presented that the performance of their method achieved up to a %19.29 [22]. In table 3, the summary of mentioned approaches and their performances are shown.



Discussion

Image comparison method is one of the common methods, which can use to overcome human face recognition under illumination variation because of variations in the illumination is to use image representations that are depended to these variations. Examples of such representations are imaging intensity derivative, edge maps, and images convolved with 2D filters. Some representative techniques of human face identification by using Image based method have been reviewed with attentions on their performances and accuracies under illumination variation. Various techniques and their performances are summarized in Table [2]. The base of these methods is using different filters to reduce illumination variation. In methodology, Gabor filter is more robust in illumination variation. This is because Gabor filter is a very sharp filter and the dimension of Gabor filter is very high. In this category, the recognition rate is good and is upper than %80.

2.3 Class based methods The base of analysis in this category is comparing knowledge of human face class. Many vision systems have been applying the similarity of face shape and the quasi-constant albedo. In Hallilan[23] approach, three aligned face images were used under different lighting conditions. He suggested a low dimensional model for human faces who not only synthesized a face image when given lighting conditions but also estimated lighting conditions when a face image. In this method, NonLambertian and self shadowing surfaces was handled. He assumed ideal conditions and got illumination invariant results for fixed viewpoints. The limitation of this technique is that the Low- dimensional reconstructions of faces do appear flat[23]. Illumination cone is another effective method that recently researchers had been interesting to it. In this method, investigators study about shadowing and multiple light sources. Georghiades and Kriegman and Belhumeur[24] worked on this subject. 121 generated images from Yale B database were used for theirexperiment. They constructed cone representation without cast shadows for cones-attachment and applied the ray-tracing and the reconstructed surface of the face to determine cast shadows. The basis vectors for a linear subspace were determined by using SVD on the 121 extreme rays. The error rate of their approach for the linear subspace was %15 and %8.6 for conesattached[24]. Batur and Hayes [25] suggested another extension to linear subspace method. They segmented each image into regions that their normal surfaces have closed direction then they determined the clustering step a specific illumination subspace which is calculated. The base of their idea in segmentation was that, if the directions of normal surface got close to each other, the success of low dimensional linear subspace approximations of the illumination cone could increase. They applied statistical performance evaluation experiments to compare their system to other popular systems. They used Yale B database and implemented Eigenfaces, 3D linear subspace method, 4D linear subspace method and Cones-attached in their experiment. The recognition error rate this study was %23.4 for Eigenfaces and %6.82 for 3D linear subspace and %4.02 for Cones-attached[25]. In Cheng et al [26] NormalShrink filter in Nan subsampled Contourlet Transform (NSCT) is used to reduce an effect of illumination variation in face identification. This method changed multiplicative illumination model into additive one by using a logarithm transform on original face images under various illumination conditions. They used Yale, Yale B and CMU PIE for their experiment. For feature extraction, they applied PCA and nearest neighbor classifier based on Euclidean was employed for classification. The average recognition rate of their method for Yale B database was %99.5 and for Yale database was %96.09 and for CMU PIE was %99.73. The advantages of their method are highlighted:

1. Edges due to the nonsubsampled contourlet transform are carried on in compare of other methods. 2. Multi scale contour structure which in the logarithm domain is illumination invariant in this method is directly detected. 3. Some prior information such as a light sources assumption and large training sample which are necessary for 3D face model, are not necessary for this technique[26]. Table 3 shows the summary of mentioned approaches and their performances.

 Discussion Comparing knowledge of human face class is the base of this category. The similarity of face shape and the quasi-constant albedo were applied in many vision systems. The various methodologies and their performances are summarized in Table [3]. To use a generic face shape is the simplest strategy to approximate different face shapes, which give these uniform images based approaches the benefit of efficiency. To better approximate face shape, researchers used 3D models from a set of facial features or from pixel-wise image intensities. Feature-based reconstructions usually need a feature point locating, which is always based on image contents. Though image-based methods often involve complex processing in considering reflection of human faces, investigators made the most use of image information by exhaustive treatment of all image pixels. Consequently, existing methods attempt to make simplistic approximations on face surfaces. Most of these techniques assume face surfaces as Lambetian surfaces, which only consider diffuse reflection and neglect specular reflection. In methodology level, NSCT + PCA with the nearest neighbor classifier is more robust to illumination variation.

2.4

3D model based approaches

Three dimensional techniques are suggested to overcome different limitations of previous methods in reduction illumination variation in face recognition or verification. One of the advantages of using 3D methods in face identification systems is that human face can be in arbitrary illumination and pose then deal with face identification across pose and illumination variations are easier than other methods. Shape From Shading (SFS) is a very good technique that one gray level image of a surface is used for computing three dimensional shape of that surface. Finding the solution of a non-linear first order Partial Different Equation (PDE) which called brightness is one of problems of this method. Atick et al [27] recommended Principle component Analysis as a tool for solving the parametric shape-form-shading (SFS) problems. A database of laser scanned human heads was used for driving the eigenheads. This database was made by the Human Engineering Division of Patterson USAF, and it was known “mini-survey”. 374 scanned heads of adult males were in this database. They used PCA to drive a low-dimensional parameters of head shape space [27]. Gross et al [28] proposed using a varying albedo reflectance for solving lighting variation. They suggested

an appearance-based algorithm for face identification under illumination variation. This algorithm estimated the light-field of the subject’s head. Firstly, they used generic training data for calculating an eigenspace of head lightfields. They performed the projection into the eigenfaces by setting up a least-squares problem. Furthermore, they used occlusions in the eigenspace approach for the projection coefficients. They applied CMU-PIE and FERET databases for their experiment. Average recognition accuracy on the FERET database was %39.4 and %16.6 on the CMU-PIE [28]. Blanz and Vetter [29] suggested a technique which it was based on a 3D morphable model. They used computer graphics for approximating 3D shape and texture of faces from single images. Fitting a statistical, morphable model of 3D faces to images caused that they can get their estimate. They used 4488 images from the publicly available CMU-PIE database and 1940 images from the FERET database. PCA was applied for feature extraction. The overall identification rate of their approach was %95 for CMU-PIE and %95.9 for FERET database [29]. Qing et al [30] regained nine spherical harmonic images of the illumination space from only one face image, which was under the arbitrary lighting conditions. They suggested a new approach for face identification under a generic illumination condition and named it Eigen-harmonics faces. They used bootstrap set containing of 3D face models with texture. The spherical harmonic images were rendered for every face also the lighting of the image was estimated. During testing, PCA coefficients were applied for face recognition. They used CMU-PIE database for their experiment and the overall recognition rate of their method was around %97 [30]. Table 5 shows the summary of mentioned approaches and their performances.



Discussion

In this section, some representative techniques of human face identification by using 3D model based approaches have been reviewed with attentions on their performances and accuracies under illumination variation. We summarized the techniques and their performances in Table [4]. One of the advantages of using 3D methods in face identification systems is that human face can be in arbitrary illumination and pose then deal with face identification across pose and illumination variations are easier than other methods. In methodology level, an Eigen-harmonics face with using PCA is more robust to illumination variation.

3 Conclusion Different approaches in human face identification under illumination variation are divided in four categories, which were explained before. In Heuristic techniques perhaps some important components waste but investigators can get better performance by using symmetry methods. In comparison methods, researchers used various filters to limit or reduce the influence of

illumination. In these methods using a single filter cannot be successful in overcoming the illumination problem. So using compound different algorithm such as PCA and filters is recommended. Class based method compares knowledge of human face class. Clearly, because of a large number of extreme rays, in many projects this kind of analysis is very difficult. One of the most important advantages of the 3D model based approaches in face identification is that human face can be in arbitrary illumination and pose but the expense of produce of data bases are very high. There are some common issues in order to the individual drawbacks for each method. First of all, it is clear that, most of the researchers have been gotten their results on manually marked data. The sensitivity to localization error still requires to be searched for all approaches. Secondly, the databases to analyze various methods are of very small size. For example, there are images of only 10 subjects in Yale B database. The accuracy and performance of illumination invariant approaches on a large scale human face database is still unknown. Usually it is very difficult to compare

the performance and accuracy between different approaches because the various methods used different databases and also the number of human face images in various approaches is different.

 Future

direction for methods to overcome illumination problems The current modernizations in image capture technology are increasing in quality the scope for constructing three dimensional models of human

faces. As a result, several techniques for capitalizing on the extra information this can supply for face identification have been recently suggested. In principle, researchers can consider to use the 3D shape model of a face to overcome illumination problems. In practice, the capture of an accurate three dimensional face model is also subject to variations in the illumination conditions.

Table [1]: Review of recording conditions of some databases Database AR BANCA CAS-PEAL CMU Hyper CMU PIE Equinox IR FERET Harvard RL KFDB MIT MPI ND HID NIST MID ORL UMIST U.Texas U.Oulu XM2VTS Yale Yale B

No.of subjects 116 208 66-1040 54 68 91 1199 10 1000 15 200 300+ 1573 10 20 284 125 295 15 10

Time 2 12 2 1-5 1 1 2 1 1 1 1 10/13 1 ++ 1 1 1 4 1 1

Illumination 4 ++ 9-15 4 43 3 2 77-84 16 3 3 3 1 ++ 1 1 16 1 3 64

Pose 1 1 21 1 13 1 9-20 1 7 3 3 1 2 1 ++ ++ 1 ++ 1 9

Facial Expressions 4 1 6 1 3 3 2 1 5 1 1 2 ++ ++ ++ ++ 1 ++ 6 1

Table [2]: Experiments and performances in Heuristic method Name of researchers

Database

Number of images

Approach

Performance

Belhumeur, et, al

Harvard Robotics Laboratory and Yale

30 images from Harvard database and 36 images from Yale

PCA, FLD [5-6]

%47.4

Bartlettand, et, al

FERET

425

ICA [7]

%48

Zhao

FERET, Yale and Weizmann

1038 images from FERET and 495 images from Yale and Weizmann databases

PCA, LDA, SFS [8]

%38.5

Chen, et, al

Yale B and CMU PIE

DCT [9]

%57 for CMU PIE and %48.6 for Yale B

64 images from

CMU PIE and 2o image from Yale B

Table [3]: Experiments and performances in Image comparison method Name of researchers

Database

Number of images

Approach

Performance

Nanni and Maio

FERET

1196

Gabor filter +PCA[11]

%94.19

Jacobs and Belhumeur and Basri

A database of faces made by Hallianan

450

Correlation and PCA[12]

%74.4 for Correlation and %76.5 for PCA

Liu, et, al

Yale B and Yale and FERET

1196

Gaussian filter and PCA[13]

%50.8 for Yale B and %55 for Yale

64 images from CMU-PIE and 2o image from Yale B

Harmonic Image[14]

%47.1 for CMU- PIE and %60 for Yale B

Qing, et, al

Yale B and

CMU-PIE

Arandjelovic and Cipolla

FaceDB100 and FaceDB60

1300

High pass filter[15]

%58.3 for FaceDB100 and %46.6 for FaceDB6

Savvides, Kumar and

Yale B and CMU-PIE

2005

Correlation filter and PCA[16]

%98.9

Du and Ward

Yale B

640

Wavelet Transform[17]

%56.2

Nanni and Lumini

Notre-Dame

198

LBP,AdaBoost,SFFS[18]

%75

Tan and Triggs

Face Recognition Grand Challeng version 1, Yale B and CMU-PIE

2414

LTP and Distance Transform[19]

%86.3 for FRGC1 and %97.2 for Yale B and %100 for CMU-PIE

Franco and Nanni

ORL and Yale-B and BioLab

2840

Multi- matcher[20]

%94.5 for ORL and %91 for Yale-B and %77.5 for BioLab

Gaoyan, et, al

CAS-PEAL and FERET and Yale-B

3000

INM and PCA and FLD and KPCA and ICA[21]

%55 for CAS-PEAL and %61 for FERET and %40 for YaleB

Hsieh and Tung

Yale-B and Weizmann

600

MR-HEIA and PCA and 2DPCA[22]

%85.71

Table [4]: Experiments and performances in Class based method Name of researchers

Database

Number of images

Approach

Performance

Georghiades and Kriegman and Belhumeur

Yale-B

121

Low dimensional model and SVD[24]

%85

Batur and Hayes

Yale-B

600

Eigenfaces and 3D linear subspace and 4D linear subspace and Cones-attached methods[25]

%23.4 for Eigenfaces and %6.82 for 3Dlinear subspace and %4.02 for Cones-attached

Cheng,et,al

Yale and Yale-B and CMU-PIE

700

NSCT and PCA and nearest neighbor classifier[26]

%96.09 for Yale and %99.5 for Yale-B and %99.73 for CMU PIE

Table [5]: Experiments and performances in 3D method based approach Name of researchers

Database

Number of images

Approach

Performance

Gross, et, al

CMU-PIE and FERET

1600

Occlusions in eigenspace[28]

%39.4 for FERET and %16.6 for CMU-PIE

Blanz and Vetter

FERET and CMU-PIE

6428

3D morphable model and PCA[29]

%95 for CMU-PIE and %95.9 for FERET

Qing et, al

CMU-PIE

900

Eigen-harmonics faces and PCA[30]

%97

4 References [1]

[2]

[3]

[4] [5]

[6]

[7]

[8]

[9]

[10]

[11]

[12] [13]

[14]

[15]

[16]

A. Goldstein, et al., "Identification of human faces," Proceedings of the IEEE, vol. 59, pp. 748760, 1971. T. Kohonen, et al., "Statistical pattern recognition with neural networks: Benchmarking studies," 1988, pp. 61-68. L. Sirovich and M. Kirby, "Low-dimensional procedure for the characterization of human faces," Journal of the Optical Society of America A, vol. 4, pp. 519-524, 1987. J. Short, "Illumination Invariance for Face Verification," 2006. P. Belhumeur and D. Kriegman, "What is the set of images of an object under all possible illumination conditions?," International Journal of Computer Vision, vol. 28, pp. 245-260, 1998. P. Belhumeur, et al., "Eigenfaces vs. Fisherfaces: Recognition using class specific linear," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, 1997. M. Bartlett, et al., "Independent component representations for face recognition," 1998, pp. 528–539. W. Zhao and R. Chellappa, "Robust face recognition using symmetric shape-fromshading," Center for Automation Research, University of Maryland, College Park, Technical Report CARTR-919, 1999. W. Chen, et al., "Illumination compensation and normalization for robust face recognition using discrete cosine transform in logarithm domain," IEEE Transactions on Systems, Man, And Cybernetics—Part B: Cybernetics, vol. 36, p. 459, 2006. Y. Adini, et al., "Face recognition: the problem of compensating for illumination changes," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, pp. 721-732, 1997. L. Nanni and D. Maio, "Weighted Sub-Gabor for face recognition," Pattern Recognition Letters, vol. 28, pp. 487-492, 2007. D. Jacobs, et al., "Comparing images under variable illumination," 1998, pp. 610-617. D. Liu, et al., "Illumination invariant face recognition," Pattern Recognition, vol. 38, pp. 1705-1716, 2005. L. Qing, et al., "Face recognition under generic illumination based on harmonic relighting," International Journal of Pattern Recognition and Artificial Intelligence, vol. 19, pp. 513-532, 2005. O. Arandjelovic and R. Cipolla, "A new look at filtering techniques for illumination invariance in automatic face recognition," 2006, pp. 449-454. M. Sawides, et al., "" Corefaces"-robust shift invariant PCA based correlation filter for illumination tolerant face recognition."

[17] [18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

[29]

[30]

[31]

S. Du and R. Ward, "Wavelet-based illumination normalization for face recognition," 2005. L. Nanni and A. Lumini, "RegionBoost learning for 2D+ 3D based face recognition," Pattern Recognition Letters, vol. 28, pp. 2063-2070, 2007. X. Tan and B. Triggs, "Enhanced local texture feature sets for face recognition under difficult lighting conditions," 2007, pp. 168-182. A. Franco and L. Nanni, "Fusion of classifiers for illumination robust face recognition," Expert Systems with Applications, vol. 36, pp. 89468954, 2009. G. An, et al., "An illumination normalization model for face recognition under varied lighting conditions," Pattern Recognition Letters, vol. 31, pp. 1056-1067, 2010. P. Hsieh and P. Tung, "Shadow compensation based on facial symmetry and image average for robust face recognition," Neurocomputing, 2010. P. Hallinan, "A low-dimensional representation of human faces for arbitrary lighting conditions," 1994, pp. 995-995. A. Georghiades, et al., "From few to many: Illumination cone models for face recognition under variable lighting and pose," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, pp. 643-660, 2001. A. Batur and M. Hayes Iii, "Segmented linear subspaces for illumination-robust face recognition," International Journal of Computer Vision, vol. 57, pp. 49-66, 2004. Y. Cheng, et al., "Robust face recognition based on illumination invariant in nonsubsampled contourlet transform domain," Neurocomputing, 2010. J. Atick, et al., "Statistical approach to shape from shading: reconstruction of threedimensional face surfaces from single twodimensional images," Neural Computation, vol. 8, pp. 1321-1340, 1996. R. Gross, et al., "Face recognition across pose and illumination," Handbook of Face Recognition, pp. 193-216, 2005. V. Blanz and T. Vetter, "Face recognition based on fitting a 3D morphable model," IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1063-1074, 2003. L. Qing, et al., "Eigen-harmonics faces: Face recognition under generic lighting," 2004, pp. 296-301. G. Heusch, et al., "Lighting normalization algorithms for face verification," 2005, pp. 3-4.