Int. J. Biometrics, Vol. 7, No. 1, 2015
Face verification using local binary patterns and generic model adaptation Elhocine Boutellaa*, Farid Harizi and Messaoud Bengherabi Centre de Développement des Technologies Avancées, BP 17, Baba Haasen 16081, Algeria Email:
[email protected] Email:
[email protected] Email:
[email protected] *Corresponding author
Samy Ait-Aoudia Ecole Nationale Supèrieure d’Informatique, Oued-Smar 16270, Algeria Email:
[email protected]
Abdenour Hadid University of Oulu, PL 8000, Oulu FI-90014, Finland Email:
[email protected] Abstract: The popular local binary patterns (LBP) have been highly successful in representing and recognising faces. However, the original LBP-based face recognition method has some problems that need to be addressed. In this work, we propose two approaches to address the histogram representation drawbacks in the LBP-based face verification system. The first approach employs vector quantization maximum a posteriori adaptation (VQMAP) model, where a generic face model is obtained by vector quantisation and the user models are inferred using maximum a posteriori adaptation. The second approach proposes an enhanced LBP histogram representation by adapting a generic face histogram to each user. Moreover, the two proposed approaches are further fused to enhance the verification performance. We evaluate our proposed approaches on two publicly available databases, namely BANCA and XM2VTS, and compare the results against the original LBP approach and its variants, demonstrating very promising results. Keywords: local binary patterns; LBPs; vector quantisation maximum a posteriori adaptation; face verification; histogram adaptation. Reference to this paper should be made as follows: Boutellaa, E., Harizi, F., Bengherabi, M., Ait-Aoudia, S. and Hadid, A. (2015) ‘Face verification using local binary patterns and generic model adaptation’, Int. J. Biometrics, Vol. 7, No. 1, pp.31–44.
Copyright © 2015 Inderscience Enterprises Ltd.
31
32
E. Boutellaa et al. Biographical notes: Elhocine Boutellaa is a PhD student at Ecole Nationale Supèrieure en Informatique (ESI), Algiers, Algeria. He is working on face recognition and analysis. He received his Eng. and MSc in Computer Science from ESI in 2007 and 2011, respectively. He is currently a Research Associate at Centre de Développement des Technologies Avancées (CDTA), Algiers, Algeria. Farid Harizi received his State Engineer degree in Electrical Engineering Option Communication from the Université des Sciences et de la Technologie Houari Boumediene USTHB, Algiers, Algeria, and Magister degree in Signal Processing from the same university in 2000. Currently, he is a researcher at the Centre de Développement des Technologies Avancées (CDTA), Algiers, Algeria. His research interests include face detection and recognition and multimodal biometrics. Messaoud Bengherabi received his State Engineer degree in Electrical Engineering Option Communication from the National Institute of Electricity and Electronics (INELEC), Boumerdes, Algeria in 1996 and Magister degree in Advanced Signal Processing from the Ecole Polytechnique (EMP), Bordj El Bahri, Algiers, Algeria in 1999. Since joining CDTA in October 1999, he has been engaged in research on low delay narrowband-wideband CELP-based speech coders and biometrics. Currently, he is the Head of the Biometric Team in CDTA. His research interests include speaker and face recognition, multimodal biometrics and forensic applications. Samy Ait-Aoudia received his DEA Diplôme d’Études Approfondies in Image Processing from Saint-Etienne University, France in 1990. He received his PhD in Computer Science from the Ecole des Mines, Saint-Etienne, France in 1994. He is currently a Professor of Computer Science at the Ecole Nationale Supèrieure en Informatique in Algiers, Algeria, where he is involved in teaching BSc and MSc levels in computer science and software engineering. His areas of research include image processing, CAD/CAM and constraints management in solid modelling. Abdenour Hadid is an Adjunct Professor (Docent) and Senior Researcher in the Center for Machine Vision Research of the University of Oulu. He made significant contributions to the state-of-the-art and his work is gaining increasing interest in the scientific community. He served as a member of the organising committee of several international workshops. He lectured already many invited talks and tutorials in international events.
1
Introduction
It is widely believed that biometrics will become a significant component of the identification technology and it is already of universal interest. The goal of a biometric system is to determine the identity of an individual using physical/biological characteristics (i.e., biometric modalities). Biometric systems have many applications such as criminal identification, airport checking, computer or mobile devices log-in, building gate control, digital multimedia access, transaction authentication, voice mail, or secure teleworking. Various characteristics can be used: from the most conventional biometric modalities such as face, voice, fingerprint, iris, hand geometry or signature, to the so called emerging biometric modalities such as gait, hand-grip, ear, body odour,
Face verification using local binary patterns and generic model adaptation
33
body salinity, electroencephalogram or DNA. Each biometric modality has its strengths and drawbacks (Li and Jain, 2009). Biometric systems can run into two fundamentally distinct modes: 1
verification (or authentication)
2
recognition (more popularly known as identification).
In authentication mode, the system aims to confirm or deny the identity claimed by a person (one-to-one matching) while in recognition mode the system aims to identify an individual from a database (one-to-many matching). Because of its natural and non-intrusive interaction, identity verification and recognition using facial information is among the most active and challenging areas in computer vision research (Li and Jain, 2009, 2011). However, despite the great deal of progress during the recent years (Li and Jain, 2011), face biometrics (that is identifying individuals based on their face information) is still a major area of research. Wide range of viewpoints, aging of subjects and complex outdoor lighting are still challenges in face recognition. There are several ways to categorise different face description approaches (Li and Jain, 2011). One of the most widely used divisions is to distinguish whether the method is based on representing the feature statistics of small local face patches (i.e., local) or computing features directly from the entire image or video (i.e., global). Lately the local methods have proved to be more effective in real world conditions whereas the other approaches have almost disappeared. However the global methods have recently started to partially reappear to complement the local descriptors (Li and Jain, 2011). The recent developments in face analysis and recognition have shown that the local binary patterns (LBPs) (Ojala et al., 2002) provide excellent results in representing faces (Ahonen et al., 2006; Pietikäinen et al., 2011). LBP is a grey-scale invariant texture operator which labels the pixels of an image by thresholding the neighbourhood of each pixel with the value of the centre pixel and considers the result as a binary number. LBP labels can be regarded as local primitives such as curved edges, spots, flat areas, etc. The histogram of the labels can be then used as a face descriptor. Due to its discriminative power and computational simplicity, the LBP methodology has attained an established position in face analysis and has inspired plenty of new research on related methods. The original LBP has some problems that need to be addressed in order to increase its robustness and discriminative power and to make the operator suitable for the needs of different types of problems. The present work proposes new solutions that address inherent problems to the original LBP-based face verification system. One problem with the LBP method, for instance, is the number of entries in the LBP histograms as a too small number of bins would fail to provide enough discriminative information about the face appearance while a too large number of bins may lead to sparse and unstable histograms. To overcome this drawback, we propose an efficient and compact LBP representation for face verification. The face is first divided into several regions from which LBP features are extracted. LBP codes in each region are then quantified into a low-dimensional feature vector. The face is represented by concatenating the vectors from all the regions. We generate reliable face model using vector quantisation maximum a posteriori (VQMAP) adaptation method (Hautamaki et al., 2008). For face verification,
34
E. Boutellaa et al.
we use the mean squared error (MSE) to match a test feature vector to the claimed user model. Another serious drawback of the LBP method lays in the feature vector robustness as the histogram estimation is not always reliable. We tackle this problem by first estimating a reliable generic feature vector obtained from a pool of users. Face images are divided into equal blocks from which LBP features are extracted and LBP histograms over blocks are concatenated to form a feature vector. The adapted histogram of a given block is obtained by weighting its histogram and the generic block one. The chi-square (χ2) distance is used to match a probe against the claimed identity model. To compensate the cohort effect introduced by the generic feature vector, we finally normalise the obtained score by subtracting the distance between the probe and the generic feature vectors. We extensively evaluate our two proposed approaches and also their fusion on two publicly available benchmark databases, namely XM2VTS and BANCA. We compare our obtained results against not only those of the original LBP approach but also those of other LBP variants, demonstrating very encouraging performance. The rest of this paper is organised as follows. Section 2 describes the original LBP-based representation for face recognition. In Section 3, our first proposed approach for efficient and compact LBP representation overcoming LBP drawbacks (i.e., sparse and unstable histograms) is introduced. Section 4 presents the second proposed approach for robust LBP feature vector estimation. Extensive experimental analysis is presented in Section 5 and conclusions are drawn in Section 6.
2
Face representation using LBPs
The LBP texture analysis operator, introduced by Ojala et al. (2002), is defined as a greyscale invariant texture measure, derived from a general definition of texture in a local neighbourhood. It is a powerful means of texture description and among its properties in real-world applications are its discriminative power, computational simplicity and tolerance against monotonic grey-scale changes. The original LBP operator forms labels for the image pixels by thresholding the 3 × 3 neighbourhood of each pixel with the centre value and considering the result as a binary number. Figure 1 shows an example of an LBP calculation. The histogram of these 28 = 256 different labels can then be used as a texture descriptor. Figure 1
The basic LBP operator
The operator has been extended to use neighbourhoods of different sizes. Using a circular neighbourhood and bilinearly interpolating values at non-integer pixel coordinates allow any radius and number of pixels in the neighbourhood. The notation (P, R) is generally used for pixel neighbourhoods to refer to P sampling points on a circle of radius R. The
Face verification using local binary patterns and generic model adaptation
35
calculation of the LBP codes can be easily done in a single scan through the image. The value of the LBP code of a pixel (xc, yc) is given by: P −1
LBPP , R =
∑ s( g
p
− g c )2 p ,
(1)
p =0
where gc corresponds to the grey value of the centre pixel (xc, yc), gp refers to grey values of P equally spaced pixels on a circle of radius R, and s defines a thresholding function as follows: ⎧1,if x ≥ 0; s( x) = ⎨ ⎩0, otherwise.
(2)
Another extension to the original operator is the definition of the so called uniform patterns. This extension was inspired by the fact that some binary patterns occur more commonly in texture images than others. A LBP is called uniform if the binary pattern contains at most two bitwise transitions from 0 to 1 or vice versa when the bit pattern is traversed circularly. In the computation of the LBP labels, uniform patterns are used so that there is a separate label for each uniform pattern and all the non-uniform patterns are labelled with a single label. This yields to the following notation for the LBP operator: LBPPu,2R . The subscript represents using the operator in a (P, R) neighbourhood. Superscript u2 stands for using only uniform patterns and labelling all remaining patterns with a single label. Each LBP label (or code) can be regarded as a micro-texton. Local primitives which are codified by these labels include different types of curved edges, spots, flat areas, etc. The occurrences of the LBP codes in the image are collected into a histogram. The classification is then performed by computing histogram similarities. For an efficient representation, facial images are first divided into several local regions from which LBP histograms are extracted and concatenated into an enhanced feature histogram.
3
Face verification using LBP and VQMAP
A simple concatenation of all local block histograms as in the original LBP approach may be subject to the curse of dimensionality (e.g., sparse and unstable histograms). To tackle this problem, we describe in this section an elegant solution.
3.1 LBP quantisation In original LBP-based face representation and most of its variants, extracted histograms over blocks are generally sparse. Most of bins in the histogram are zero or near to zero, particularly in the case of small blocks. Indeed, the number of LBP labels in a block depends on its size. On one hand, large blocks produce dense histograms that badly represent local face changes. On the other hand, small blocks are robust to local changes but create unreliable sparse histograms, as the number of histogram bins exceeds by far the number of LBP patterns in the block. Another problem with LBP representation is that the number of bins of the histogram is function of the number of neighbourhood sampling points P. The number of histogram bins grows considerably when P increases
36
E. Boutellaa et al.
(there are P ∗ (P − 1) + 3 bins per block). Hence, small neighbourhood yields in compact but poor representation whereas large neighbourhood produces huge and unreliable feature vectors. Furthermore, not all LBP labels are present in a given face region. Labels with low occurrences can be considered as noise, produced by one bit transition in the LBP code, and thus are useless for characterising the face region. Therefore, a block can be efficiently characterised by a more accurate low dimensional vector by ignoring those patterns. To tackle these problems, we apply vector quantification to each block of the face in order to dynamically obtain a more accurate feature vector that represents the face in a best way. Hence, patterns of each block are clustered into a fixed number of groups and the face is represented by resulting codebook. Thus, only relevant LBP labels of a given block will be represented while other labels are ignored. This yields into a feature of the patterns which are face-specific and thus suitable for face representation. In our proposed approach, the clustering of LBP labels is achieved by LBG algorithm (Linde et al., 1980). LBG algorithm is like a K-means clustering algorithm which takes a set of vectors S = {xi ∈ Rd|i = 1, ..., n} as input and generates a representative subset of vectors C = {cj ∈ Rd |j = 1, ..., K} with a specified K
