Universiteit van Amsterdam
Real-time face verification using a low resolution camera Author: Willem Bressers
[email protected] Student number: 5687063
Supervisor: Nicu Sebe
[email protected] Universiteit van Amsterdam
March 1, 2009
Coarse: BSc AI Thesis BA Kunstmatige Intelligentie Universiteit van Amsterdam Spui 21, 1012 WX, Amsterdam The Netherlands
Abstract The demand of an real-time face detection system using a low resolution camera is rapidly growing, while the purpose of these demands vary from security to gaming. This thesis describes the methods used for real-time face verification. It proves a robust real time verification application is possible using a low resolution camera. It uses the very rapid boosted cascade classifiers for object detection as proposed by Viola et al [11] and extended by Lienhart et al [7]. The recognition of an object is strongly based on the proposal of Turk et al [10] and verified by using distance measurement. This Thesis will describe some experiments with an implementation to show the speed and accuracy of the application.
Acknowledgments I would like to thank my supervisors Nicu Sebe and his PhD student Roberto Valenti for motivating and support. Im indebted to Bert Bredeweg for making an exception in the planning. Also i want to thank Leo Dorst for supporting my thesis proposal.
Contents 1 Introduction 1.1 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Theoretical background 2.1 Object detection . . . . . . . . . . . . . . . . . . . . 2.1.1 Features . . . . . . . . . . . . . . . . . . . . . 2.1.2 Multi-stage cascade classifier . . . . . . . . . 2.2 Learning . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.1 Principal Component Analysis . . . . . . . . 2.2.2 Optimal eigenface and principal components 2.3 Verifying . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.1 Projection . . . . . . . . . . . . . . . . . . . . 2.3.2 Nearest neighbor . . . . . . . . . . . . . . . . 2.3.3 Threshold . . . . . . . . . . . . . . . . . . . .
3 3 4 4
. . . . . . . . . .
5 5 5 7 8 8 9 10 10 10 12
3 Implementation 3.1 Graphical User Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Objective-C vs C++ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
13 13 14
4 Experiments 4.1 Data sets . . . . . . . . . . . 4.1.1 Yale face database B . 4.1.2 Personal face database 4.2 Results . . . . . . . . . . . . . 4.2.1 Real time performance 4.2.2 Face detection . . . . 4.2.3 Face verification . . .
15 15 15 17 17 17 19 21
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
1
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . . . . .
. . . . . . .
. . . . . . . . . .
. . . . . . .
. . . . . . . . . .
. . . . . . .
. . . . . . . . . .
. . . . . . .
. . . . . . . . . .
. . . . . . .
. . . . . . . . . .
. . . . . . .
. . . . . . . . . .
. . . . . . .
. . . . . . . . . .
. . . . . . .
. . . . . . . . . .
. . . . . . .
. . . . . . . . . .
. . . . . . .
. . . . . . . . . .
. . . . . . .
. . . . . . . . . .
. . . . . . .
. . . . . . .
5 Conclusion & Further Research 5.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Further Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
23 23 23
Bibliography
24
2
Chapter 1
Introduction Face recognition has been a wide researched area, but the problem with recognition is that it has to be designed to recognize every face. Practically spoken this isn’t feasible, because that would mean that everyone should be known by the system. Also the system should be updated constantly. But let’s look at the problem on a smaller scale, let’s assume that we want to recognize the people that we know. This leads to the main question. Research Question Is it possible to create an application that real time detects a person in front of a webcam and compares the detected face with known people on the computer, in order to react to that information? It’s now no longer a problem of recognizing the faces but more verifying the detected faces with the known faces. This Thesis is based on the research question.
1.1
Related work
Because face recognition has been widely researched, verification could use the same principles. A good starting point for reading the related work is to read the survey form Zhao et al [12]. This paper is an extended research in the face recognition area. Turk et al [10] proposes an method which uses Principal Component Analysis (PCA) to span an image space and use distance measurement to classify the face. Barlett et al [2] proposed a similar method to Turk et al but instead of using a unsupervised statistical method like the PCA, it uses an supervised statistical method named Independent Component Analysis (ICA). This method is sensitive to high-order statistics. Etemad et al [3] proposed a similar method to the PCA namely the Linear Discriminant Analysis (LDA). This method uses the vectors that best discriminate the space into different classes. This is useful when there is a small number of images available. Liu et al [8] proposes a recognition
3
method also based on the PCA but it uses Genetic Algorithms (GA) for classifying the face. The fitness function is defined in terms of accuracy and class separation. There are many more research papers about face recognition, but the papers mentioned above are closest related to the approach used in this Thesis.
1.2
Goals
The main goal of this Thesis is to answer the research question. But because this a vague and general description, it needs to be split up in several sub-questions. First Is it possible to detect a face in a real-time video stream ? Second How should the application learn ? Third Using a real time detector, is it possible to verify faces real time ?
1.3
Overview
Chapter 2 describes the methods that are used, which are split up into 3 sections. Section 2.1 describes the detection of an object in an image. In section 2.2 the learningis described, and section 2.3 describes the way the system verifies the objects. Chapter 3 explains how the theory is implemented in a working system. Chapter 4 describes the experiments that has been done and the results. Chapter 5 is used for conclusion and discussion.
4
Chapter 2
Theoretical background This chapter explains the Theory that has been used for completing the sub goals. It will discuss the Theory and the methods that are used.
2.1
Object detection
For real-time object detection a method is needed that is fast and accurate in detecting the correct object. Viola et al [11] have proposed a multi-stage classification method called the haar cascade classifier, that reduces computational time with almost the same accuracy compared to a slower single-stage classifier. The performance increase is achieved by using features instead of pixel values. The haar cascade classifier is an very fast mulit-stage classification method to find objects in an image. It uses an library of features which are hierarchical ordered. The classifier takes regions of different sizes of the original image, which are called subwindows. Because each feature is scaleable and defines specific domain knowledge, they all together represent much more information on the subwindow than the raw pixel values. Each subwindow is classified in order to define if there is a possible object.
2.1.1
Features
A feature is a collection of at least 2 rectangulars. The rectangulars are placed in specific positions with specific dimensions so that they represent knowledge about the domain. This description is a bit vague, but because the features are scaleable, it isn’t possible to give exact dimensions. This knowledge is difficult to extract from raw pixel values. A pool of different unique features represent knowledge in a specific domain.
5
Figure 2.1: Collection of features
The features in Figure 2.1 are inspired by the features used by Papageorgiou et al [9] and represent some knowledge about the domain. For example feature (1b) represents the horizontal edge, while feature (1c) represents a horizontal line. Because these features are scaleable it’s possible to tell how long or thick the line is. A feature is defined by one ore more rectangles, a rotation and a threshold. Each rectangle is defined by the following variables rec = (x, y, w, h, ω) with x, y ≥ 0 w, h > 0
x+w
