Robust Precise Eye Location by Adaboost and SVM Techniques Xusheng Tang, Zongying Ou, Tieming Su, Haibo Sun, and Pengfei Zhao Key Laboratory for Precision and Non-traditional Machining Technology of Ministry of Education, Dalian University of Technology Dalian, Liaoning 116000, China
[email protected], {ouzyg,tiemings}@dlut.edu.cn
Abstract. This paper presents a novel approach for eye detection using a hierarchy cascade classifier based on Adaboost statistical learning method combined with SVM (Support Vector Machines) post classifier. On the first stage a face detector is used to locate the face in the whole image. After finding the face, an eye detector is used to detect the possible eye candidates within the face areas. Finally, the precise eye positions are decided by the eye-pair SVM classifiers which using geometrical and relative position information of eye-pair and the face. Experimental results show that this method can effectively cope with various image conditions and achieve better location performance on diverse test sets than some newly proposed methods.
1 Introduction Robust eye detection is a crucial step towards face recognition and man-machine interaction because these applications need to normalize faces or extract features according to eye positions [1]. Many eye detection algorithms have been proposed in recent years such as grayscale projection [2],[3] template matching [4],[5] and deformable template matching [6],[7] etc. They can achieve good result under some constraints; however, they all have some limitations. Grayscale projection depends on the lighting conditions. The drawback of template matching method is the initial position of the eye template has to be set approximately to the position of the eyes and their computations are complicated. In practice, eye patterns become quite complex due to various factors, such as variation of size, pose, and illumination conditions etc. Robustly and precisely locating eyes center is still a challenging task. In this paper, a novel approach for precisely and fast locating eyes is devised. The proposed approach locates face areas using Adaboost face detector [8] prior to detecting eye location. After obtaining the position of face in the image, we can gain good initial constraints on the location of eyes. Then a followed Adaboost eye detector extracts eye candidates. Finally, precise positions of eye are decided by a SVM post classifier which is constructed using intrinsic geometrical relation information of eyepairs and face. Due to the non-accidental properties of geometrical relation information, the eye-like eye candidates such as eyebrow can be excluded easily. The experimental results demonstrate that this approach can robustly cope with different light condition and achieve high detection rate on diverse test sets. J. Wang, X. Liao, and Z. Yi (Eds.): ISNN 2005, LNCS 3497, pp. 93–98, 2005. © Springer-Verlag Berlin Heidelberg 2005
94
Xusheng Tang et al.
2 Proposed Method We assume the people appear in the image whose head rotation 10° both in and out of image plane. The background is complex. Our system firstly detects the face in the image and then extracts the eye candidates within the upper part of face region using the method of [8]. Next, using a procedure proposed in this paper, the system extracts geometrical features of eye-pair candidates. Finally, precise eyes positions are decided by SVM post classifier using geometrical features. The training and detection process scheme for our system can be summarized as shown in Fig. 1.
Fig. 1. Training and detection of Hierarchical Adaboost detector with SVM as post classifier
2.1 Face Region Location The problem of eye location is difficult since there are almost infinite numbers of specious objects with eye-like features in an input image. On the other hand, face detection is easier than eye detection as face pattern have more features than eye pattern. If we can localize the face in the input image the problem can be simplified due to the background being restricted to the face. It can also save searching time. Viola’s method [8] is used to detect face. The face classifier is obtained through supervised AdaBoost learning. Given a sample Set {xi,yi}, the AdaBoost algorithm selects a set of weak classifier {hj(x)} from a set of Haar-like rectangle features and combine them into a strong classifier. The strong classifier H(x) is defined as follow: 1 H ( x) = 0
∑Tj α j h j ( x ) ≥ θ
.
(1)
otherwise
The threshold is adjusted to meet the detection rate goal. The Haar-like rectangle features can be computed very quickly through a new image representation called “Integral Image”. The “cascade” method can quickly filter out non-face image areas. More details can be found in [8]. Our face training set is drawn from FERET, ARData and ORL database. The 7098 face images are processed with gray scale normalization and size normalization to 20*20 pixels. In order to select the effective negative examples for targets detection, the bootstrapping method is used for selecting negative samples, that is, the false accept samples are later served as negative samples during the training process. In detection process, face and non-face image regions can be discriminate according to Eq.(1).
Robust Precise Eye Location by Adaboost and SVM Techniques
95
2.2 Detect Eye Candidates We extend the above approach to build an eye candidates detector. There are total 7000 eye samples with the eye center being the center of the image and resize to 16*8 pixels. The negative samples are non-eye images, which are cropped from the face image due to eye searching area being restricted to the face. After the face region is located, we only search the upper area of the face region according to the prior knowledge about face model. The reduction of searching area can save the detection time and dramatically reduce the possible false eye candidates. Since the eye detector based on Haar-like Feature only encode local gray intensive information of the eye, it is difficult to train an appropriate threshold to discriminate eye and eye-like images in the face regions such as eyebrows, thick frames of glasses, etc. So in this step we set low threshold to avoid false reject according to Eq. (1), that is, let eye candidates detector be with a relative high false accept rate and low false reject rate. 2.3 Precise Eyes Location Based on Eye-Pair SVM Post Classifier Just as mentioned above, there are still some false eyes which cannot be discarded by hierarchical Adaboost classifier. Thus we propose a method that use SVMs technology to construct the eye-pair classifier based on geometrical model of face. Firstly, eye candidates are grouped into eye-pair candidates. Secondly, the true eye-pair can be selected from the eye-pair candidates by using the SVMs classifier. Finally, according to classifying results we can obtain the position of eye center. Due to the geometrical relation information encoded by SVMs, the disadvantage of eye detector based on Haar-like features can be overcome. 2.3.1 Geometrical Features and Training Data The results of prior detection process are a set of eye candidates’ rectangles and associated face rectangles. We assume the detected face rectangle is Wf×Hf pixels and randomly selecting two detected eye candidate rectangles are Wi×Hi pixels and Wj×Hj pixels respectively. We define Wij, Hij as the horizontal and vertical distance of center of two eye candidates respectively. These distances can be easily calculated based on the prior results of detection. The geometrical relations are shown in Fig.2 As we know, the sizes of two eyes are similar in the front-view face. The positions of two eyes in the face are symmetry. The direction and distance of the line joining the centers of both eyes are also very useful cue to select the true eye-pair. According to these prior information about geometrical face model, we choose Li/Hf, Lj/Hf, Wij/Wf, Hij/Hf, Hij/Wij, Wi/Wf, Hi/Hf, Wj/Wf, Hj/Hf, Lif/Wf, Ljf/Wf, |Wi-Wj|/Wf, |HiHj|/Hf total 13 measurements for eye-pair classifying features. For each image in the training set, we apply the following steps to get eye-pair SVM training data: 1. Applying Adaboost face and eye finder to the image, obtained a face rectangle Rf and a set of rectangles of eye candidates Rec
96
Xusheng Tang et al.
Fig. 2. Geometrical relation of Eye-pair Candidates
2. Selecting two eye candidates as eye-pair candidate Feye-pair , j 3. For each eye-pair candidates and associate Rf, calculate 13 measurements of eyepair classifying features and store them into a feature vector xeye-pair, j. Manually labeling each xeye-pair, j , obtaining labeled samples (xeye-pair, j , yj) A training set of 400 images has been selected from the above-mentioned database. The 400 position samples and 2672 negative samples are obtained using this method. 2.3.2 Support Vector Machines Let labeled samples (xi,yi) ,where xi°R13, yi°{±1} and the kernel function K. SMO[9] is used as training algorithm. The Support Vectors sj, the weight ¢j and threshold b can be obtained by training. Then the decision function can be written as: f ( x ) = sign( ∑ α j y j K ( s j , x ) + b) .
(2)
where x is the input object to be classified. As the numbers of negative samples are much more than positive samples during the training process, SVMs showed a tendency towards false rejection (FR) rather than false acceptance (FA). In order to compensate for that, we modify Eq.(2) with the introduction of a threshold . Then the decision function is written as follows: 1 ( ∑ α j y j K ( s j , x ) + b) ≥ θ f ( xi ) = . − 1 otherwise
(3)
By varying we can adjust the performance of the SVMs. In this paper, we set obtain equal error rate (EER) that is FA=FR.
to
2.3.3 Precise Eyes Location by SVM Eye-Pair Post Classifier In practical detection, after eye candidates have been extracted by face and eye finder, any two eye candidates are grouped as eye-pair candidates. Then eye-pair geometrical features vectors can be obtained by the method described in above section. According to Eq. (3), the position of the eye-pair’ eye-center classified as 1 is considered as the position of the eye center of the face. If more eye-pair candidates are classified as 1, then the average position of theirs is considered as the eye center.
Robust Precise Eye Location by Adaboost and SVM Techniques
97
3 Experimental Results Two open face databases, BioID [12] and JAFFE [13], are used in our experiments. The BioID database consists of 1521 images. The JAFFE database consists of 213 images. A relative error measure [10] is used to evaluate the precision of eye localization. Let dl, dr be Euclidean distance between the detection left(right) eye position and the marked left(right) eye position respectively, dlr be Euclidean distance between the marked left and right eye. Then the relative error is defined as follows: err = max(d l , d r ) / d lr .
(4)
Our system is tested on AthlonXP 2500+ PC with 512MDDR memory. We adopt two criterion measures, err
