recognition and detection of occluded faces by a neural network

RECOGNITION AND DETECTION OF OCCLUDED FACES BY A NEURAL NETWORK CLASSIFIER WITH RECURSIVE DATA RECONSTRUCTION T. Kurita, M. Pic Neuroscience Research Institute, AIST takio-kurita,mickael.pic
@aist.go.jp

ABSTRACT This paper describes how to improve the robustness to occlusions in face recognition and detection. We propose a neural network architecture which integrates an auto-associative neural network into a simple classifier. The auto-associative network is employed to recall the original face from a partially occluded face image and to detect the occluded regions in the input image. The original face can be reconstructed by replacing those regions with the recalled pixels. By applying this reconstruction process recursively, the integrated network is able to classify occluded faces robustly. To confirm the effectiveness of this method, we performed experiments on face image classification and face detection. It is shown that the classification performance is not decreased even if  - of the face images is occluded. 1. INTRODUCTION Face images taken in a general situation are often occluded by uninteresting objects (e.g., sunglasses). To compensate degradations cased by such occlusions, the face recognition system has to have prior knowledge about faces, namely knowledge what faces looks like [1]. Once the system learns the prior knowledge from the given training samples, occluded regions in a face image can be automatically detected by comparing the prior knowledge with the input face image. Also the system can fill in the missing information using the prior knowledge. If the system has such ability for occlusions, it is expected that the recognition performance will be improved and the applicability of the system will be extended. One of the methods to realize such abilities is to incorporate auto-associative memory into the classifiers. The auto-associative memory can recall the whole image from its partial image[2]. We can use the recalled image to discriminate occluded pixels as outliers from those belonging to a face. It is also possible to reconstruct the original face by replacing pixel values in the occluded regions with the recalled pixel values. In recent studies regarding view-based pattern recognition, dimensionality reduction techniques such as Prin-

T. Takahashi Ryukoku University [email protected]

cipal Component Analysis (PCA) are often employed before classification. The eigenface method is typical[3]. We can consider that such methods have a feed-forward computation architecture. In contrast to this, we adopt an architecture which performs recursive computation. Reconstruction processes by using an auto-associative memory are repeated to modify the input data recursively. By replacing pixel values in the occluded regions with the recalled pixel values, dimensionality-reduced feature components are extracted without being affected by pixel values in occluded regions. Therefore, we can improve the robustness of classification against occlusions. We employ a Multi-Layer Perceptron (MLP) to implement the above function of auto-associative memory. A three-layer Perceptron can be used as auto-associative memory when an identical number of units are given in the input and the output layers and the Perceptron is trained to map each input vector onto itself. In the case that the hidden layer has a smaller number of units than the other layers, it is known that the Perceptron performs dimensionality reduction which is equivalent to that by PCA[4]. Hence, this approach has an advantage in which it can replace PCAbased methods without difficulty. In the next section, we describe how to use a MLP as auto-associative memory. We also introduce a method to detect pixels in occluded regions and to replace those values with estimated values by means of this auto-associative memory[5]. In Section 3, they are combined with a simple classifier model in order to make a robust classifier[6]. Section 4 shows some experimental results of the application of our method to face recognition and face detection. 2. AUTO-ASSOCIATIVE NETWORK AND RECURSIVE DATA RECONSTRUCTION This section introduces an auto-associative network which performs dimensionality reduction and reconstruction of input data. A Multi-Layer Perceptron(MLP) is used as the auto-associative network. Then we describe a computation process called recursive data reconstruction. It plays a crucial role in improving the robustness against occlusions us-

z

1

y x

... 1

M

W ...

y

H

U ...

1

zi

z

x^

M

x^ i βi

x

(a) A three-layer Perceptron

+

1-β i

xi

(b) Data reconstruction

Fig. 1. MLPs for auto-associative memory. ing the auto-associative network. 2.1. MLP for Auto-associative Memory Let us consider a MLP with one hidden layer as shown in Fig. 1. The MLP has units in both the input and the output layers, respectively, and units in the hidden layer. It is assumed that all the units have a linear activation function and  . Let  be a given input vector, and let  denote a weight vector connecting the input layer and the  -th hidden layer unit. The output of the  -th hidden layer unit  then becomes  

!"$#&%(')')')%* ,+-'

(1)

The output of the . -th output layer unit /0 is also given in the same way as /1023 054

6.78#%)')'('-%9 :+ ,



F?

C



(3)

K

4 K$F .

(4)

When this MLP is trained to approximate each input data by using its output, it can perform as auto-associative memory. For a given set of training data N1POQ:DRS?+  @ a weight vector connecting the hidden layer and the . -th output layer unit. Denoting the output of the MLP as C weight matrices as F8 D/G 5J  and KL 3E