Hybrid Technique for Human Face Emotion ... - Semantic Scholar

(IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 1, No. 6, December 2010

Hybrid Technique for Human Face Emotion Detection Renu Nagpal, Pooja Nagpal

Sumeet Kaur

M.Tech Student,CE deptt. Yadavindra College of Engineering Guru Khashi Campus Talwandi Sabo, Bathinda, (Punjab)

Assistant Professor,CE deptt. Yadavindra College of Engineering Guru Khashi Campus TalwandiSabo,Bathinda,(Punjab)

Abstract—This paper presents a novel approach for the detection of emotions using the cascading of Mutation Bacteria Foraging optimization and Adaptive Median Filter in highly corrupted noisy environment. The approach involves removal of noise from the image by the combination of MBFO & AMF and then detects local, global and statistical feature form the image. The Bacterial Foraging Optimization Algorithm (BFOA), as it is called now, is currently gaining popularity in the community of researchers, for its effectiveness in solving certain difficult real-world optimization problems. Our results so far show the approach to have a promising success rate. An automatic system for the recognition of facial expressions is based on a representation of the expression, learned from a training set of pre-selected meaningful features. However, in reality the noises that may embed into an image document will affect the performance of face recognition algorithms. As a first we investigate the emotionally intelligent computers which can perceive human emotions. In this research paper four emotions namely anger, fear, happiness along with neutral is tested from database in noisy environment of salt and pepper. Very high recognition rate has been achieved for all emotions along with neutral on the training dataset as well as user defined dataset. The proposed method uses cascading of MBFO & AMF for the removal of noise and Neural Networks by which emotions are classified. Keywords-biomertics;adaptive median filter; bacteria foraging optimization;feature detection;facial expression

I.

FACIAL EXPRESSION RECOGNITION

Biometric is the science and technology of recording and authenticating identity using physiological or behavioral characteristics of the subject. A biometric representation of an individual. It is a measurable characteristic, whether physiological or behavioral, of a living organism that can be used to differentiate that organism as an individual. Biometric data is captured when the user makes an attempt to be authenticated by the system. This data is used by the biometric system for real-time comparison against biometric samples. Biometrics offers the identity of an individual may be viewed as the information associated with that person in a particular identity management system. Automatic recognition of facial expressions may act as a component of natural human machine interfaces Such interfaces would enable the automated provision of

services that require a good appreciation of the emotional state of the service user, as would be the case in transactions that involve negotiation[1], for example Some robots can also benefit from the ability to recognize expressions . Noise is added as unwanted variations in the image when image is transmitted over the network [22]. It causes a wrong conclusion in the identification of images in authentication and also in pattern recognition process. Firstly there should be the removal of noise from the image then features are detected. Noise in imaging systems is usually either additive or multiplicative. In practice these basic types can be further classified into various forms such as amplifier noise or Gaussian noise, Impulsive noise or salt and pepper noise, quantization noise, shot noise, film grain noise and nonisotropic noise. However, in our experiments, we have considered only salt and pepper impulsive noise. . II.

RELATED WORK

Automated analysis of facial expressions for behavioral science or medicine is another possible application domain. From the viewpoint of automatic recognition, a facial expression can be considered to consist of deformations of facial components and their spatial relations, or changes in the pigmentation of the face[1]. There is a vast body of literature on emotions. Recent discoveries suggest that emotions are intricately linked to other functions such as attention, perception, memory, decision making, and learning. This suggests that it may be beneficial for computers to recognize the human user’s emotions and other related cognitive states and expressions. Ekman and Friesen [1] developed the Facial Action Coding System (FACS) to code facial expressions where movements on the face are described by a set of action units (AUs). Ekman’s work inspired many researchers to analyze facial expressions by means of image and video processing. The AAM approach is used in facial feature tracking due to its ability in detecting the desired features as the warped texture in each iteration of an AAM search approaches to the fitted image. Ahlberg [7] use AAM in their work. In addition, ASMs - which are the former version of the AAMs that only use shape information and the intensity values along the 91 | P a g e

http://ijacsa.thesai.org/

(IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 1, No. 6, December 2010

profiles perpendicular to the shape surface are also used to extract features such as the work done by Votsis et al. [9]. Many algorithms have been developed to remove salt/pepper noise in document images with different performance in removing noise and retaining fine details of the image, like: Simard and Malvar [10] shows image noise can originate in film grain, or in electronic noise in the input device such as scanner digital camera, sensor and circuitry, or in the unavoidable shot noise of an ideal photon detector. Beaurepaire et al. [2] tells the identification of the nature of the noise is an important part in determining the type of filtering that is needed for rectifying the noisy image. Noise Models from Wikipedia [11] shows the noise in imaging systems is usually either additive or multiplicative. Image Noise [12] shows in practice these basic types can be further classified into various forms such as amplifier noise or Gaussian noise, Impulsive noise or salt and pepper noise, quantization noise, shot noise, film grain noise and nonisotropic noise. Al-Khaffaf [13] proposes several noise removal filtering algorithms. Most of them assume certain statistical parameters and know the noise type a priori, which is not true in practical cases. Prof. K. M. Passino [8] proposed an optimization technique known as Bacterial Foraging Optimization Algorithm (BFOA) based on the foraging strategies of the E. Coli bacterium cells. Until date there have been a few successful applications of the said algorithm in optimal control engineering, harmonic estimation in Ref [15], transmission loss reduction in Ref [16], machine learning in Ref [14] and so on. Its performance is also heavily affected with the growth of search space dimensionality Kim et al [17] proposed a hybrid approach involving GA and BFOA for function optimization. Biswas et al [18] proposed a hybrid optimization technique, which synergistically couples the BFOA with the PSO. Up to our knowledge this is the first time we are using this hybridized technique for the detection of emotions in noisy environment. However, in our experiments, we have considered only salt and pepper impulsive noise. 2.1 Organization of the Paper The rest of the paper is organized as follows: Section 3 describes general set up of proposed technique, experimental results have been discussed with respect to percentage of correct recognition considering JAFFE facial image database under salt and pepper noisy environment in section 4 and comparative analysis of the proposed technique with existing techniques at the end of section 4. The paper is concluded with some closing remarks and future scope in section 5. III.

THE GENERAL SET UP

The design and implementation of the Facial Expression Recognition System can be subdivided into three main parts:  The first part is Image Pre-processing.

 Second part is a Recognition technique, which includes Training of the images.  Third part is testing and then there is result of classification of images. A. Image Preprocessing The image processing part consists of image acquisition of noisy image. Filtering, Feature Extraction, Region of Interest clipping, Quality enhancement of image. This part consists of several image-processing techniques. First, noisy face’s image acquisition is achieved by scanner or from JAFFE database by introducing the noise in the images then adaptive median filter is used to remove noise from the image then Mutation Bacteria Foraging Optimization Technique is used to remove noise that is remained there after using median filter and finally features are extracted. The region of interest is eyes, lips, mouth (eyes and lips) that are independently selected through the mouse for identification of feature extraction. Statistical analysis is also done by that mean, median and standard deviation of the noisy frame, restored frame, cropped frame and enhanced frame are calculated. The significance of these shows that mean and standard deviation should be less in enhanced frame as compared to noisy frame. These extracted features of image are then fed into Back-Propagation and Radial Basis Neural Network for training and then emotions are detected. B. Training of Neural Network The Second part consists of the artificial intelligence, which is composed by Back Propagation Neural Network and Radial Basis Neural network. First training of the neurons is there and then testing is done. Back-propagation and RBF algorithms are used in this part. It consists of two layers. At input to hidden layer Back-propagation Neural Network is used which consists of feed forward and feed backward layers and at hidden to output layer, RBF Neural Network is used for classification of expressions.  As the recognition machine of the system, a three layer neural network has been used that is trained with several times using hit and trial method on various input ideal and noisy images forced the network to learn how to deal with noise. The window size is of 9.We can increase the size of window with that computation time is also increased. The variation in the density of noise is taken from 0.05 to 0.9.  The combination of adaptive median filter and Mutation BFO removes noise up to 90% from the image with more accuracy and the learning ranges from 0.1 to 0.9. Main accuracy or goal is 0.01, for that it takes more computation time. For single image total of 10 iterations are needed for zero error but for 21 images the error is zero in total of 15000 epochs because there are different images and different types of motions so more iterations are there. C. Testing The third phase consists of testing of expressions that shows the percentage of accurate results and result of classification for different expression.

92 | P a g e http://ijacsa.thesai.org/

(IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 1, No. 6, December 2010

 A special advantage of the technique is that the expression is recognized even there is more noise density in the image up to 0.9. The range of density is taken from 0.05 to 0.9. Second by taking statistical features like mean, median and standard deviation the classification is easy and there will be more correctness to recognize the facial expression. Third, dual enhancement of image is there, first at the removal of noise by Adaptive Median Filter and Mutation BFO and second by using histogram equalization.  The flowchart and Implementation Overview of Facial Recognition System with proposed method is shown in figure 1 and in figure 2.

Input Frame affected with salt and pepper noise

Adaptive Median Filter to Remove Salt and Pepper Noise

Mutation based Bacterial Foraging Optimization

Start

Noisy Input Frame

Feature Extraction Stage Region of Interest Eyes + Lips

Adaptive Median Filter

Mutation Based Bacterial Foraging Optimization

Feature Extraction stage Eyes/Lips/Face

Eyes

Quality Enhancement by Histogram Equalization

Lips Modified Back Propagation Radial Basis Neural Network

Angry Classi ficatio n

Happy Fair Normal

Quality Enhancement by Histogram Processing Implementation Overview of Facial Expression Recognition System RBF Neural Network

Classification

Flowchart of Facial Expression Recognition System

Block Diagram

A.

Input Noisy Image The first module shows the input phase. To this module a noisy face image of salt and pepper noise is passed as an input for the system. The input image samples are considered from JAFFE database. The input image is randomly picked up from the database used for training and evaluated for the recognition accuracy. B.

Adaptive Median Filter A median filter is an example of a non-linear filter [20] .It is very good at preserving image detail. To run a median filter: 1. Consider each pixel in the image 2. Sort the neighboring pixels into order based upon their intensities 3. Replace the original value of the pixel with the median value from the list  Algorithm of Adaptive Median Filter 1. Initialize w = 3 and Zm = 39. 2. Compute Z min, Z max and Z med 3. If Zmin < Zmed < Zmax, then go to step 5. Otherwise w = w+2. 4. If w ≤ Zm, go to step 2. Otherwise, replace Zxy by Z med 5. If Zmin< Zxy< Zmax, then Zxy is not noisy pixel else replace Zxy by Zmed. where,

93 | P a g e http://ijacsa.thesai.org/

(IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 1, No. 6, December 2010

Z = Noisy image. Zmin = Minimum intensity value. Zmax = Maximum intensity value. Zmed = Median of the intensity values. Zxy = Intensity value at coordinates (x, y). Zm = Maximum allowed size of the adaptive median filter window.  A median filter is a rank-selection (RS) filter, a particularly harsh member of the family of rank-conditioned rank-selection (RCRS) filters, a much milder member of that family, for example one that selects the closest of the neighboring values when a pixel's value is external in its neighborhood, and leaves it unchanged otherwise, is sometimes preferred, especially in photographic applications. Median filter is good at removing salt and pepper noise from an image, and also cause relatively little blurring of edges, and hence are often used in computer vision applications. C.

Mutation Bacteria Foraging Optimization During the first stage the input image corrupted with a saltand-pepper noise of varied densities from 0.05 to 0.9 is applied to the adaptive median filter. In the second stage, both the noisy and adaptive median filter output images are passed as search space variables in the BFO technique [20] to minimize errors due to differences in filtered image and noisy image.  Bacterial Foraging Optimization with fixed step size suffers from two main problems I. If step size is very small then it requires many generations to reach optimum solution. It may not achieve global optima with less number of iterations. II. If the step size is very high then the bacterium reach to optimum value quickly but accuracy of optimum value gets low. Similarly, in BFO, chemotaxis step provides a basis for local search, reproduction process speeds up the convergence, elimination and dispersal helps to avoid premature convergence. To get adaptive step size, increase speed and to avoid premature convergence, the mutation by PSO is used in BFO instead of elimination and dispersal event by equation 1.

 ( j  1, k )   ( j  1, k )  *r1 * C1 ( ( j  1, k )   global) i

i

i

………1

 i ( j, k )

= Position vector of i-th bacterium in j-th chemotaxis step and k-th reproduction steps.

 global=Best position in the entire search space

In-preprocessing step of hybrid soft computing technique, adaptive median filter is used to identify pixels that are affected by noise and replacing them with median value to keeps the uncorrupted information as far as possible. The BFpfPSO follows chemotaxis, swarming, mutation and reproduction steps to obtain global optima. The algorithm of BF-pfPSO is presented below.

 Step by step algorithm of mutation based BFO Initialize Parameters p, S, Nc, Ns, Nre, Ned, Ped and C (i), i= 1, 2... S. Where, p = Dimension of search space S = Number of bacteria in the population Nc = Number of chemotaxis steps Ns = Number of swimming steps Nre = Number of reproduction Steps Pm = Mutation probability C (i) = Step size taken in the random direction specified by the tumble i (j, k)= Position vector of the i-th bacterium, in j-th chemotaxis step, in k-th reproduction step and in l-th elimination and dispersal step Step 1: Reproduction loop: k = k+1 Step 2: Chemotaxis loop: j = j+1 a) For i= 1,2…S, take a chemotaxis step for bacterium i as follows b) Compute fitness function J (i, j, k, l) c) Let Jlast =J (i, j, k,) to save this value since we may find a better cost via a run. d) Tumble: Generate a random vector

(i)   p with each

element  m (i ) , m = 1, 2…p, a random number on [-1 1] e) Move: Let

 i ( j  1, k )   i ( j, k )  C (i)

(i) T (i)(i)

f) Compute J (i, j+1, k, l) g) Swim i) Let m =0 (counter for swim length) ii) While m < Ns (if have not climbed down too long) o Let m = m+1 o If J (i, j+1, k, l) < Jlast (if doing better), Let Jlast = J (i, j+1, k, l) and let

 i ( j  1, k )   i ( j  1, k )  C (i)

(i)

 (i)(i) T

And use

this  ( j  1, k ) to compute the new J (j+1, k). o Else, let m = Ns. This is the end of the while statement. i

h) Go to next bacteria (i+1) if i S Step 3: Update

 pbest ( j, k )

and  global . If j