Unlocking Mobile Devices using Improved Face Recognition and Eye

International Journal of Applied Engineering Research ISSN 0973-4562 Volume 13, Number 24 (2019) pp. 16907-16909 © Research India Publications. http://www.ripublication.com

Unlocking Mobile Devices using Improved Face Recognition and Eye Blinking Technique Dr. K. Arutselvan1, Dr. Sridhathan. C2, Dr. M. Senthil Kumar3 Associate Professor, Department of Electronics and Communication Engineering, Nalla Malla Reddy Engineering College, Hyderabad – 500 088, India. 1,2,3

Abstract In recent days, face recognition with eye blink counting techniques are very popularly used in smart phone platform for unlocking the devices. In conventional face recognition system some drawbacks are there such as photograph image of a original face can be used for unlocking the device. So it leads to insecure operation and leak of confidential information without the owner’s knowledge. In this paper, face detection with eye blink counting based recognition methods for unlocking the mobile devices is discussed. Keywords: Face detection, Eye blinking, Face recognition

INTRODUCTION Nowadays smart mobile devices are preloaded with some social networks application and software associated with the manufacturers. User also installs application software for mobile banking, UPI banking, internet banking and for some other applications. But sometime mobile phones are stolen and personal data like contacts, e-mails, photos, videos and confidential data are used for fraudulent money transaction. To prevent the unauthorized access, user protects his mobile device with a password or face recognition techniques. Face recognition techniques has many applications in various field such as mobile phones, biometric surveillance, banking sector, retail stores and in airports for reducing the crime and violence. All face recognition technique, starts with face detection and recognition components. Many researchers are proposing improved face recognition technique including the eye blink counting component for unlocking the mobile devices. Therefore in this paper, we give an brief discussion about face recognition with eye blink counting technique for preventing the fraudulent person access on the mobile phones.

LITERATURE SURVEY In many vigilance operations such as drowsiness detection, eye blink monitoring help in preventing adverse effect. In some cases, the computer warns the user when they stare the screen without blinking to prevent the dry eye syndrome. Several eye motion detection techniques are using Viola-Jones type detector. Correlation matching is performed in open and closed eye templates for active shape eye modelling [1].

There are many biometric methods for personal identification such as finger print, iris scan which are gaining acceptance from the public. Facial recognition is considered to be passive and non-intrusive method for authentication. Face recognition technique is similar to the way human beings recognise each other. Face recognition is sometime insecure, since the people involved may be unaware of being captured [2]. Realtime robust techniques use facial landmarks such as face image, eye corner and eye lids. Eye blinking is partially subconscious action of closing and opening of the eyelid [1]. With the use of camera to detect the human face with eye movement, eye open and close activity made the human machine interface easy. Disabled persons can use the eye movement for driving the wheelchairs[3]. Face detection system using polynomial neural classifier with different architecture and parameter values are used for effective face detection [4]. In some special cases, communication is done mainly by the eye movement where the computer system captures the eye movement, blinks and replaces them with mouse movement [5]. According to physiology, facial muscles such as, lateral frontalis, corrugator levator palpebrae superioris, supercilia and orbicularis oris are responsible for facial feature changes like eye blink, lip suck, eyebrow raiser etc [6]. Consumers prefers mobile phones for all shopping and online transactions. Users protects the mobile phones by passwords or non-biometric authentication which are not reliable. Biometric features are unique for each and every individuals, they are most reliable in the aspect of security. The camera was fixed and the eyes were detected and its movements were tracked [1]. Eye features were compared with the database. But the eye resolution was not good and the face detection was less accurate [7]. Haar feature based face detection is a machine learning approach and cascaded function derived from positive and negative shades of the image [8]. Human eye normally blinks 14 to 16 times per minute and takes 300 to 400 milliseconds time in between each blinking [9, 10]. In this proposed method, face recognition with eye blink count detection are used for authenticating the mobile user. Proposed method has low computation complexity and high precision for mobile user authentication.

16907

International Journal of Applied Engineering Research ISSN 0973-4562 Volume 13, Number 24 (2019) pp. 16907-16909 © Research India Publications. http://www.ripublication.com METHODOLOGY

RESULTS

The proposed eye blinking with face recognition based mobile unlocking technique consists of five phases, which are 1) face detection 2) face data analysis 3) face recognition 4) eye detection and 5) eye blink detection. First and foremost step is detecting the face by using the front camera of the mobile phones. The detected face is matched with the face samples which are stored in the data base. Matched face will be authenticated by mapping eye region and eye blinks. Eye region is primarily detected and the eye blink are counted.

In this proposed method, detection of eye and eye blink is performed on persons without spectacles. The results are evaluated using the recall curve which trial and error basis. The front camera of the mobile phone is used for face detection. Test was performed with the light intensity ranging from 100 to 1000 lumens. Eye blinking is tested for a single participant to study and evaluate the performance of the proposed method. The results of the proposed method was compared with the traditional methods and found to be faster unlocking rate with high success rate.

Face Detection Android Open CV is used for face detection and recognition function. Open CV uses Haar feature which reduces the execution time. Eigenface, Fisher Face image and Nth Nearest Neighbour models are used for comparing the image points.

Eye Detection After the face image detection we locate the eye part using the binarization where the pupil of the eye is identified. Static Recognition method is used for finding the eye open or close status. Two geometric functions are used for locating the eyes in the face image. Height (H) and Width (W) of face image are used for locating the eyes. Normally, eyes will be positioned at 2/5H to 4/5H vertically from the bottom of the chin. Left eye is located 4/7W to W from left boundary of the human face and right eye located between 0 to 3/7W as shown in Figure 1. After detecting the eye location, we use the horizontal edge projection analysis to identify the edge.

Proposed method was capable for detecting the face at different angle. The performance evaluation is given in the Table 1 and shown in Figure 2.

Table 1: Performance of Proposed Method Methods/ Face Correct Unsuccessful Performance Parameter Detection Detection Detection Counting 15 11 04 73.3% Skin Colour Model 15 12 03 80% Adaboost Method 15 14 01 93.3% Proposed Method

Proposed Method Adaboost Skin Colour 0

20

40

60

80

100

Success Rate of Detection

Figure 1: Geometry measurement of human face and eye detection

Figure 2: Graph for Success Rate of Detection Execution time for face detection took about 8 milli seconds and duration for eye blink was about 18.10 mill seconds. The comparison is shown in the Table 2 and shown in Figure 3. The face detection with eye open and eye closed are shown in the Figure 4.

Eye image is obtained by using the Equation 1. 𝑅(𝑥 ′ , 𝑦 ′ ) =

∑𝑥′,𝑦′ 𝑇(𝑥 ′ ,𝑦 ′ )𝐼(𝑥+𝑥 ′ ,𝑦+𝑦 ′ ) √∑𝑥′,𝑦′ 𝑇(𝑥 ′ ,𝑦 ′ )2 𝐼(𝑥+𝑥 ′ ,𝑦+𝑦 ′ )2

(1)

where ‘T’ is the templates of the image, x and y are the image coordinates. Correlation function is used to find the state of the eye. It matches with the stored templates of eye open to find the time of each eye blink. For executing the proposed method, we need Windows 7 operating system or higher, Java (Eclipse) or higher version and Android open CV software.

16908

Table 2: Processing Time Methods

Single Face Sequence in milli second Skin Colour Model 21.7 Adaboost 19.9 Proposed Method 18.1

International Journal of Applied Engineering Research ISSN 0973-4562 Volume 13, Number 24 (2019) pp. 16907-16909 © Research India Publications. http://www.ripublication.com Han, Seongwon, et al., “Eye guardian: a framework of eye tracking and blink detection for mobile device users,” in Proc. of the Twelfth Workshop on Mobile Computing Syst. & Applications, ACM, 2012. [4] Lin-Lin Huang, Akinobu Shimizu, Yoshihiro Hagihara, Hidefumi Kobatake, “Face detection from cluttered images using a polynomial neural network”, Neurocomputing 51 (2003) pp.197 – 211. [5] Emmanuel Jadesola Adejoke, Ibiyemi Tunji Samuel, “Development of Eye-Blink and Face Corpora for Research in Human Computer Interaction”, International Journal of Advanced Computer Science and Applications, Vol. 6, No. 5, 2015. [6] M. Valstar, M. Pantic, Z. Ambadar, and J. Cohn. Spontaneous vs. posed facial behavior: Automatic analysis of brow actions. In Int. Conf. on Multimodal Interfaces, pages 162– 170, 2006. [7] Viola, Paul, and Michael J. Jones, “Robust real-time face detection,” International Journal of Computer vision, vol. 57.2, pp. 137-154, 2004. [8] Viola, Paul, “Feature-based recognition of objects,” in Proceeding. of the AAAI Fall Symposium on Learning and Computer Vision, 1993. [9] Lee, Won Oh, Eui Chul Lee, and Kang Ryoung Park, “Blink detection robust to various facial poses,” Journal of Neuroscience Methods, vol. 193.2, pp. 356-372, 2010. [10] Torricelli, Diego, et al, “An adaptive blink detector to initialize and update a view-based remote eye gaze tracking system in a natural scenario,” Pattern Recognition Letter, vol. 32.12, pp. 11144-1150, 2009. [3]

Proposed Method Adaboost Skin Colour

16

18

20

22

Processing time in ms

Figure 3: Graph for Execution time

(a)

(b)

(c)

Figure 4: a) Face Detection b) Eye open and c) Eye Closed Detection

LIMITATIONS Atmospheric effects and lighting conditions affect the performance of the proposed system. Proposed system is capable of detecting the face and eye at a standard room condition. Proposed system is capable of detecting the face during the facial expression such as laughing with more execution time and less accuracy. It cannot be used for the person who are wearing spectacles or sunglass.

CONCLUSION The usage of smart mobile phones has increased among the people and it is used for many confidential data transactions. To provide privacy and data security, we proposed face detection with eye blinking recognition and screen unlock technique. This system is low cost, user friendly and has no complicated procedures to authenticate the user.

REFERENCE [1]

[2]

Tereza Soukupova and Jan Cech, “Eye Blink Detection Using Facial Landmarks”, 21st Computer Vision Winter Workshop, Rimske Toplice, Slovenia, February 3–5, 2016. Emmanuel Jadesola Adejoke, Ibiyemi Tunji Samuel, “Development of Eye Blink and Face Corpora for Research in Human Computer Interaction”, International Journal of Advanced Computer Science and Applications, Vol. 6, No. 5, 2015.

16909