Gesture Control Using Single Camera for PC - Core

Available online at www.sciencedirect.com

ScienceDirect Procedia Computer Science 78 (2016) 146 – 152

International Conference on Information Security & Privacy (ICISP2015), 11-12 December 2015, Nagpur, INDIA

Gesture Control Using Single Camera For PC N.LALITHAMANI a a

Assistant Professor (SG) - Department of Computer Science and Engineering, Amrita School of Engineering, AMRITA VISHWA VIDYAPEETHAM (University), Coimbatore, India

Abstract In the past decade, more stress has been given to user interface since technology is made available to every human being. This includes advancement of touch technology over buttons and many more gesture control mechanisms. These technologies enable even a laymen to better use the day-to-day technologies like cell phone and personal computer easily with navigation. This work focuses on one such technology that can be implemented on a PC (Personal Computer). We use single web camera as input device to recognize gestures of hand. Some of these gestures include controlling the mouse cursor, the clicking actions, and few shortcuts for opening specific applications. This w o r k includes Face Recognition model as an replacement for default lock screen. The Face Recognition model uses Viola and Jones method for detection of face and PCA (Principal Component Analysis) for recognition and identification of algorithms. © 2016 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license © 2016 The Authors. Published by Elsevier B.V. (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-reviewunder underresponsibility responsibility organizing committee of the ICISP2015. Peer-review ofof organizing committee of the ICISP2015

Keywords: Gesture recognition; Face recognition; PC ___________________________________________________________________________________________

1. Introduction Face recognition system is one of the applications of a computer for verifying and identification of a person. This field is related to biometrics application which includes others like fingerprint, eye iris recognition systems. Face Recognition systems are based on an very old era of 2D algorithms, taking us back to the early 1960s. The first face recognition methods used the geometry of key points. .Hand Tracking is technique used to move and control mouse cursor without using an external mouse device. Our paper proposes a basic model of integrating Face Recognition method with Hand Tracking where the authenticated user alone can access their system without using mouse thereby providing access privilege to the user. Hand Tracking is been carried out with either Edge/Shape

1877-0509 © 2016 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of organizing committee of the ICISP2015 doi:10.1016/j.procs.2016.02.024

N. Lalithamani / Procedia Computer Science 78 (2016) 146 – 152

detection or Gesture Recognition or Blob Detection for past one decade. In this paper we propose an efficient Blob Detection method by changing threshold accordingly to light illumination to identify the active regions more clearly. In Face Recognition, camera, a type of sensor plays an active role on capturing these images. One way of doing this is a comparison between a source image and target image which is stored in database. Since we are going to use videos as input and verification it is on a greater advantageous module due to repeated textures, frames at different angles. There are different ways to recognize a face. But in this work, we extract Eigen values from an image that is converted to Eigen faces which is composed of Eigen features. The advantage of extracting Eigen values is that it gives much accurate results than any other method. This method is called Viola and Jones which is named after them. The advantage of using Viola and Jones is that it does not require full frontal pose of a face. It uses cascading objects instead of measuring the ratio of eyes and nose. Since Eigen feature gives values of different points from a face, it can be later used to compare with query face. When such comparison is carried out only faces that matches most of the Eigen feature with other face, will be recognized. Blob method is then used to carry out hand tracking method for analyzing cursor movement on screen. Only the recognized person is allowed to control that system. The advantage of using Blob method is that it uses differential effective result.

2. Problem Definition We propose a multi-view recognition that is insensitive to pose variations. The usage of single view has serious disadvantages like information loss, data insufficiency etc., during the database entry of an image. So, target image formation is greatly affected. On usage of multi-view by multiple cameras these are reduced and pose variations can be included into consideration. The usage of multiple cameras overcomes the challenges of single camera. Hence, face recognition can be achieved in any angle. In normal face recognition, face`s pose plays a vital role, a small change in angle will make it worse by not detecting it but this can be alleviated. Since, video is used as input and data storage in the system we can obtain different images in varied orientations at a given point of time. The main factors that still affect pose orientations are resolution and calibration of the camera. Irrespective of any angle of the face being exposed, this software has to authenticate. The transformations that take place in the video has to be mapped to the sphere model that has been created. Texture mapping is used for mapping the Eigen features. Once the Eigen features of the face is extracted by using Viola and Jones method, Principal Component Analysis method is used to compare between the faces in different images. The mouse control then comes into play where the recognized user accesses the mouse using fingers. Blob method is one way to do so by tracking the active regions. The active regions are defined by Euclidean distance from a particular point. The proposed work deals with various types of collated hand motion in real time on a machine with higher end results. This work is highly cohesive by latest technologies and advancements on track of humans. However, directly applying the techniques on track of humans is not so efficient because of various challenges focussed by. Henceforth careful switch over to increase the efficiency is required. The combination of Face Recognition and Hand Tracking technique together defines a system which provides security 3. Related Works Face Recognition is used in analyzing facial characteristics, storing featues in a database, using them to identify users. It helps in encoding the relationship between the facial features, to find structure features of face and set of images that capture the variants of faces that exist even when lightning conditions vary. Face recognition using Eigen values in which analysis of face is done and then it is converted to Eigen faces which are composed of Eigen vectors. The comparison of Eigen faces is used to identify the presence of face and identify it, which was well suited in controlled circumstances. Some disadvantages are lighting, angle,

147

148

N. Lalithamani / Procedia Computer Science 78 (2016) 146 – 152

distance. Next face recognition in humans plays an important role than doing it for computer. This includes some points like humans save images in low resolution images, degradation is avoided by increasing familiarity, eyebrows plays a key role in detecting. Single-view videos have been modelled by applying the Hidden Markov Models 1, or ARMA models 2 and Gaussian mixture model 3. A structural face construction and detection system is presented by Sankarakumar. The proposed system consists of different lightning, rotated facial image, skin colour etc. Identifying a person in a group by scaling and orientations of their actions carried out is widely focussed with the help of camera network with multiple views. The findings determined are collated and normalized by utilizing a minimum variance estimator and human movement track is obtained using a Kalman filter. Hence an optimized framework to track the right object in a registered camera network from publicly available resources are made available. They adapt and the features are mapped to the right subject with proper paths generated and obtained for each of the computed objects pertaining to the subject where recognition chain is possible irrespective of the invariants.

3. Motivation In traditional face recognition methods, no more accurate results are obtained. There exists no model where mouse control is secured, i.e. there were systems where once an administrator had accessed a system, anyone later can control the mouse irrespective of superuser’s boundary control. This may result in loss of various aspects of security. Hence to provide such initial restriction, we propose a model where only the responsible person who is authentic and authoritative could access the recognition system. To track the motions pertaining to hand, modelling of a point towards the conditions on the line are possible. The position of the point at any given point of time is instantaneous and being oriented by its position at the previous thread associated. Each reference has an attribute, say the field, which keeps changing with time depending on the position of the modelled point. In this framework, the field of reference is the only thing that is outlined and to be represented and projected probably to the outside world. It brings out similarity to the case of face videos irrespective of the invariant orientation styles of faces as the position of the point and the re-appearance of the face as the field of coincidence. The dependence of the appearance on the pose is analogous to that of the field of orientation on the accessibility. We propose a method that is vision oriented, hand gesture recognition system that can generalize over different users and operating modes, and show robustness under challenging visual settings. In addition to the general study of robust descriptors and fast classification schemes for hand gesture recognition, we are motivated by recent research showing advantages of gestural interface. Lot among the various interfaces and movements reported there are lot of strategic asssessment evaluation measures that produces acceptance testing results positive. Hence it had been proposed for a combined frame work of face recognition followed by virtual mouse control for users’ high level and secure access. This detection is based on colour channels. Hence different colours are adopted for various orientations and style. Here we use red colour for cursor movement, blue for double click, green for single click and so on. Detection of object and initializing them have produced or developed a system to be robust and also made object tracking simple. The model had been proposed for an efficient object detection mechanism. Earlier methods were not so efficient in producing optimized results or speedy recovery in the process flow. They also could not show dependencies on the path or navigate the path. 4. Proposed System In proposed system, the Face recognition model makes use of the distance between eyes and nose to detect the frontal face features. By applying such methods it does not allow for varying the posture in identification. When it comes to hand gestures, we can find shape of the hand and find variant gestures using the number of peak values and the distance between them. This method finds the background image and subtracts it with image where hand is introduced to find gesture. When implemented or on execution, we use a webcam which captures our body and face which are ought to move often making the default approach difficult to find the

N. Lalithamani / Procedia Computer Science 78 (2016) 146 – 152

shape of hand since there are many moving parts in our body other than hand.

Fig. 2. (a) Finger pre-processing ; (b) Finger peak detection

5. Innovative Content In Face Recognition module, for detecting the face we use Viola and Jones algorithm, which unlike many other detection algorithms does not need the frontal pose of the person. In the tracking part we find Eigen points on face and by considering the movement of these points in next frame, detection is rendered almost pose invariant. When coming to the mouse control module, we detect the coloured caps using RGB (Red Green Blue) channels, subtract them with grayscale and take binary image using appropriate threshold values. The binary image makes easy detection of Red, green, blue objects easily. The Red blob is detected using red channel as base and the centroid for that region is calculated. That centroid’s coordinate is mapped to the mouse cursor’s x and y coordinate. So, any object of bright red colour can be moved in front of the camera to control the cursor. Similarly we detect whether a blob exists using Blue and Green channels as base, if so they are mapped to enable double click and right click respectively. 6. Methodology For face detection module the cascade objects are used to find the presence of edge, line and region by moving these cascade objects pixel by pixel and finding the difference between average of intensity values between opposite regions. The same is continued with cascade objects of different sizes and various angles, the harr features found from doing the above steps are used to detect the face. After the face is being detected, Eigen features are found for the face, then the change in Eigen point position is utilized in detecting the face in next frame. The recognition part uses PCA(Principle Component Analysis). The face in training set are

149

150

N. Lalithamani / Procedia Computer Science 78 (2016) 146 – 152

converted to vector form and the average of these face vectors is found which gives the common features in training set faces. This average vector is subtracted from each face vector to get normalized vector form. The co-variance matrix c= A.(A transpose) is found and the desired Eigen faces are selected such that each face can be represented by sum of weight * Eigen faces. If two faces have similar set of weights then the face itself is considered similar. As far as the hand tracking part, the RGB(Red, Green, Blue) channels are taken for each frame, subtracted with grayscale image, converted into binary image using appropriate threshold values. The undesired regions found in binary image are taken care of by eroding the binary image with appropriate magnitude. After eroding the desired region’s area in binary image becomes small, to compensate for the area loss we dilate the eroded image with appropriate magnitude to get back the area lost in desired region. Median filter is taken on the final image to get rid of any noise in binary image. The final image is undergone blob detection and the centroid is found by finding the centroid of the bounding box of that particular region. The Red blob is detected using red channel as base and the centroid for that region is calculated. That centroid’s coordinate is mapped to the mouse cursor’s coordinate by using java.util.robot package. So, any object of bright red colour can be moved in front of the camera to control the cursor. Similarly we detect whether a blob exists using Blue and Green channels as base, if so they are mapped to enable double click and right click respectively. We made use of area of region to open My Computer when red cap is brought closer to the camera. Similarly when the Blue blob is mover towards the camera, the increase in blog area is noted and minimizes all current open applications. We do the above by using keyboard shortcuts windows +E and windows+D. This is achieved using java.awt. event. keyEvent package to control the keyboard strokes. 7. Result The face is detected using Viola and Jones. Eigen points are taken and the change in position is used to track the face. The PCA algorithm is used to find the nearest similar face in database. Then the control is given to hand tracking. The red cap controls cursor movement while blue and green caps are used to double click and right click respectively.

Fig. 2. (a) Face detection; (b) Matching

N. Lalithamani / Procedia Computer Science 78 (2016) 146 – 152

Fig. 3. (a) Hand tracking; (b) Movements of hand control

8. Limitations For the Face Recognition module the accuracy is affected by the lighting variation. In hand tracking module, detection becomes almost impossible if the background is bright red/green/blue colour, so the background is maintained as a light colour. The application however fails clearly in darkness which is not the case with real mouse, so the illumination is always provided 9. Future Work In the forthcoming future, this work could be improvised with Facial Biometrics. Biometric systems integration services which combine face recognition software with other biometric traits as well as existing identification, recognition aspects to be carried out. A person's face will be the private, secure and convenient password. Applications range from static to dynamic, uncontrolled face identification in a cluttered background. Locating the face from a general view point under different illumination conditions , facial expressions and aging effects is the real challenge to be resolved. It is a real focussed challenge to automatically locate the face. 10. Conclusion It had been proposed on a multi-view face recognition algorithm that does not require any estimation of pose to be done or any registration in model to be set up. It had been modelled as one of the good algorithmic identification of object towards the similarities of feature measures. It had also been implemented with algorithmic performance standards of objects available in the database. Under various illuminations that are invariant we could compensate the combinatorial effects by pre-processing the objects that exist in the database by means of extracting the feature set. However, extreme conditions derived can cause changes in the assumptions to be not validated. Hence a new track of motion of human system in a simplified versatile combinatory effect had been projected which will be useful for initializing and optimizing. It performance improvises the interfaces for future correspondences and directions that move forward for various forthcoming associations effectively in a quantified way with standard strategy testing mechanisms. References 1. X. Liu and T. Chen, “Video-based face recognition using adaptive hidden Markov models,” in Proc. IEEE Conf. Comput. Vis. Pattern

151

152

N. Lalithamani / Procedia Computer Science 78 (2016) 146 – 152

Recognit., vol. 1. Jun. 2003. 2. O. Arandjelovic, G. Shakhnarovich, J. Fisher, R. Cipolla, and T. Darrell,“Face recognition with image sets using manifold density divergence,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., vol. 1. Jun. 2005,. 3. G. Aggarwal, A. K. Roy-Chowdhury, and R. Chellappa, “A system identification approach for video-based face recognition,” in Proc. Int. Conf. Pattern Recognit., Aug. 2004. 4. Q. Cai, A. C. Sankaranarayanan, Q. Zhang, Z. Zhang, and Z. Liu, “Real time head pose tracking from multiple cameras with a generic model,” in Proceeding Computer Vision and Pattern Recognition Workshops, June 2010, pp. 25–32. 5. H. S. Lee and D. Kim, “Generating frontal view face image for pose invariant face recognition,” Pattern Recognition Letter on intent, volume 27, May 2006, pp.747–754. 6. R. Chellappa, M. Du, P. K. Turaga, and S.K.Zhou, “Face tracking and recognition in video,” in Handbook of Face Recognition, S. Z. Li and A. K. Jain, Eds., 2nd ed. New York, NY, USA: Springer-Verlag, 2011, pp. 323–351. 7. S. Ba and J. M. Odobez, “Probabilistic head pose tracking evaluation in single and multiple camera setups,” Multimodal Technology Perception Humans, June 2008, volume 4625, pp. 276–286. 8. X. Chai, S. Shan, X. Chen, and W. Gao, “Locally linear regression for pose-invariant face recognition,” IEEE Transactions on Image Processing, volume 16, July 2007, no. 7, pp. 1716–1725. 9. H. S. Lee and D. Kim, “Generating frontal view face image for pose invariant face recognition,” Pattern Recognition Letter on intent, volume 27, May 2006, pp. 747–754. 10. S. Li, X. Liu, X. Chai, H. Zhang, S. Lao, and S. Shan, “Morphable displacement field based image matching for face recognition across pose,” in Proceeding European Conference on Computer Vision, Oct. 2012, pp. 102–115. 11. V. Blanz, T. Grother, P. J. Phillips, and T. Vetter, “Face recognition based on frontal views generated from non-frontal images,” in Proceeding IEEE Conference on Computer Vision and Pattern Recognition, volume 1, June 2005, pp. 454–461 12. P. Breuer, K.-I. Kim, W. Kienzle, B. Scholkopf, and V. Blanz, “Automatic 3D face reconstruction from single images or video,” in Proceeding IEEE International Conference on Automated Face Gesture Recognition, Sep. 2008, pp. 1–8. 13. A. Pentland, B. Moghaddam, and T. Starner, “View-based and modular Eigenspaces for face recognition,” in Proceeding IEEE Conference on Computer Vision and Pattern Recognition, Jun. 1994, pp. 84–91. 14.N.Lalithamani, Arun Prasad P, Gowtham Raj R, Guru Prasath A, Logeshkumar D,” Face Recognition and Tracking using Video streams”,in the International Conference on Emerging Trends in Science, Engineering, Business and Disaster Management. 15. A. Pentland, B. Moghaddam, and T. Starner, “Multi-view face recognition by nonlinear tensor,” in Proceeding International Conference on Pattern Recognition, Dec. 2008, pp. 1–4. 16.A.

Pnevmatikakis and

L.

Polymenakos,

Far-field,

Multi-Camera,“Video-to-Video

Face Recognition”. Chennai, India: InTech,

2007, pp. 468–486. 17.Siddharth S. Rautaray and, Anupam Agrawal “Real Time Hand Gesture Recognition System for Dynamic Applications”, International Journal of UbiComp (IJU), Vol.3, No.1, January 2012. 18. B.Stenger, A.Thayananthan, P.H.S.Torr, and R.Cipolla, “Model-based Hand Tracking using a Hierarchical Principal Component Analysis filter”, in Pattern Analysis and Machine Intelligence, 2006. 19. V. Athitsos and S. Sclaroff, “Estimating 3D Hand Pose from a Cluttered Image”, in Computer Vision and Pattern Recognition, 2003. 20http://mind.kaist.ac.kr/mixedreality.php 21.http://principles.ou.edu/mag/earth.html 22.http://engineeringprojects101.blogspot.in/2012/07/facedetectionusingmatlab.html 23.http://www.face-rec.org/