A New Algorithmic Approach for Fingers Detection ...

A New Algorithmic Approach for Fingers Detection and Identification Arslan Mubashar Khan, Waqas Umar, Taimoor Choudhary, Fawad Hussain, Muhammad Haroon Yousaf Department of Computer Engineering University of Engineering and Technology Taxila, Pakistan ABSTRACT Gesture recognition is concerned with the goal of interpreting human gestures through mathematical algorithms. Gestures can originate from any bodily motion or state but commonly originate from the face or hand. Hand gesture detection in a real time environment, where the time and memory are important issues, is a critical operation. Hand gesture recognition largely depends on the accurate detection of the fingers. This paper presents a new algorithmic approach to detect and identify fingers of human hand. The proposed algorithm does not depend upon the prior knowledge of the scene. It detects the active fingers and Metacarpophalangeal (MCP) of the inactive fingers from an already detected hand. Dynamic thresholding technique and connected component labeling scheme are employed for background elimination and hand detection respectively. Algorithm proposed a new approach for finger identification in real time environment keeping the memory and time constraint as low as possible. Keywords: Image processing, hand detection, finger detection, hand gesture recognition.

1. INTRODUCTION Since the invention of computer mankind strives for easier human computer interaction to facilitate the human being specially disabled persons. Different types of techniques have been proposed in different fields for human computer interaction. One of the widely used techniques is to use “glove based devices” [1, 2, 3, 4, 5] they measure the joints of hands/harms and their spatial location with mechanical devices. The main disadvantage of these devices is that the user has to wear cumbersome devices which limit the movement of user. Another emerging technique for HCI is Computer vision. This technique uses set of video cameras and computer vision techniques to identify the gestures [8]. Most of the proposed solution in later category uses hand tracking, hand gesture recognition [9, 10] and pose estimation [11]. All of the proposed techniques have their own performance levels and criteria for different types of scenes. Most of the techniques used template matching [12], decision trees or classifiers which gave them accurate results but they were too slow or require lot of memory to save the database which is not suitable for real time environment. Also lightening environment decreases the efficiency of the techniques. Here we are proposing an algorithm to find the active fingers and inactive Metacarpophalangeal (MCP’s) without having any prior knowledge of the scene. This makes it an ideal algorithm for real time environment. System developers nowadays are striving to develop real time algorithms to detect different hand gestures and perform different tasks based upon them. The main constraint they are facing is development of time and memory critical systems which do not depends upon the previous knowledge.

2. RELATED WORK Hand Gesture Recognition has attracted quite a few system developers for interaction with the robots and in different control systems. Researchers have used different techniques to detect gestures which had their own performance levels. The work done by Xingyan Li [6] uses Fuzzy C means Logic to detect hand gestures in which he divided the hand detected image into 12 equal regions and then generated a 12 number array based on the mean intensity levels of each region. Then he matched the array with already defined array and recognizes the gesture which is closest to the input array. It approximates the intensity level hence it can’t give accurate results every time, also matching is used which requires lot of memory therefore it’s impossible to implement in time critical environment.

Asanterabi Malima, Erol Özgür, and Müjdat Çetin [7] proposed a fast algorithm to detect number of active fingers by drawing a circle of radius 0.7% of the farthest active pixel from center of gravity centered at center of gravity of the active region. They achieved an accuracy of 91%. The problem with this approach is that they do not recognize which finger is active and that limits it capability for advance human computer interaction applications.

3. PROPOSED ALGORITHM The most important part of recognizing the hand gesture is to find which fingers are active and which fingers are inactive. Complete block diagram of the proposed system is given in figure 2. Constructing a circle at the center of the hand like region with the radius of 0.73 of the maximum of the farthest active point from the center can give the number of active fingers at the outer boundary of the circle as described in [2]. A simple hand filing algorithm from top can give the tops of a hand if combined with the active fingers which can give the active and inactive fingers of a hand. Hand Detection

Hand Normalization

Find COG

Drop Rays

Draw Circles

MCP Detection

Count Active Fingers

Valleys Removal

Valleys Detection

Combine Active Fingers and MCP

Binary Number

Figure 1. Block Diagram

Our proposed system consists of following steps: I. II. III. IV.

Localizing the hand like regions based upon the largest component methodology. Calculating the center of gravity of the hand like region and constructing the circle at the center to find active fingers. Finding inactive MCP’s Combining MCP’ with the active fingers to find which finger is active and which is not.

3.1. Normalizing Hand Like Region In past lot of techniques have been used to detect hand from an image which includes different pattern recognition techniques like camshift algorithm [3] [4] and ABCshift[5] have been applied. They are quite useful for complex background but assuming the simpler background and static hand these techniques are not quite useful in our case therefore we have applied dynamic thresholding technique to eliminate background. We cut the histogram in half and

kept the intensity level with lower frequencies which results in a binary image containing different components present in scene. Camera is very close to hand so it is assumed that the hand is the largest component in the scene. Connected component labeling algorithm is implemented to detect the largest connected component which is the hand in the image. This process is shown in figure 2.

Figure 2. Different Steps of Normalizing hand like region

3.2. Finding COG and Counting Active Finger Given the hand like region its COG can be found as:

(1)

Next we calculated maximum distance from COG to the edge of the hand which is calculated by the following equation:

(2) Where, X2=x coordinate of COG Y2=y coordinate of COG D=distance from COG to the point (x, y) A circle is drawn with center at COG and radii, 0.73 times the maximum distance calculated from COG and boundary thickness of 0.3. If the finger is active, its tip would be at least, a distance, greater than 0.73 times the maximum distance calculated so, the boundary of the circle drawn would intersect the active fingers. The connected component which lied upon the boundary of the circle would represent the active fingers as shown in figure 3.

Figure 3. Using COG to find the active fingers

3.3. Filling the hand to find MCP MCP is usually the prominent part of the hand (figure 4). So it can be differentiated from any other part. Consider dropping a ray from the farthest part of the farthest active point from the center to the opposite direction which can only duplicate itself only in left right and forward direction but not in the backward direction. This will differentiate the MCP’s from the rest of the hand as shown in figure 2.

Figure 4. Metacarpophalangeal

Further to decrease the false rate a contour tracking is applied to detect the tops and valleys by using the fact that both sides of the valleys go upwards while both sides of tops go downwards to eliminate the false detected valleys as shown in figure 5.

Figure 5. Procedure for finding MCP

3.4. Combing MCP’s with Active fingers After finding MCP combine the active fingers with the detected MCP as show in figure 6. A simple counting of connected regions from either left or right side and concatenating ‘0’ for gray and ‘1’ for white region would result in a binary number (in our example if we do it from left to right it would be ‘00110’) which can be used for control devices to do a specific task.

Figure 6. combining MCP’s with Active Fingers

4. CONCLUSION AND FUTURE WORK A simple but fast algorithm is proposed to identify the hand gesture. From the images of hand the algorithm detects the inactive MCP’s and the active fingers. Then combine it to and generate a binary number to detect which finger is active. Proposed approach works well in simple conditions with an accuracy of 90% but due to weaker localizing portion its efficiency decreases with complex backgrounds. The algorithm can be further improved with introduction of stronger hand recognition portion like camshaft or other pattern recognition techniques. This algorithm can be used in any kind of environment but its efficiency will be higher in the environment where camera is at the top of the hand this algorithm can be used for controlling purpose, like facilitating the disabled person by Home automation based upon hand gestures, controlling different devices by hand like computer or television. As it generates only a binary number it can be implemented as control system.

REFERENCES [1]. Pranghofer, M. “Wheels and white canes: Tips for helping blind wheelchair users.” Braille Monitor. Baltimore (MD):National Federation for the Blind; 1996. [2]. S.S Fels and G.E hinton, “Glov-talk: A neural network interfae between a data-glove and a speech synthesizer,” IEEE transactions on Neural Networks, vol. 4, pp. 2-8, jan. 1993. [3]. D.L. Quam, “Gesture recognition with a dataglove,” in proceedings of the 1990 IEEE National Aerospace and Electronics Conference, vol. 2, 1990. [4]. T. BAudel and M. Baudouin-Lafon, “Charade: remote control of objects using free hand gesturees,” Communications of the ACM, vol. 36, no. 7,pp. 28-35, 1993. [5]. C. Wang and D.J. Cannon, “A virtual end-effector pointing system in point-and-direct robotics for inspection of surface flaws using a neural network based skeleton transform,” in Preceedings of IEEE International Conference on Robotics and Automation, vol. 3, pp.784-789, may1993 [6]. Xingyan Li, ”Gesture Recognition Based on Fuzzy C-Means Clustering Algorithm”, Technical report, 2005. [7]. "A Fast Algorithm for Vision-based Hand Gesture Recognition for Robot Control ," Asanterabi Malima, Erol Ozgur, and Müjdat Çetin, IEEE Conference on Signal Processing and Communications Applications, Antalya, Turkey, April 2006.

[8]. N. Gupta, P. Mittal, R. Sumantra Dutta, S. Chaudhury, S. Banerjee (2002). Developing a gesture based inter- fa ce,IETE, Journal of Research: Special Issue on Visual Media Processing. [9]. M. J. Black (1996). EigenTracking: Robust Matching and Tracking of Articulated Objects Using a View Base d Representation, International Journal of Computer Vision, 26(1), pp. 63–84, 1998. [10]. E. Ueda, (2003), A Hand Pose Estimation for Vision-Based Human Interfaces, IEEE Transactions on Indus trial Electronics, Vol. 50, No. 4, pp. 676–684. [11]. Shan, C., Wei, Y., Qiu, X., and Tan, T.: ‘Gesture recognition using temporal template based trajectories’. Proc. 17th Int. Conf. Pattern Recognition, 2004, vol. 3, pp. 954–957 [12]. COMANICIU, D. and MEER, P. Mean Shift: A robust approach toward feature space analysis. IEEE Trans. On Pattern Analysis and Machine Intelligence, vol.24, No. 5. 2002. [13]. Bhattacharya Distance. http://en.wikipedia.org/wiki/Bhattacharyya_distance. Last updated 29 May 2008, at 05:07. [14]. FUKUNAGA, K. and HOSTETLER, L. The estimation of the gradient of a density function, with applications in pattern recognition. IEEE Transactions for Information Theory. Volume 21, Issue 1, Page(s): 32-40 : Jan 1975.