Gesture Recognition Based On Elastic Deformation Energies Radu-Daniel Vatavu1,2 , Laurent Grisoni1 , and Stefan-Gheorghe Pentiuc2 1
Laboratoire d’Informatique Fondamentale de Lille, Villeneuve d’Ascq 59650, France 2 University Stefan cel Mare, Suceava 720229, Romania
[email protected],
[email protected],
[email protected]
Abstract. We present a gesture recognition method based on deformable shapes and curvature templates. Gestures are modeled using a spline representation that is enhanced with elastic properties: a gesture trajectory as a whole or any of its parts may stretch or bend. We regard such an approach as well-suited for dealing with the inherent variability of human gesture execution. The results of our gesture classifier are demonstrated with a video-based acquisition approach. Key words: gesture recognition, elastic matching, deformation energies
1
Introduction
Gesture has been given a lot of attention in the last decades as an effective mean for human computer interaction. The motivation is given by naturalness and efficiency: people use gestures in the real world to interact with real objects and to convey information. An interface that is based on the use of gestures may prove to be ideal comparing to other interaction techniques. That is of course if we consider an ideal gesture framework that does not importunate, distract or adds significant cognitive load [1, 2]. Successful implementation of a gesture recognition system includes selection of an appropriate technology for acquisition, defining a model for gesture representation, implementing a robust classifier and finally, giving feedback to the user. Many of the above issues have been widely discussed in the literature. Good overviews on the state of the art in gesture interaction including gesture taxonomies for HCI, existing technologies, recognition and interpretation techniques are given in [3–5]. A particular problem that may be identified among the ones listed above relates to gesture trajectory. Trajectory recognition is difficult due to the variability that comes with gesture execution: different users will input different patterns for the same gesture type and even more, the same user performs the same gesture with a certain degree of variability at different moments in time. Hence robust approaches are needed to support variations in trajectories that may translate into local deformations of parts such as stretching or extra bending, articulations or any other small differences in the gesture shapes.
4
Gesture Recognition Based On Elastic Deformation Energies
In this paper we propose a gesture trajectory recognition approach based on previous results on deformable shapes [6–10]. The proposed method provides energy formulations that make the whole process flexible enough for handling the variability of human gestures. The continuous structure of gesture is used for taking advantage of a spline-based gesture representation. Robust energy terms for measuring local deformations are proposed together with a procedure for computing gesture templates from a given set of samples. We believe that the proposed terms are well-suited when dealing with variability in gesture execution. Also we think that the elastic view on gesture may lead to interesting results as stated under the conclusions section. For the purpose of implementation and demonstrating our gesture classifier, we built a recognition system with video-based acquisition of hand gestures. The paper is structured as follows: section 2 relates to previous works on gesture recognition with an interest on visual based approaches. Section 3 describes the details of our video-based acquisition process, the spline representation we chosen for gestures as well as notes on curvature functions we obtained on our experimental acquired gesture trajectories. Section 4 describes the classification method based on average curvature templates together with an overview on the energetic terms and the main idea behind elastic shapes with references in the literature. Experimental results for gesture matching are finally presented.
2
Related research
Gesture recognition research has been widely given attention lately and the idea of interacting by gestures received support among public as it was induced by various media. Gestures are now perceived by many or most people as the next upcoming interaction technology. To support this, a great variety of devices for gesture acquisition were developed and trackers, pointing or whole hand or body devices are now available [11]. Among all, video gesture recognition has the main attractions of not being intrusive and not requiring users to wear additional equipments or devices. The final feeling is thus of comfortability and naturalness. Video based acquisition comes however with several drawbacks such as: real-time interaction requires high processing power especially when more cameras are involved for a 3D reconstruction process; dependency on working scenario parameters such as lighting, user skin color, changing background; hands occlusion. Surveys on visual gesture recognition may be found in [11, 12]. When it comes to hand gestures, common approaches are to follow colored gloves [13], detect skin color [14, 15] or track local features [16, 19, 17] and active shape models [20]. KLT features, named after Kanade, Lucas, Tomasi are good features to track [16]; sets or flocks of KLT features have been used by Kolsch and Turk [17] for robust hand tracking under the HandVu system; the Haar-like detectors of Viola and Jones [18] were also analyzed for robustness against hand posture tracking [19]. Recognition is performed using Markov models [21], finite state machines [22], temporal motion templates [23], probability signatures [24] or various shape similarity measures [25].
Gesture Recognition Based On Elastic Deformation Energies
5
In what concerns gesture trajectory recognition, many researchers have considered shape analysis approaches that work with local parameters such as curvature: curvature scale space representations [29] detection of high curvature points [26], similarity measures for curvature differences [27] or different curvature based representations [28]. Trajectory recognition is a difficult problem, as already mentioned in the introduction, due to the variability that comes with gesture execution. Gestures may be looked upon as motion trajectories and standard recognition methods have been applied however there is the need of a robust yet flexible model for the variability within gesture in order to support a reliable recognition.
3
Gesture acquisition and representation
We further consider a gesture as a point moving in time gesture(t) : [t0 , t1 ] →
