Medicine Meets Virtual Reality 20 J.D. Westwood et al. (Eds.) IOS Press, 2013 © 2013 The authors and IOS Press. All rights reserved. doi: 10.3233/978-1-61499-209-7-465
465
Video Game interfaces for Interactive Lower and Upper Member Therapy Alvaro URIBE-QUEVEDOa,1 and Byron PEREZ-GUTIERREZb and Silas ALVESc a Multimedia Engineering, Mil. Nueva Granada University b Virtual Reality Center, Mil. Nueva Granada University c Electric and Computing Engineering, São Paulo University, Brazil
Abstract. With recent advances in electronics and mechanics, a new trend in interaction is taking place changing how we interact with our environment, daily tasks and other people. Even though sensor based technologies and tracking systems have been around for several years, recently they have become affordable and used in several areas such as physical and mental rehabilitation, educational applications, physical exercises, and natural interactions, among others. This work presents the integration of two mainstream videogame interfaces as tools for developing an interactive lower and upper member therapy tool. The goal is to study the potential of these devices as complementing didactic elements for improving and following user performance during a series of exercises with virtual and real devices Keywords. 3DUI, Interaction, Rehabilitation
Introduction Recent advances in 3D displays allow increasing the immersive experience through 3D smart TV, autostereoscopic and stereoscopic projectors and screen [1]. At the same time, interactions between people and machines have also improved by becoming more ergonomic and comfortable, guaranteeing the success of a virtual reality application [2]. Even though researches on these topics have been going for almost 20 years, due to costs and complexity just recently they are becoming part of our daily life [3]. Nintendo, Sony and Microsoft developed 3DUIs based on inertial and optical sensors for several in game activities like fit, dance, fight or sports [4][5][6], for taking advantage of such trend. The goal of this work is to implement a simple interactive environment using as input devices common 3DUI gaming controllers as suitable tools for complementing traditional lower and upper member rehabilitation exercises with two interactive scenarios using a virtual environment and a mobile robot for distracting the user during therapy.
1
[email protected]
466 A. Uribe-Quevedo et al./ 160 C. Guerrero and A. Uribe-Quevedo/ Kinect-based Posture Tracking
1. Methods For capturing lower and upper member movement, the Wiimote was considered since it allows interaction through its buttons and three axes accelerometer. The Wiimote interactions were programmed using the Wiiuse API for configuring the navigation accordingly to rotations captured through the user hand gestures. For guaranteeing a natural interaction between the user and the device, complementary commands were programmed using the depth map information from the Kinect for navigating the virtual environment and allowing parameter control for setting up camera view and speed. 1.1. System Architecture The system architecture, as presented in Figure 1, is composed of the user, the input devices, the motion commands, the virtual environment, the visual feedback composed of a 3D navigational environment and a didactic robot for distracting the user. The motion commands are programmed accordingly to each 3DUI's libraries allowing their integration with the virtual environment developed in OpenGL. Once the lower and upper member motions are mapped, the data is used for transforming the avatar position and camera orientation through the user movement, which is also used for controlling a mobile robot as a complementary distractor while executing the corresponding exercises.
Figure 1. 3DUI interactions.
2. Results The tests were performed with a population of male patients suffering from cauda equina, spinal cord injury, stroke and radius/ulna fracture. The Wiimote was attached to the lower member where the therapist thought to be the most significant location, above the knee and above the ankle, and held by hand for the upper member interactions. When using the Kinect the hand became the pointer for navigating and moving the mobile robot, while the leg’s movement allowed walking and moving a serial robot, as presented in Figure 2. The user experience was validated through a survey focused on intuition, impact, comfort, utility and ease of use, giving scores from 1 to 5, being 1 the most negative and 5 the most positive value for each parameter. The results presented an overall satisfaction with the tool are presented in Figure 3.
A. Uribe-Quevedo et al./ 160 C. Guerrero and A. Uribe-Quevedo/ Kinect-based Posture Tracking
a) Wiimote motion
467
c) Robotino operation
b) Serial robot Kinect operation
Answer
Figure 2. Usage survey 6 5 4 3 2 1 0
ease of use discomfort utility intuition impact 1
2
3
4
5
Parameter
Figure 3. Usage survey
3. Conclusions & Discussion 3DUIs presented an interesting option as complementing tools for assisting therapy processes, even though some limitation were encountered, such as a discomfort when the patient did not have enough muscular strength to move his leg or arm, along with small and trembling motions. From the survey analysis, the tool showed its potential for helping improve the rehabilitation process. The feedback provided by the therapists and patients justifies future works in further analysis and motion comparison for monitoring user evolution, along with enriched VR environments for offering more scenarios and interactions for increasing user interest.
Acknowledgements The authors acknowledge the support of FAPESP processes 2009/05396-4 and 2010/02000-0, the Integrated Automation and Robotics Laboratory of the University of Campinas, the Occupational Therapy Center of the Colombian Army Health Battalion, and the VR Center of the Mil. Nueva Granada University.
References [1] H. Iwata and T. Fujii, "VIRTUAL PERAMBULATOR: A Novel Interface Device for Locomotion in Virtual Environment" Virtual Reality Annual International Symposium (1996), 60-65. [2] G. Burdea and P. Coiffet, Virtual Reality Technoloy, Wiley, 2003. [3] J. LaViola, "Bringing VR and Spatial 3D Interaction to the Masses through Video Games" Computer Graphics and Applications, IEEE, 28,(2008), 10-15. [4] C. A. Wingrave, B. Williamson, P. D. Varcholik, J. Rose, A. Miller, E. Charbonneau, J. Bott and J. J. LaViola, "The Wiimote and Beyond: Spatially Convenient Devices for 3D User Interfaces" IEEE Computer Graphics and Applications, 30, (2010), 71-85. [5] J. C. Lee, "Hacking the Nintendo Wii Remote" IEEE Pervasive Computing, 7, (2008), 39-45. [6] E. Suma, B. Lange, A. Rizzo, D. Krum and M. Bolas, "FAAST: The Flexible Action and Articulated Skeleton Toolkit", Virtual Reality Conference (VR), 2011 IEEE, (2011), 247-248.
