Performance Evaluation of 3D Reaching Tasks Using a Low-cost Haptic Device and Virtual Reality Emilia Scalona1, Darren Hayes1&2, Eduardo Palermo1,, Zaccaria Del Prete1 and Stefano Rossi3 Email:
[email protected],
[email protected],
[email protected],
[email protected],
[email protected] 1 Department of Mechanical and Aerospace Engineering, “Sapienza” University of Rome, Roma, Italy 2 Seidenberg School of CSIS, Pace University, New York, USA 3 Department of Economics, Engineering, Society and Business Organization (DEIM), University of Tuscia, Viterbo, Italy
Abstract — In this paper we propose a new protocol based on Virtual Reality and a low-cost haptic device for evaluating motion performance during perturbed 3D reaching tasks. The protocol presented herein was designed to assess how different force amplitudes and different reaching directions influence motor performance of healthy subjects. We developed a novel gaming scenario using Unity 3D and the Novint Falcon, a lowcost haptic joystick. The protocol consisted of six 3D point-topoint reaching tasks, which were performed by means of the Falcon while six different force fields were applied. Five subjects were enrolled in the study. During each task, subjects were asked to reach 80 targets. The trajectories of the end-effector, during each task, were recorded to calculate the following kinematic indices: duration of movement, length ratio, lateral deviation, aiming angle, speed metric and normalized jerk. The coefficient of variation was calculated to study the intra-subject variability to establish which indices better assessed the accuracy and the smoothness of the trajectories. Subsequently, two-way repeated measurement ANOVA tests were performed for all indices in order to ascertain effects of the 6 levels of force and the 8 directions of the reaching task. Length ratio and speed metric have proven the highest intra−subject repeatability as accuracy and smoothness indices, respectively. Statistical analysis demonstrated that all the accuracy indices are not sensitive to amplitude variation of the applied force field, nor to different target directions. Conversely, the smoothness indices showed statistical differences in both forces and directions. In particular, the speed metric is sensitive to the applied force, and the normalized jerk depends on the target directions. Keywords—Virtual Reality; haptic; low-cost; 3D reaching; upper limb rehabilitation;
I. INTRODUCTION Neuro-motor rehabilitation is based on brain plasticity, the ability of the brain to recover its lost function, modifying its own structure and creating new neural pathways in response to repetitive motion training [1]. Scientific evidence suggests that intensive and specific rehabilitation tasks are necessary for motor recovery. In this context, robot mediated therapy (RMT) is an emerging strategy for neurologic and motor rehabilitation of upper limbs. In the last few years, researchers have begun to recognize the potential of applying Virtual Reality (VR) and haptic technologies to neurological assessments and rehabilitation treatments [2]. VR allows for creating immersive
978-1-5386-0979-8/17/$31.00 ©2017 IEEE
and interactive environments, where several targeted motor tasks can be proposed as serious games. Rehabilitation exercises can be easily tailored to the patient’s needs and the software used to develop VR scenarios usually allows for recording the kinematics of patients, and providing data for an objective evaluation of performance. However, traditional VR environments do not provide the sense of touch, which is fundamental in rehabilitative motor tasks, which include reaching and grasping [3]. A variety of robotic haptic devices, producing informative haptic feedback for virtual interactions, have been proposed in various publications. These types of devices have shown to improve the manipulation capabilities of the user in a VR environment, thereby also providing additional information about the physical characteristics of virtual objects [4], [5]. Despite the widespread use of RMT, robotic devices are characterized by high costs and can only be used in clinical environments. As a result, over the last few years, use of lowcost mechatronic devices have become prominent, as they can be used at home by patients, with lower therapist supervision [6]. Low-cost haptic devices, originally developed as game controllers, might represent a viable alternative to devices normally used in RMT. They could simulate the kinesthetic sense of the user and generate forces through the end-effector [7]. There are several haptic devices available on the market: Omni Phantom [8], Haption Virtuose 3D [9], FD Delta [10], and Novint Falcon [11]. When comparing existing devices, the Novint Falcon has been emerged as the most suitable for rehabilitation purposes as it can generate a greater feedback force. Moreover, the user-friendly control interface fosters the use of the Falcon in different applications [12]. Several studies have investigated the use of this device in neuro-rehabilitation treatments. Palsbo et al. [13] used the Falcon to train fine movements, like writing, in a group of typically developing children. The results showed that the Falcon provided significant improvements in hand writing performance. Chortis et al. [16] have designed six games, integrated with the Falcon, to evaluate the application of a haptic joystick in home rehabilitation. Eight post stroke subjects were enrolled in the study, divided into two groups. The preliminary results indicated that the subjects were able to correctly perform the rehabilitation tasks. Cappa et al. [7] used the Falcon to study how a group of healthy subjects responded
to different force feedback in a virtual environment. Subjects were asked to perform specific movement trajectories perturbed by different force fields, thereby demonstrating that the force field application improves the accuracy of movement. In this promising scenario, we sought to develop a new haptic-based VR system for assessing motor performance. We proposed and tested a novel protocol to evaluate participant performance during perturbed 3D reaching tasks. More specifically, our protocol consisted of several tasks with increasing difficulty levels, which can be very useful in assessing the functional recovery of an arm, impaired by a neuro-motor disease [14]. We validated our protocol by studying the feasibility of using the most widely implemented kinematic indices for the evaluation of the motor performance in reaching tasks. Thus, we investigated how different force amplitudes and different reaching directions affected the kinematic indices and, more generally, the motor performance of healthy subjects.
B. Experimental protocol Five right-handed healthy adults (age ranging between 27 and 29 years) were enrolled in the study. The experimental protocol consisted of six reaching tasks to be performed with varying force levels: F0, F2, F4, F6, F8, and F10. During each task, the subject had to reach 80 targets, which were divided in 40 forward (from the center to the end of the workspace) and 40 backward (from the end of the workspace to the center) movements. Between two consecutive reaching tasks, a time of 60 s was set to allow the subject to rest. Before starting the data acquisition, the subject performed a task of 80 targets, where no force field was applied, to familiarize with the task.
II. MATERIALS AND METHODS A. Experimental setup and game design Our experimental setup consisted of a 3D haptic low-cost interface (Novint Falcon, Washington, PA) and a game scenario developed using the Unity3D game engine. The Novint Falcon is classified as a delta robot since its motors are placed in parallel and not serial. The end-effector is able to translate in a workspace of 101.6 mm × 101.6 mm × 101.6 mm; rotations are not allowed. The Falcon can deliver forces up to 8.8 N, controllable in amplitude and direction. Forces are generated by three DC brushed motors, each mounted on the base of one of the three Falcon arms. Each motor is equipped with a linear encoder resulting in a spatial resolution of the end-effector position of about 60 μm for each axis. Unity 3D does not natively support the Novint Falcon, so a C++ library was made and compiled into a dynamic linked library using the open source library libnifalcon (https://github.com/kbogert/falconunity). The game consisted of a start screen with the main menu (Fig.1a) and a second screen where the user can choose the game options (Fig.1b). The game scene (Fig.1c) is set in the outer space and the user, using the Falcon, can move a starship positioned on a base (home-base) placed in the origin of the scene. Eight bases (target-bases) are positioned at a depth of 30 mm along the axis, perpendicular to the screen plane (Z axis), equidistant along a circumference with a 40 mm radius, in the plane of the screen (XY plane). In each task, the initial position of the starship is at home-base in the center of the screen. When the start button is pressed, an asteroid, which represents the target of the reaching movement, randomly appears in one of the eight target-bases. The user, moving the end-effector of the Falcon, has to reach the target and then return to the homebase, as shown in Fig.1d. The game ends when 80 targets have been reached. The users can select from 6 levels of force by selecting from the options in the main menu on the second screen: F0 (zero force imposed), F2 (1.0 N), F4 (2.0 N), F6 (3.0 N), F8 (4.0 N) and, F10 (5.0 N). The applied force is opposed to the direction travelled by the starship to reach the target. Before starting the game, a calibration procedure is needed to allow the Falcon to move in the scene workspace.
Fig. 1. (a) main menu of the game; (b) menu with the game options; (c) game scenario; (d) subject performing the task.
C. Data Analysis The position of the end-effector was acquired with a sampling frequency of 50 Hz. The acquired data was processed offline with Matlab (MathWorks, 2012b, Natick, MA, USA). Initially, the position of the end-effector was filtered with a 6th order, zero phase shift low-pass Butterworth filter, with a cutoff frequency of 10 Hz, and then derived to obtain speed, acceleration, and jerk. The processed data were divided in 8 reaching movements according to direction: North (N), NorthEast (NE), East (E), South-East (SE), South (S), South-West (SW), West (W), and North-West (NW). Only forward movements were analyzed. In particular, each movement was assumed to start when the speed magnitude became greater than 10% of the peak speed; the movement was assumed to end when the starship hit the target. To characterize the kinematics of the movement, the following indices were calculated: Duration of Movement (T), Length Ratio (LR), Lateral Deviation (LD), Aiming Angle (AA), Speed Metric (SM), and Normalized Jerk (NJ), [15]. T is the time between the onset and the end of the movement. LR, LD and AA were used to quantify the accuracy of the movement. Of note, LR is the ratio between the path actually travelled by the subject and the ideal one, i.e. the minimum distance between the centers of the home-base and the target-
base. LD is defined as the highest deviation from the straight line connecting the starting and the ending points of the movement trajectory. AA is the angle between the line connecting the starting and ending target, and the line from the starting point to the peak speed point. Higher values of LR, LD and AA represent a reaching task performed with a low accuracy. SM and NJ are indices of movement smoothness. SM is measured as the ratio between the mean and the peak velocity. The SM value increases when movement smoothness increases. Finally, NJ is the Normalized Jerk, as proposed by Teulings et al. [16]. Lower values of NJ indicate smoother movements. After calculating the indices, coefficients of variation (CV) were calculated to study the intra-subject variability. In particular, CV was calculated as the percentage ratio between the standard deviation and the average value of each index considering each subject, direction, and force level. Two-way repeated measurements ANOVA tests were performed for all indices in order to determine statistical differences among the 6 levels of force and the 8 directions of the reaching task. Therefore, forces and directions were considered as independent variables. The Greenhouse-Geisser correction was adopted if the assumption of sphericity was violated. If the interaction effects were significant, we broke down the interactions comparing force levels at each direction with oneway repeated measurements ANOVA. A Bonferroni’s test for multiple comparisons was performed when statistical differences were found. III. RESULTS AND DISCUSSIONS Table I reports mean values and standard deviations of the CV values among direction and subjects. TABLE I. MEAN VALUES AND STANDARD DEVIATIONS OF CV CALCULATED AMONG SUBJECTS AND DIRECTIONS
T [%] LR [%] LD [%] AA [%] SM [%] NJ [%]
F0
F2
F4
F6
F8
F10
25.0 (6.3) 6.6 (2.2) 27.3 (5.7) 40.5 (9.0) 13.0 (3.2) 59.0 (10.7)
21.4 (3.8) 6.3 (1.5) 23.2 (3.8) 45.5 (13.9) 12.7 (1.9) 54.1 (8.9)
21.0 (1.4) 6.3 (2.5) 26.5 (5.3) 54.5 (26.9) 13.4 (3.9) 50.6 (10.9)
22.9 (5.2) 6.7 (2.7) 28.6 (3.7) 56.1 (27.6) 15.2 (4.9) 53.1 (12.2)
21.4 (5.8) 6.8 (2.7) 26.2 (4.4) 46.1 (19.3) 15.6 (1.9) 54.2 (17.5)
20.5 (3.8) 7.5 (2.9) 27.6 (5.8) 44.9 (14.8) 15.7 (2.6) 52.2 (10.9)
The variability of T ranged between 20.5% and 25.0% and the highest CV value was obtained during the zero force task (F0). As regards the variability of the accuracy indices, LR
resulted the least variable index. More specifically, CV of LR did not exceed 8% and the highest value was related to the LR evaluated during F10. The CVs related to LD ranged between 23.2% (F2) and 28.6% (F6). A greater variability was associated with the CV computed for AA, which exceeded 50% during F4 (CV=54.5%) and F6 (CV=56.1%) tasks. Considering the smoothness indices, the CV of SM ranged between 12.7% (F2) and 15.7% (F10). The CV associated with NJ always exceeded the 50% of variability. By examining these results, we can state that when no force is applied, T is more variable and, as the force magnitude increases, the variability decreases, making the task more repeatable. In relation to the accuracy indices, we determined that the LR is the most repeatable accuracy index and therefore more suitable than LD and AA for the trajectory accuracy evaluation. Considering the indices of smoothness, the SM appears more robust than NJ, demonstrating a not negligible intra-subject variability among tasks and subjects. In Fig.2, we report mean values and standard deviations among subjects for all of the considered indices calculated for each force and each direction. Statistical differences are highlighted in the figure. Regarding the ANOVA results, the interaction factors were not significant for each force direction and magnitude. Statistical differences were found among magnitude levels (p=0.003) for the SM, and in target directions (p=0.003) for NJ. No significance differences were found for all the comparisons for the other kinematic indices. Looking at SM, from the Bonferroni’s multiple comparisons, statistical differences were found between F2 and F8 (p=0.009), F2 and F10 (p=0.030), and F4 and F8 (p
