Preliminary Usability Tests of the Wearable Joystick for Gloves-On Hazardous Environments Jaewook Bae 1 , Richard M. Voyles 2 , Roy Godzdanker 3 1 EE 2,3 CE
Department, University of Minnesota, MN, 55455, USA, e-mail:
[email protected] Department, University of Denver, Co, 80208, USA, e-mail:(rvoyles, roy.godzdanker)@du.edu
Abstract— In this paper, we describe usability tests to evaluate a gestural joystick as a new input device. The usability of a device addresses five main issues: learnability, efficiency, memorability, errors, and satisfaction. These different measures of usability are used in an attempt to compare the gestural joystick, called ”wearable joystick”, with four different input devices. This paper reviews the wearable joystick and then describes the quantitative methods for the measures. The claimed features of the wearable joystick include easy and wire-free installation into regular gloves, unencumbered operation, and intuitive control. While the analyses are preliminary, with only five users per device, the results are mixed. In general, results suggest the glove provides adequate performance for users, scoring within norms of other devices for a small sample size. However, several of the measures show that the performances of the wearable joystick are worse than other input devices. Despite such drawbacks, the wearable joystick, of all input devices tested, has significant advantages, such as wearability, a simple but strong sensor mechanism, and a natural and intuitive use paradigm. We envision that as the data processing algorithm of the wearable joystick are improved and the full range of benefits can be compared, the wearable joystick will overcome these drawbacks in the near future.
I. I NTRODUCTION Recently, the state-of-art robotic technologies have been developed actively in urban search and rescue (USAR) [1], [2]. Over the last decade, incorporation of USAR robots have become more common for on-site work and training exercises. For instance, in September 11th attack, scientists contributed to USAR teams to search for trapped victims [?]. USAR Robots were controlled via teleoperation with traditional human/computer input interfaces [2]. The conventional input devices have technically been improved with various new features, such as size, power consumption and carrying ability. However, since it is still difficult for many scientists to seamlessly apply the devices into USAR task, they have focused on wearability [3], [1]. Thus, the traditional input devices, such as mouse, joystick and trackball, are hampered by mandated protective gear including safety glasses, hard hats, respirators, and, most importantly, gloves in urban search and rescue (USAR) task. Since operators in USAR are required to wear heavy gloves to insulate themselves from the hazardous environment, we began to exploit these gloves as an opportunity to embed wearable sensors [1]. In prior work, we presented a prototype of the wearable input interface, ”wearable joystick”, based on unencumbered installation and operation for a specific type of USAR application which we implemented with the USAR robot. Conventional glove interfaces, such as Power Glove, CyberGlove, MIT LED glove
and others [3], have shown to be highly accurate for the recognition of various hand gestures [4]. However, these glove interfaces require complex and fragile sensor structures and wires and are not suitable for use in rugged environments. The hot issue among human/computer interface researchers has been how to intuitively and efficiently interact between computers and human hands in various fields. Many scientists [5], [6] have challenged and applied different types of glove interactive systems to various applications for natural and intuitive approach. Rekimoto [7] developed a unobtrusive wearable interactive device to recognize the change of wrist width for different gestures and sensor pad. This system supported hands-free operations with an unobtrusive characteristic. However, the system is still at an early stage. Most of them discuss mainly about technological implementation. However, although a few of them described the usability test of input interactions, they are still in infancy. Since the evaluation of an input device is strongly based on the usability test, the success of the development of an input device, especially the gestural joystick with special functions, largely depends on the result of the usability test. The results from the usability tests are important metrics to measure the performance of input device as well as to reversely affect the design parameters at the design process. Recently, many scientist have proposed usability evaluation methods to evaluate their own new input interfaces in [8]. Especially, [9] concerned with 5 main topics: learnability, efficiency, memorability, errors, and satisfaction. Hence, in order for a new input device to be valuable, the evaluation results from the above topics should be shown satisfied or at least similar among conventional input devices. In this paper, based on a standpoint of the topics, we conduct the usability test to evaluate the wearable joystick by using ANOVA test. The usability test consists of move-to-target screen test, which looks alike with USAR robot obstacle course. In the test, we compare the wearable joystick with conventional input devices, such as a mouse, a joystick, and a trackball, in variations on NIST gate model. In later sections, we present the quantitative results for the usability test. II. S YSTEM S TRUCTURE A. Layout The wearable joystick [10] provides unencumbered wearability that can be equipped on many styles of traditional gloves. Moreover, the hardware structure of the system can be quite simple and robust in terms of operation and installation. In
the glove part the the wearable joystick, it doesn’t include wire mechanism, additional circuitry and battery in Fig. 1. The embedded rare-earth magnets are relatively small and the sensor pad is worn on the wrist or embedded in the fringe of the sleeve. The magnets and sensors interact with no interconnecting wires, allowing the glove to be easily removed. We can simply install magnets over regular safety gloves without large changes, such as any wire and complicate sensors. The glove of the wearable joystick includes nothing but four rare-earth magnets. Therefore, it is allowed to be washable and never broken under tough USAR environment. Fig. 1 (a) and (b) show that the wearable joystick can be embedded over regular clothing.
for the right wheel:
ωr = αlr × r υ − βlr × r θ
(2)
where ω , α and β represent velocity command and empirical constants, respectively. r
Y r
U
Y
υ r
θ r
X
Magnets U
Wire-free
X
Fig. 3. Global coordinate (U X, U Y ) and robot coordinate (r X, rY ) system with desired motion command vector (r υ , r θ ).
Sensor Pad
C. Hardware Configuration
(a)
(b)
Fig. 1. (a) Full installation of the wearable joystick system over clothing. (b) The wearable joystick on a regular glove.
B. Interpretation of Wrist Gestures
The prototype in Fig. 4 (a) consists of 4 modules: 1) CPU (Atmega128) and sensor interface, 2) Bluetooth and 3) the sensor pad that is embedded with eight sensors, and 4) the traditional glove that is embedded with 4 arc rare-earth magnets. The CPU module in Fig. 4 (b) processes eight analog signals from the eight magnetic sensors and then converts the data into a series of motion command, such as xy screen coordinate or a USAR robot command. Currently, the sensor pad and CPU module are physically connected, but we are developing a wireless system over Bluetooth communication. The sensor pad, shown in Fig. 4 (c), contains 8 GMR sensors (AA004 from NVE Inc. ) and 8 op-amps on a flexible printed circuit board (PCB). The sensor pad can be completely wrapped around the wrist. The flexible PCB is about the width of a piece of paper making it easy to embed into a sleeve. III. U SABILITY T EST AND E VALUATION
(a)
(b)
Fig. 2. 2D XY-coorinate motion of (a) conventional joystick and (b) wearable joystick.
The wearable joystick shown in Fig. 2 interprets wrist motion as motion command to control a robot. As a hand moves around wrist, we can measure the two separate directional information, forward-backward and left-right displacement. The directional information is converted into velocity (r υ ) and rotational (r θ ) information for a robot in Fig. 3. A USAR robot for our usability test is a two-wheeled operation mechanism. We send the relative wheel command which computed by Eq. 1 and 2 to the robot. Two wheel command equations are as follow : for the left wheel:
ωl = α f b × r υ + β f b × r θ
(1)
The ISO ”Guidance on Usability” [11] defines usability as follows: ”the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.” The word ”specified” in the definition provides us with important meaning in the usability context. Our ultimate application is intended to be urban search and rescue. Emergency responders are generally non-technical people wtih little experience operating robots. Furthermore, every emergency situation is different, making it difficult to compare performances from one scenario to the next. Therefore, we endeavor to provide a measurable environment that exhibits relevant aspects of the real environment in order to make comparisons between devices and users. We also want to use non-technical operators in these tests. Our chosen task involves driving a robot from a fixed start position, through a ”gate”, and to a fixed goal position. This is exemplified in the laboratory setup shown in Fig. Fig. 5
Regular Glove (w/t Magnets)
Sensor Pad
Gate to pass
Obstacles USAR robot
CPU & Sensor Interface
Goal Point (a)
Start Point 135mm 18mm
(a)
Bluetooth CPU
(b)
(c)
Fig. 4. (a) Whole wearable joystick system. (b) Main processing unit including CPU, sensor interfaces and Bluetooth (c) Flexible PCB for sensor pad including the GMR sensors and amplifiers.
(a). In addition to the test of driving a real robot, we also implemented a simulated driving task to provide training (Fig. 5 (b)). In [9], the usability mainly consists of 5 topics as follows: learnability, memorability, efficiency, errors, and satisfaction. The time slope to completion with different gate widths and the number of trial are able to show learnability and memorability. In this paper, since we concern with the USAR environment, we consider the efficiency as wearability rather than the efficiency to use. Hence, it can be determined by the degree of the wearability without hindering other tasks. Under already known environment, path tortuosity shows how much accurately an operator can work with an input device (errors). The user satisfaction is one of difficult metrics to measure, since it is highly related to human mental matter. In this paper, we use questionnaire method to determine the satisfaction. A. Background of Fractal Dimension We describe the immense complexity of the path tortuosity with a simple equation, called ”Fractal dimension” [12]. Fractal dimension (FD) has been well known as a way to quantify the complexity with simple equations. There are several methods to determine FD of an object over bounded length scales. The box-counting dimension out of the number of the methods is a popular and easy way to compute. The number of boxes (N) of the scale factor (S) are counted to cover an object to measure. N = SD
(3)
where N is the number of pieces, S is the scale factor and D is Fractal dimension. Therefore, box-counting Fractal dimension (D) can be easily obtained from Eq. 3: D = logN/logS
(4)
Start Point
Goal Point (b)
Fig. 5. (a) The real USAR arena model based on NIST gate model including: USAR robot, obstacles, gate to pass through, and goal position. (b) Screen test software to simulate (a) environment.
IV. D ESIGN OF U SABILITY T EST With screen test software, we conducted usability test for different gate widths and a number of trial. Total 43 undergraduate students from University of Denver were involved in the usability tests for 5 different types of input devices, such as the wearable joystick, a traditional joystick, a mouse position (MousePos), a mouse velocity (MouseVel), and a trackball. MousePos controls the position of a cursor by using the wheel position of the mouse. MouseVel controls the velocity of a cursor by using the wheel velocity of the mouse. In the usability test, we used 4 different gate widths (30, 60, 90 and 120 pixels) to pass through between start point and goal point in Fig. 5 (b). They are asked to perform 5 trials for each test. In order to be objective, we eliminate extremely large and small data among the usability data and then we take the middle range of the data. During the usability test, we observed that both the time to completion and path tortuosity were important parameters to evaluate the input devices. [13], [10] showed that the path tortuosity was highly correlated with (errors).
A. Learnability and Memorability During the screen test, we measured the time to completion slope versus gate widths and the time to completion slope versus the number of trials. In Fig. 6 (a), as the operators tried the test up to 5 trials with same gate width 90 pixels, we measured the time to completion slope versus the trials. With same gate width, we can determine how fast user can adapt to the same operation. The variance of G(wearable joystick) shows that some of the users could easily learn how to operate G but some couldn’t. We could see that the result came from the bad calibration over the operation. The median score of G shows better performance than other devices. In Fig. 6 (b), as we decreased the gate widths from 120 to 30 by 30 pixels, we measured the time to completion slope versus the gate variation. As the tasks became difficult during Fig. 6 (b) test, most slope values were scored around zero. However, the median score of G shows better performance than other devices. In both of the tests, the smallest number in the time to completion slope axis represented the fastest learning, which meant that the operators could be rapidly adapted to the input devices. The ANOVA results of the Fig. 6 (a) and (b) were F-statistic:1.227 and p-value:0.3311 and Fstatistic:1.478 and p-value:0.2464, respectively. Both of the pvalues are greater than 0.05. Therefore, from the p-values, it is clearly verified that the performance of the wearable joystick is not different from other input devices. Further, it proves that the performance of the wearable joystick is evaluated objectively.
B. Efficiency We should design a wearable input device to avoid an encumbrance for urban search and rescue(USAR) team. Thus, the uncumbersome characteristic is an important metric to judge the efficiency to carry the input device. Therefore, we conduct jungle gym test to show the degree of encumbrance by measuring the time to completion versus the number of hands free. We assume that the jungle gym looks alike USAR environment and a suit case is a control box, which contains input devices, such as a traditional joystick, trackball or mouse. The sequence of the jungle gym test is as follows: (i) All of them started in front of the first platform. (ii) They started either with two suite case, 1 suitcase or no suitcase. (iii) They ran through the jungle gym 3 times with each suitcase combo for a total of 9 runs. (iv) the run include steeping up to the platform, crawling through the tube, stepping up another platform, and going down a slide. (v) The stop watch is stopped when they stand up after the slide holding everything they started with. Time to complete in the both hands free case is faster or equal to other cases in Fig. 7. By the test, we can predict that the wearable device which allows both hands free can give an operator more flexibility to move in hazardous USAR environment than traditional input devices. Hands Free Vs. Time to Complete
Time to complete vs Trials
Time slope
Time to Complete (Sec)
25.00
Input devices
20.00 15.00 10.00 5.00 0.00 0
(a)
Time Slope
Time to complete vs Widths
5
10 15 23 Operators Both Hands Free 1 Hand Free
20
25
No Hands Free
Fig. 7. 23 operators conduct this jungle gym test for 3 cases, such as both hands free, 1 hand free and no hands free.
C. Errors
Input devices
(b) Fig. 6. 5 representative operators conducted in this test. (a) Time to completion vs Trials test. and (b) Time to completion vs Widths test. G, J, MP, MV, and T represent wearable joystick, joystick, mouse position, mouse velocity and trackball, respectively.
We measured path tortuosity with FD to evaluate errors. The ANOVA results of the Fig. 8 (a), (b) and (c) are F-statistic:6.97 and p-value:0.001103, F-statistic:0.1716 and p-value:0.6896, and F-statistic:0.5115 and p-value:0.728, respectively. The pvalue of Fig. 8 (a) was less than 0.05, which the wearable joystick could not be claimed that its performance was not different from other input devices for different gate widths. However, the p-value of Fig. 8 (b) was greater than 0.05, which the wearable joystick could claim that its performance was not different from mouse velocity for different gate widths. The pvalue of Fig. 8 (c) was greater than 0.05, which the wearable
FD slope
FD vs Widths
Input devices (a)
FD vs Widths
In Fig. 9 (a), the wearable device does not show good result, since the wearable device has some errors on signal processing and the operators is not still compatible due to short of training time for the device. However, from the p-value of Fig. 9 (b), it is clearly verified that the satisfaction degree of the wearable joystick is not different among mouse velocity and trackball.
Rank(1:best, 5:worst)
joystick could claim that its performance was not different from mouse velocity for the number of trials.
Questionnaire
Input devices (b) FD vs Trials
Rank(1:best, 5:worst)
FD slope
Input devices (a) Questionnaire
Input devices
FD slope
(b) Fig. 9. 13 representative operators conducted in this test. (a) Ranking from the questionnaire for 5 input devices. (b) Ranking from the questionnaire for 3 input devices. G, J, MP, MV, and T represet wearable joystick, joystick, mouse position, mouse velocity and trackball, respectively.
Input devices (c) Fig. 8. 5 representative operators conducted in this test. (a) As decreasing gate widths from 120 to 30 by 30pixels, FD vs widths test for 5 input devices. (b) FD vs widths test for 2 input devices out of 5 input devices in (a). (c) As increasing the number of trials, FD vs trials test for 5 input devices. G, J, MP, MV, and T represet wearable joystick, joystick, mouse position, mouse velocity and trackball, respectively.
Since the calibration of the wearable joystick was not still perfect, the wearable joystick might show some wrong operation in bad installation. However, we envision that the calibration technique will be improved in future work. D. Satisfaction We used questionnaire method to determine the user satisfaction for 5 input devices. At the questionnaire, we asked the operators to rank the input devices with the numbers between 1(the best device) and 5(the worst device) in Fig. 9. The ANOVA results of the Fig. 9 (a) and (b) are F-statistic:7.97 and p-value:0.00003199 and F-statistic:0.907 and p-value:0.4128, respectively.
V. C ONCLUSIONS AND F UTURE WORK Among 5 input devices, this paper has described quantitative comparison based on ANOVA analysis method. The wearable joystick has showed good performance in learnability, memorability and efficiency but bad performance in errors and satisfaction. The bad performance originally comes from the calibration problem which is not solved well. Since the human has different size of wrist and a little bit different gestures, some errors occur from the wrong installation during the operation. In the evaluation, by using both the time to completion and FD, we can quantitatively present the performance result of the input devices in terms of the 5 usability topics. We have presented ANOVA analysis technique to provide quantitative comparison data for the 5 usability topics. As a result, the performance of the wearable joystick was not better than the other conventional input devices but clearly not different with them as a input device. Despite of the drawbacks, the wearable joystick has strong advantages, such as wearability, intuitive and natural way, uncumbersome mechanism, no accessary circuitry, and no battery, for USAR task.
As a future work, we are improving calibration technique and sensor data processing algorithm to obtain accurate sensor data. Then, we will do another usability test again with same input devices. We envision that the errors and satisfaction performance of the wearable joystick will be dramatically improved. VI. ACKNOWLEDGEMENTS This work was sponsored in part by the NSF Safety, Security and Rescue Research Center. The authors wish to thank the National Science Foundation’s Research Robotics for Rescue and Response project and Dr. Robin Murphy for the loan of the Inuktun robot. R EFERENCES [1] R. M. Voyles, A. C. Larson, M. Laoint, and J. Bae, “Core-bored searchand-rescue applications for an agile limbed robot,” in Proceedings of the 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 1, 2004, pp. 58–66. [2] J. Casper and R. Murphy, “Human-robot interactions during the robot assisted urban search and rescue response at the world trade center,” IEEE Systems, Man and Cybernetics, vol. 33, no. 3, June 2003. [3] D. Sturman and D. Zeltzer, “A survey of glove-based input,” in IEEE Computer graphics and Applications, 1994. [4] B. Thomas and W. Piekarski, “Glove based user interaction techniques for augmented reality in an outdoor environment,” Virtual Reality, vol. 6, no. 3, pp. 167–180, 2002. [5] M. Goldstein, D. Chincholle, and M. Backstrom, “Assessing two new wearable input paradigms: The finger-joint-gesture palm-keypad glove and the invisible phone clock.” Personal and Ubiquitous Computing, vol. 4, no. 2/3, 2000. [6] Y. Kim, B. Soh, and S. Lee, “A new wearable input device: Scurry,” IEEE Transactions on Industrial Electronics, vol. 52, no. 6, pp. 1490– 1499, Dec. 2005. [7] J. Rekimoto, “Gesturewrist and gesturepad: Unobtrusive wearable interaction devices,” in IEEE International Symposium on Wearable Computer, 2001. [8] M. Cabral, C. Morimoto, and M. Zuffo, “On the usability of gesture interfaces in virtual reality environments,” in Latin American Conference on Human-Computer Interaction, vol. 2, no. 4, Oct. 2005, pp. 263–283. [9] J. Nielsen, “Usability engineering,” Morgan Kauffmann, 1993. [10] J. Bae and R. Voyles, “Wearable joystick for gloves-on human/computer interaction,” in Proceedings of the SPIE Defense and Security Symposium, Orlando, FL, April 2006. [11] ISO, “Guidance on usability,” ISO9241-11, 1998. [12] M. BB, “How long is the coast of britain? statistical selfsimilarity and fractional dimension,” Science, 1967. [13] M. Voshell, D. Woods, and F. Phillips, “Overcoming the keyhole in human-robot coordination: simulation and evaluation,” in Proceedings of the Human Factors and Ergonomics Society, 2005, pp. 26–30. [14] A. Steinfeld, T. Fong, D. Kaber, M. Lewis, J. Scholtz, A. Schultz, and M. Goodrich, “Common metrics for human-robot interaction,” in Human-Robot Interaction Conference, March 2006.
