Hand Eye Coordination Patterns in Target Selection - CiteSeerX

Hand Eye Coordination Patterns in Target Selection Barton A. Smith, Janet Ho, Wendy Ark, and Shumin Zhai IBM Almaden Research Center 650 Harry Road San Jose, CA 95120 USA +1 408 927 1606

{basmith, zhai}@almaden.ibm.com ABSTRACT In this paper, we describe the use of eye gaze tracking and trajectory analysis in the testing of the performance of input devices for cursor control in Graphical User Interfaces (GUIs). By closely studying the behavior of test subjects performing pointing tasks, we can gain a more detailed understanding of the device design factors that may influence the overall performance with these devices. Our Results show there are many patterns of hand eye coordination at the computer interface which differ from patterns found in direct hand pointing at physical targets (Byrne, Anderson, Douglass, & Matessa, 1999).

Keywords Eye tracking, gaze, hand eye coordination, pointing, target selection, mouse, touchpad, pointing stick, motor control

1. INTRODUCTION Human computer interface research has traditionally focused on performance. A typical topic of such a nature is computer input. Input devices and techniques are usually tested against a set of standard tasks in which user’s performance on task completion time and error rate is measured and analyzed (e.g. Card, English, & Burr, 1978). The results of the performance analysis serve as the basis for refinement and redesign of the devices and techniques. However, observations at the performance level often overlook important information on how users actually accomplish the task, which may offer additional insights toward a better understanding of the interaction process and design solutions. As the field matures, process oriented research, should begin to contribute to the understanding of interaction. In human motor control research, the study of the “micro-structure” has served similar purposes (Jagacinski, Repperger, Moran, Ward, & Class, 1980). In order to understand the usability of various 6 degree-offreedom (DOF) devices, Zhai and Milgram (Zhai, 1998) recently studied both the performance and the trajectory of 6 DOF manipulations of 3D objects. Their trajectory analysis revealed critical differences between devices in terms of coordination, which would not be found in time performance data alone.

In conjunction with trajectory analysis, Eye-tracking provides a comprehensive approach to studying interaction processes. In the field of HCI, eye tracking has helped to improve the understanding of how users search and select menu items (Card, 1982), (Aaltonen, Hyrskykari, & Raiha, 1998), (Byrne et al., 1999). Eye tracking has also been used in studying human pointing tasks with hands in the physical world. Given that most motor control movement is either initiated or guided by perception, it is necessary to understand how eye gaze relates to hand movement. For example, Helsen and colleagues studied the temporal and spatial coupling of gaze and hand movement (Helsen, Elliott, Starkes, & Ricker, 1998). In a reciprocal pointing task with two fixed targets, they found a rather invariant patterns of hand eye movement relationship: eye gaze tended be initiated 70 ms earlier than hand movement; eye gaze typically makes two saccades to land on target and the first saccade tended to undershoot. The pattern of task termination was also very consistent: eye gaze stabilizes on target at 50% of the total hand response time. Pointing on a computing screen with an input device may or may not follow the patterns found in pointing with the hand to physical targets. There are many reasons for different behavior, due to the various “disparities” between the hand motion and the cursor motion (Wang & MacKenzie, 1999). For examples, direct hand pointing is carried out with proprioceptive feedback of hand position in the human arm. Pointing at graphical objects on a screen is carried out with a cursor, which does not have a direct, absolute mapping with hand motion. This means that the user may have to sample the cursor location with gaze in the course of a pointing trial, unless the cursor motion can be perceived by peripheral vision. The mapping between cursor motion and input device is often a complex transfer function, which may further increase the complexity of the hand eye relationship in target acquisition tasks with a computer cursor. In the case of computer mice, most of them are power mice with non-linear acceleration schemes. More precisely, the control gain in a power mouse is not a constant, but depends on the speed of the mouse motion. Faster movement of the mouse results in higher control gain. In the case of a pointing stick such as Trackpoint®1, the input force is mapped onto velocity by a complex transfer function (Rutledge and Selker), with various plateaus to provide a cursor speed easier for the eye gaze to follow. Detailed study of gaze pattern is surely useful for further refining the transfer functions in these devices. 1

TrackPoint is a trademark of the International Business Machines Corporation.

In the case of a small touchpad often seen in laptop computers, multiple strokes often have to be made in order to move the cursor to a distant target. Does this mean the user has to gaze at the cursor in order to make each stroke? In summary, understanding the eye hand relationship serves as an important foundation for understanding and designing input methods. Study has shown invariant hand-eye coordination patterns in direct hand pointing tasks (Helsen et al). The disparities between hand and cursor motion on GUI interface suggest possibly much more complex hand eye behavior in computer target acquisition tasks, such as occasional gaze switch between cursor and target, or gaze focus on cursor. This study makes an initial attempt to test and understand hand eye coordination patterns at the computer interface.

2. METHODS 2.1 Participants 24 volunteers participated in the experiment. All participants had normal or corrected to normal vision. All were right-handed and were experienced computer users in the Windows platform with at least three years of continuous usage. Half of the participants had experience with the pointing stick. None of the participants had experience with the touchpad.

2.2 Experimental Design The participants were required to perform two different tasks with three input devices. The three input devices in question were mouse, touchpad, and pointing stick. Each participant switched to another input device after performing the two different tasks. In total, each participant performed six tasks. The order in which they used the input devices and the order of the two tasks were randomly counter-balanced. Task one was a reciprocal pointing task. A pair of identically sized circular targets was placed diametrically around the center of the computer monitor at specified distances and directions. Targets were presented with all possible combinations of the following: distance, radius, and angle. The center-to-center distances were 200, 400, and 600 pixels. The radii were 10, 20, and 30 pixels. The angles (from horizontal) were –45, -30, 0, 30, and 45 degrees. Each target pair was used for two trials, resulting in a total of 90 trials for this task. The participants were to look at the monitor screen and use the input device to point and alternately select by clicking on the presented target circles. Task two was a random pointing task. The participants pointed and clicked on a set of randomly distributed circular targets presented sequentially on the monitor screen. The targets had radii of 10, 20, and 30 pixels in random order. Each participant was also required to complete ninety trials of this task. Task 1 is most common in Fitts’ law based input device research. Task 2 is closer to what a user typically does on a computer screen by pointing. The key difference between the two types of tasks lies in the predictability of the target. For Task 1, the first click on a pair of targets started the actual data collection for that pair of targets. For the following two measured trials, the participant already knew where the next target was. For Task 2, the participant could not predict where a target would appear until the trial actually began. This difference may influence the hand-eye coordination pattern.

Each subject received exactly the same set of targets. The target generation program used was a Java application called IDTest, available from http://www.almaden.ibm.com/u/basmith/. The IDTest program ran on a Pentium-based 167 MHz computer. The targets were displayed at a resolution of 1024 by 768 pixels by high colors (16 bits) on an IBM P201 monitor, using an ATI 3D PRO Turbo PC2TV video card. The viewable area of the screen was 0.365 m horizontal by 0.28 m vertical at a distance of 0.64 m from the eye. The screen refresh rate was 90 Hz. Three input devices were used in this experiment: an IBM mouse (model 12J3618), a Cirque SmartCat™2 touchpad, and an IBM TrackPoint pointing stick in a desktop keyboard.

2.3 Eye-tracking system Eye gaze position was tracked by an Applied Science Laboratories (ASL) Model 504 eye tracker unit. The tracking software ran on a Pentium® 200MHz computer. This unit tracks gaze position by observing the position of the pupil and front surface reflection from a single eye. A chin rest was used to stabilize the participants’ viewing position and distance. A scan converter (Focus Enhancements’ TView Gold) was used to produce a combined video image signal of the targets displayed on the computer monitor and the eye position calculated by the eyetracker unit. This composite view was used by the experimenter to verify that the gaze tracking was working. Eye movement data were recorded every 60th of a second (60 Hz update rate) and averaged over every four data points through the ASL eye tracker interface program. The cursor coordinates were recorded by IDTest. The eyetracker movement data were streamed into the P167 computer through a serial port and IDTest then combined eyetracker movement data and cursor data in one single data file with respect to time. The eyetracking data obtained from the ASL eyetracker were converted into the same coordinate system as the cursor coordinates. The calibration points on the ASL eyetracker and the stimulus machine were used to obtain the parameters for the conversion.

3. RESULTS We first look at the effect of device on overall pointing performance. Figure 1 shows the task completion time (the sum of all 90 trial completion times) for each device averaged over all tasks and subjects. Figure 1 shows that the device significantly affected performance time, F(2,44)=50.19, p