One was the use of virtual cubits, the distance of the user's ... We used meters as opposed to virtual cubits (Poupyrev et al., 1998) because this experiment only ...
Baseline Factors for Raycasting Selection Chadwick A. Wingrave
Doug A. Bowman
3D Interaction Group, Virginia Tech (cwingrav | bowman)@vt.edu
Abstract Selection is a common task in Virtual Environment (VE) interfaces. Several techniques have been created to perform the task and two testbed studies have performed comparison studies between the techniques. In this study focusing on Raycasting selection, we created an understanding of the known factors influencing selection and create a model of selection time and angular error. We found evidence that users try to maximize visual feedback by purposefully creating angular error and developed an argument for a possible visual feedback parameter to explain behavior across multiple studies.
1.
Introduction
Selection in Virtual Environments (VEs) is one of the universal tasks along with travel, manipulation, symbolic input, system control and wayfinding (Bowman, Kruijff, LaViola & Poupyrev, 2004). It deals with indicating objects and has been implemented with multiple techniques, each trying to improve performance measures such as accuracy, speed and comfort. Work in the past has compared the differing selection techniques in testbed environments (Poupyrev, Weghorst, Billinghurst & Ichikawa, 1998)(Bowman, Johnson & Hodges, 1999) showing that there is no best technique but only better techniques under certain combinations of user, task, and environment characteristics.
Figure 1. The test environment after a Raycasting trial presented the participant with feedback that motivated their continuing performance.
This experiment is a meta-analysis of Raycasting experiments; it develops a methodology for creating models of interaction techniques which results in a predictive model of Raycasting. We focus on the creation of a model of Raycasting selection [Figure 1] (Bowman et al., 2004) because it is a commonly used selection technique, can operate at many distances, produces low levels of fatigue, is fast and is easily understandable. Raycasting selection, according to (Bowman et al., 2004, p 151), is where “...the user points at objects with a virtual ray that defines the direction of pointing and a virtual line segment attached to the hand visualizes the pointing direction”. A button is usually pressed to signal when to select. The development of a model of Raycasting will enable researchers to compare performance of new techniques and past experiments without spending resources on human testing. As new experimental results emerge, new factors can be added to improve the model. This paper is split into 4 sections. The first section gives a background of Raycasting and the types of experiments that are relevant to this work. The second section will explain our experimental setup. The third and fourth sections will give our results in terms of interesting discoveries and models created from the experiment. We will end with conclusions and future work.
2.
Background
There have been multiple factors considered in experiments that compare selection techniques. The results have always been considered as comparisons between techniques and not as a part of a model for a single technique. Additionally, without knowing all the factors involved in a selection technique, meta-comparisons between studies become difficult because information important to the comparison is not reported, confounding the comparison. Here, we will look at the factors studied in two VE testbeds of selection techniques and review similar Raycasting studies conducted in the real world. (Bowman et al., 1999) compared three different types of selection techniques (Go-Go, Raycasting and Occlusion selection (Pierce, Forsberg, Conway, Hong, Zeleznik & Mine, 1997)) on tasks that varied the distance from the user to the object, the size of the target object and the density of objects around the target object. They found that the Go-Go technique was slower than the other techniques and that overall nearer and larger objects were easier to select. They also found occlusion selection produced more fatigue than Raycasting. When considering demographic information, males were faster across all techniques and spatial ability and VE experience did not predict performance. There was, however, no model of how selection time increases as a function of both the object size and distance from the user. The authors did not model fatigue as a function of time in the environment and tasks performed, nor was there a model of how user demographics affected performance. Gender was listed as having an effect on selection time but recent experiments in VEs conflict, showing that the gender effect can drop off with time (Buckwalter, Rizzo, van der Zaag, van Rooyen, Larson & Thiebaux, 1999). Also, spatial ability and experience with VEs was listed as not having an effect but later studies disagree with this (Wingrave, Tintner, Walker, Bowman & Hodges, 2005). Overall, comparative guidelines were the goal of the study and not models of the techniques. (Poupyrev et al., 1998) also compared the Go-Go and Raycasting techniques in a similar testbed environment examining the effects of varying distance to object, object size, interaction technique and visual feedback. This study had two major differences from (Bowman et al., 1999). One was the use of virtual cubits, the distance of the user’s maximum reach, to define distance while the other was the use of target visual size to describe objects so as to separate the effect of distance and object size on user performance. Users in this study had moderate experience with VEs as compared to (Bowman et al., 1999) which recruited randomly. Distance, size and visual feedback were all significant as were the interactions between technique and distance, size and visual feedback but neither Go-Go nor Raycasting were categorically best. The techniques were similar up close but as distance increased, Raycasting performance decreased faster than Go-Go, especially for smaller objects. Visual feedback was shown to improve selection at a distance. In general, Raycasting with visual feedback was better unless selecting objects requiring higher visual accuracy. Subjects rated Go-Go as more enjoyable and intuitive just as in (Bowman et al., 1999). While the detailed discussion of the interactions was useful, this study did not create a predictive model of the factors. Additionally, it did not consider user differences, although it did control them somewhat by choosing participants with some VE experience. Many studies dealing with real world pointing can potentially be applied to VR pointing. Most of this research falls under motion planning and the effects of visual and motor systems on speed and accuracy. A good review can be found in (Desmurget, Pelisson, Rossetti & Prablanc, 1998). Studies in non-virtual environments have limitations in their applicability to VE selection tasks. Studies dealing with real world objects such as (Hagg & Hallbeck, 2001) had limited depth in their study of tool handles. Laser pointer studies (Myers, Bhatnagar, Nichols, Peck, Kong, Miller & Long, 2002) (Oh & Stuerzlinger, 2002) are similar but differ in that the laser point only appears at the plane of the target and not as a ray to the target. Laser and tool handle studies both correspond in their results that inline handles are more accurate and grips, like a saw or gun, are less accurate.
3.
Methodology
Previous experiments have identified factors that influence user performance on selection tasks. They have also shown factors that can be used to differentiate performance between the selection techniques but have not developed predictive models using the factors. The goal of our experiment was to gather the data needed to create a model of the known factors that influence performance.
3.1. Purpose This experiment seeks to develop models that will predict user performance time and accuracy. (Wingrave et al, 2005) showed that users of Raycasting and Occlusion selection in VEs employ strategies to increase performance according to conditions of the task. Because of this, developing a model of selection requires a model of the task conditions as well. This experiment has the goal of producing a predictive model of Raycasting selection in a simple
environment with as few affecting conditions as possible. Conditions such as target position and size, user expertise and individual differences, feedback of the techniques, and practice at the task will be evaluated for their usefulness in models of performance. This process will most likely lead to other concepts that might be predictors of performance. This work specifically ignores the effects of distractor object density around the target object as in (Bowman et al., 1999) and the occlusion of the target objects by distractor objects as in (Wingrave et al., 2005).
3.2. Independent and dependent variables Three within-subject variables were chosen. The first was object depth, with values of 4, 10 and 20 meters from the user. We used meters as opposed to virtual cubits (Poupyrev et al., 1998) because this experiment only considers Raycasting selection, a 2D raybased selection technique that is not affected by the user’s arm length. The second variable was the target object position varied along 5 horizontal positions (+35,+15,0,-15,-35 degrees) and 3 vertical positions (+35,0,-35 degrees). Additionally, object visual size was used as it takes into account the two factors of depth and size. (Poupyrev et al., 1998) mentioned that visual size does not account for the difference in object depth, but this is only important for selection techniques like Go-Go which require users to position a virtual hand at a particular depth. Since we used Raycasting, visual size does indeed account for both object size and depth. We varied visual size by 3-5, 8-13 and 14-20 degrees [Figure 2]. This gave us a 3x5x3x3 within-subjects design. Two between-subject variables were chosen. We varied expertise so novices and experienced 3D gamers and/or VE gamers were compared against each other. Computer experience, especially for fun and graphics was shown in (Wingrave et al, 2005) to correlate with improved Raycasting performance. Additionally, we changed visual feedback conditions so that one group saw an object highlight when pointing within 10 degrees of an object, while the other group’s objects did not provide this feedback. We measured multiple dependent variables. Three measures of performance were considered but only two were used. The first performance measure was angular error from the center of the target object at the end of a trial. The second measure of performance was time to complete a selection. The last measure (ultimately not used) was the number of misses (incorrect selections). Since a common method of implementing Raycasting is with a certain amount of acceptable error, the techniques accepted 10 degrees of angular error. Because of this, there was not a single miss during the experiment, which we attribute in some part to our motivation of participants (see section 3.4). These are common measures of performance in the literature. Raycasting requires users to consider the orientation of a ray in space, so we accounted for user spatial aptitude by the redrawn Mental Rotations Test (MRT) (Vandenburg & Kuse, 1978)(Peters, Laeng, Latham, Jackson, Zaiyouna & Richardson, 1995). Additionally, practice at the task was considered so trial number was included in the analysis.
3.3
Experimental procedure
We recruited participants through undergraduate and graduate computer science class listservs, as well as by word of mouth. Participants were then screened and selected for having VE experience and/or 3D game playing experience. The novices were generally found by word of mouth because of the difficulty in finding subjects of this type in a computer science department. Additionally, because of the literature regarding spatial ability differences between genders, only males were used. Eight participants, four experts and four novices between the ages of 18 and 25, were then selected. No participant was colorblind. Once in the lab, the participants were given a mental rotations test and a comfort ratings test before the VE testing began. The experiment was explained to them as was the equipment used in the study. Additionally, the method of evaluating the participants against each other was explained (see section 3.4). They were also told that they had several practice trials before data was to be taken which gave them the ability
Figure 2. The placement of objects in the environment is along 5 horizontal directions, 3 vertical and 3 depths. The shown sphere is 20m away, +15 horizontal and +35 vertical.
to experiment with the interface and become comfortable before they competed. We used a Virtual Research V8 head mounted display which, along with a wand device, was tracked with an InterSense IS-900 VET tracking system. The application was written with the Simple Virtual Environment Library (Kessler, Bowman & Hodges, 2000) and CHASM (Wingrave & Bowman, 2005) and ran on an Apple Powerbook G4 550 MHz. The application did not tax the hardware and frame rates did not fall bellow 30 fps. The environment used in all phases of this experiment had the user standing on the edge of a pit with a mountain in the background [Figure 1]. The pit and mountain had no effect on the experiment, other than providing a general reference for the users to understand as ‘forward’, and with which they could align themselves. The user’s hand was represented with a simple virtual hand avatar. Connected to the virtual hand was a ray used to point into the environment. The ray extended beyond the far clipping plane in the environment and was 4 cm in width and height. The participants were given 5 practice trials and were allowed to ask any questions at this time. After this, three sets of 45 trials (the 135 within-subject trials were randomly ordered) were given to the participants with a short break between trials where users were given the comfort ratings test again to monitor their ability to continue. No participants dropped out of the experiment and comfort levels were never low. After the trials, a post hoc questionnaire was given to gain participants’ impressions of the experiment. They were also asked verbally about the experiment. Participants moved through trials at their own pace. The first scene they were shown displayed two buttons floating in front of their face, titled ‘Prepare’ and ‘Go’ with ‘Prepare’ highlighted. They had to point at the ‘Prepare’ button and press the InterSense wand button. This caused the target object to appear, move from its current location to the target location over the course of 3 seconds, scale to the proper visual size over 3 seconds and then disappear (see section 3.5). Translation and scaling were separated so users understood the depth of the sphere which would have been lost if scaling occurred while translating the sphere. The two buttons then reappeared, this time with the ‘Go’ button highlighted. Once pressed, the trial began and users had to point at the target object and press the wand button. For participants with feedback on, the target object turned yellow when participants pointed near the target. Once they pointed to within 10 degrees of error from the edge of the target object and pressed the wand button, the target object disappeared and information appeared displaying trial time, their average time so far and the number of misses in the last trial. After the last trial, the experiment stopped; otherwise the two buttons appeared and the trial repeated. The ‘Go’ button was positioned directly in front of the user’s face at a distance so that the trial always started with the ray pointing roughly at the same position. Participants have different motivational forces acting on them which can invalidate data. One motivation is to perform well so as to help the researcher while another might just be to perform well so as to not be embarrassed. From our experience with this type of experiment, the dominating force after a period of time is to complete the experiment so the participant can leave. This is problematic for comparing early and later trials. To motivate them, we displayed participants’ trial times to give them something to measure their performance with and to foster their competitiveness. Performance feedback was successful in (Bowman & Wingrave, 2001). In this experiment, participants were aware of their improvement and made many comments during the trials about particularly fast or slow trials, leading us to believe they were motivated from start to finish despite the long trial times.
3.4
Speed-accuracy tradeoff
There is a tradeoff when asking participants to perform both quickly and accurately, as they cannot optimize for both simultaneously. This leads to the compromise between accuracy and speed that plagues most experiments. To address this issue, we developed a method of rating user performance that had a built-in sliding scale of the speedaccuracy tradeoff. To score the users, the fastest participant was given 1 point, the 2nd fastest was given 2, etc. and the participant with the fewest misses was given 1 point, the 2nd was given 2, etc. The participant with the lowest total score combining the two measures was given a prize of two movie tickets. We believe this is a good representation of the real world speed/accuracy tradeoff. Participants that prefer speed or accuracy more will not be penalized since they are competing not in relation to a performance function, but are competing against other participants where performance is determined by the group, just as it would be on any real-world application. We did not score participants on their angular error of selection and made this clear. The goal of real-world selection tasks is to select objects, not point directly at their center. Had our performance measure included angular error, it would undoubtedly have affected the results.
3.5
Pilot study
We performed a pilot study to test our method and setup, resulting in the study described above. During the pilot study, the target object was constantly visible, which allowed participants to practice selecting the object before the
trial began. This decreased their trial time and error rate. Since we wanted to make sure participants knew where the target object was going to appear so as to not confound the selection time with a search task, we could not just remove the target object from the scene until the trial started. To stop the participants from practicing trials and not involve a search component, the target object was hidden after it was animated into position and it then reappeared when the ‘Go’ button was displayed. Thus, the participants knew generally where the target object was and were not required to perform a search task. Additionally, the target could potentially be hard to see at further distances. This was overcome by animating the target object into position, the motion making it easy to see.
4.
Results
4.1. Object visual size vs object actual size As speculated in (Poupyrev et al., 1998), visual size correlates more with error and trial time than actual size. This was supported by our experiment with visual size being more negatively correlated with time (the larger, the faster the selection) than actual size (r2=.0918,p
