Calibration of Locomotion due to Visual Motion in a Treadmill-based Virtual Environment Betty J. Mohler, William B. Thompson, Sarah H. Creem-Regehr and Peter Willemsen University of Utah and Herbert L. Pick, Jr. University of Minnesota and John J. Rieser Vanderbilt University
This paper describes the use of a treadmill-based virtual environment (VE) to investigate the influence of visual motion on locomotion. First, we establish that a computer-controlled treadmill coupled with a wide field of view computer graphics display can be used to study interactions between perception and action. Previous work has demonstrated that humans recalibrate their visually-directed actions to changing circumstances in their environment. Using a treadmill VE, we show that recalibration of action is reflected in the real world as a result of manipulating the relation between the visual indication of speed, presented using computer graphics, and the biomechanical speed of walking on a treadmill. We then extend this methodology to investigate whether the recalibration is based on perception of the speed of movement through the world or on the magnitude of optic flow itself. This was done by utilizing two different visual displays which had essentially the same magnitude of optic flow but which differed in the information present for the speed of forward motion. These results indicate that changes in optic flow are not necessary for recalibration to occur. The recalibration effect is dependent at least in part on visual perception of the speed of self-movement. Categories and Subject Descriptors: I.3.7 [Computer Graphics]: Three-Dimensional Graphics and Realism—Virtual reality General Terms: Experimentation, Human Factors Additional Key Words and Phrases: treadmill virtual environments, locomotion, visual self-motion
Authors’ addresses: Betty J. Mohler, William B. Thompson, and Peter Willemsen, School of Computing, University of Utah, 50 So. Central Campus Dr., Room MEB-3190, Salt Lake City, UT 84112-9205; email:
[email protected],
[email protected],
[email protected]; Sarah H. Creem-Regehr, Department of Psychology, University of Utah, 380 South 1530 East, Room 502, Salt Lake City, Utah 84112-0251; email:
[email protected]; Herbert L. Pick, Jr., Institute of Child Development, University of Utah, 51 East River Road, Minneapolis, MN 55455-0345; email:
[email protected]; John J. Rieser, Department of Psychology and Human Development, Box 512 GPC, Vanderbilt University, Nashville, TN 37203; email:
[email protected]. Permission to make digital/hard copy of all or part of this material without fee for personal or classroom use provided that the copies are not made or distributed for profit or commercial advantage, the ACM copyright/server notice, the title of the publication, and its date appear, and notice is given that copying is by permission of the ACM, Inc. To copy otherwise, to republish, to post on servers, or to redistribute to lists requires prior specific permission and/or a fee. c 20YY ACM 0000-0000/20YY/0000-0001 $5.00
ACM Transactions on Applied Perception, Vol. V, No. N, Month 20YY, Pages 1–17.
To appear in the ACM Transactions on Applied Perception
2
1.
·
Betty J. Mohler et al.
INTRODUCTION
Treadmill virtual environments (VEs), in which treadmill walking is coupled to an appropriately updating visual display, have the potential to allow natural walking through large-scale simulated spaces. The availability of such devices could impact a broad range of applications, including education and training, design and prototyping, physical fitness, and rehabilitation. For some of these applications, natural walking provides a level of realism not obtainable if movement through the simulated world is controlled by devices such as a joystick [Usoh et al. 1999]. For other applications, realistic walking is a fundamental requirement. The utility of treadmill VEs for many applications depends in large part on the presumption that perception-action coupling in the VE operates in a manner similar to the real world. If this is true, then treadmill VE systems can also serve as a tool for better understanding the interaction between vision and locomotion in humans. Investigations of perception-action coupling in treadmill VE systems have been relatively limited. One study in a treadmill VE has shown that VE distance judgments were compressed more than in the real world [Witmer and Sadowski 1998]. It has also been shown that postural sway and entrainment of the step cycle to the visual display can be induced in a treadmill VE [Warren et al. 1996; Kay and Warren 2001]. Pelah et al. [1998] designed a treadmill VE to investigate visuo-motor interactions in locomotion. Their treadmill VE was designed with the intention of investigating adaptation to combinations of optic flow, walking, running or cycling. With this treadmill VE they found a reduced perception of self-motion relative to that indicated by the visual motion [Thurrell et al. 1998]. They also argue that the reduction of perceived visual speed during walking was dependent upon the visual stimulus indicating locomotion [Thurrell and Pelah 2002] and that attentional and vestibular mechanisms do not contribute to the reduction of perceived visual speed [Pelah et al. 2002]. More recently, the subjective sense of how well the visual speed of graphics match biomechanical walking speed has been explored in a treadmill VE [Banton et al. 2005; Durgin, Gigone et al. 2005]. In these studies, visual motion rendered to correspond to the actual speed of walking appeared too slow. Stappers used a treadmill VE to argue that both visual information and proprioceptive information influences a person’s perceived walking speed [Stappers 1996]. The goal of the present research was to show that the perception-action coupling in our treadmill-VE is similar to the real world, to further investigate visual-motor recalibration effects and to specifically investigate a question about the nature of the visual information that contributes to visual-motor recalibration. This last issue involved the use of a manipulation that could not be performed in the real world, but was straightforward on a treadmill-VE system. The present work contributes to our understanding of perception-action couplings in treadmill VEs in two ways. We start by showing that the visual-motor calibration of human locomotion in the real world can be altered by prior walking on a treadmill VE when the visual flow associated with self-motion is mismatched relative to biomechanical walking speed. This replicates a previous experiment done without the use of computer graphics or other “virtual” displays [Rieser et al. 1995]. We follow this with a study which explores more specifically the visual cues that are involved in the visual-motor calibration of human locomotion. We investigate the influence of 3–D perception of ACM Transactions on Applied Perception, Vol. V, No. N, Month 20YY.
Calibration of Locomotion due to Visual Motion in a Treadmill-based Virtual Environment
·
self-motion on the visual-motor calibration of human locomotion while we maintain essentially constant magnitude of optic flow. 2.
A TREADMILL-BASED VIRTUAL ENVIRONMENT
Most previous research involving treadmill VEs used exercise treadmills coupled with either head-mounted displays or fixed video displays of limited extent. In a treadmill virtual environment natural walking is difficult to accomplish. Though treadmills easily allow for walking infinitely in one direction, natural turning is problematic. Most treadmills force a constant walking speed. If variable walking speed is supported, the inertial forces associated with acceleration and deceleration are incorrect. Also, many exercise treadmills are limited by the need for a handrail. This could be an important non-visual cue for the lack of actual self-motion [Durgin, Pelah et al. 2005]. In addition, completely immersive visual display systems are not yet available for natural locomotion. The results described below utilized a system supporting more natural walking and running as well as a substantially more visually immersive experience than most treadmill VEs. Walking was done on the Sarcos Treadport [Hollerbach et al. 2000], a custom built computer controlled treadmill with a 10’ longitudinal by 6’ lateral walking surface (See Figure 1).1 Three 8’ by 8’ back-projected screens with borderless mounts are used. The bottom of the screens is even with the walking surface. The orientation of the two side screens is 60◦ from the main screen. This results in an approximately 180◦ horizontal field-of-view for someone standing near the center of the belt. The viewing distance to each of the three screens is approximately 2 meters. Extensive light shielding is used to minimize the visibility of the treadmill and other items in the laboratory, though the belt and frame of the treadmill are clearly visible to someone walking on the Treadport. Stereo display is not currently supported. The translational position of viewpoint was based on sensed body position. Eye height was fixed based on measured values obtained while the user was standing still, since dynamic adjustments of rendering eye height based on tracking is disorienting for some users. Rendering frame rate was ≥ 30 fps at all times. For these experiments, belt speed was held constant within trials. This reduced the effects of rendering latency on gait control, though as with all other virtual environment systems delays in viewpoint updating may adversely effect the experience of natural locomotion. 3.
CALIBRATION OF LOCOMOTION
Perceptually guided actions require that the appropriate scaling be maintained between visual and proprioceptive information about body movements. Rieser et al. [1995] demonstrated that this scaling is adaptive and that recalibration can occur rapidly as a result of changing circumstances as a person interacts with the world. 1 Visible in the figure is a harness worn by the user and two mechanical devices attached to the harness. One of these attachments is a safety strap which protects the user in case of a fall. The other provides position sensing for establishing the correct viewpoint for the graphics and for situations in which locomotion speed is under user control, and can apply forces to simulate walking up and down hills. In the work described here, neither user control of speed nor slope forces were enabled. A more detailed technical description of the Sarcos Treadport is available in [Hollerbach et al. 2000].
ACM Transactions on Applied Perception, Vol. V, No. N, Month 20YY.
3
4
·
Betty J. Mohler et al.
Fig. 1.
Sarcos Treadport virtual environment
Their methodology involved a test of subjects walking without vision to a previously viewed target before and after an adaptation period with vision. Numerous studies have found this blind walking task under normal circumstances to reveal accurate distance perception from a range of 2 to 20 meters [Rieser et al. 1990; Loomis et al. 1992]. The manipulation used in Rieser et al. [1995] involved an adaptation phase in which subjects walked on a treadmill while experiencing visual motion consistent with either faster or slower self-motion than the walking speed on the treadmill belt. The experimental apparatus involved an exercise treadmill towed on a trailer behind a tractor, thus allowing control of a subject’s motion through the world to be independent of walking speed. Rieser et al. [1995] found that subjects’ performance on a blind walking task was influenced by the treadmill-tractor manipulation. Subjects who experienced visual motion that was slower than their biomechanical walking speed overshot the distance to the target in the blind walking post-test, relative to their performance on the pre-test. Those who experienced visual motion that was faster than their speed of walking showed an undershoot in the post-test relative to the pretest. An account for this distinction in walking performance after the treadmill intervention is a recalibration of walking to visual flow. At least two studies have been published exploring this recalibration effect in virtual environments. Withagen and Michaels [2002] placed an exercise treadmill inside a CAVE immersive visual display which used 3m x 3m backprojected screens for three walls and the floor and supported stereoscopic viewing. Their results were similar to Rieser et al. [1995] in that for a given target distance, post-adaptation blind walking distance increased as the visual speed decreased relative to the speed of the treadmill belt. Unlike Rieser’s results, however, subjects walked farther than the pre-adaptation distance for visually slower, visually same, and visually faster ACM Transactions on Applied Perception, Vol. V, No. N, Month 20YY.
Calibration of Locomotion due to Visual Motion in a Treadmill-based Virtual Environment
·
presentations, rather than showing the undershoot Rieser found for his visually faster condition. However, interpretation of these results was made difficult by variations in walking speed across the different conditions. Durgin et al. [2002] investigated the same recalibration effect using a head mounted display (HMD) while walking on the ground. They reported a substantially larger magnitude of overshoot in their visually slower condition than the equivalent case in Rieser et al. [1995], which they presumed was due to walking over the ground rather than on a treadmill during adaptation. Evidence of perceptual-motor calibration extends beyond the laboratory effects of blind walking tasks, demonstrating the importance of systems that adjust for changing environmental circumstances. For example, Pelah and Barlow [1996] induced a visual motion illusion by having subjects run on a stationary treadmill in the real world in several different circumstances such as with eyes open, blindfolded, restricted field-of-view and running close to a textured wall. They also investigated these same visual motion illusions after running outdoors or riding on a stationary bicycle. Their work provides support for how important immediate perceptual-motor calibration can be for adaptation to a changing environment. This is understood in our every day experience by how rapidly and accurately humans can adapt their behavior to changing circumstances. For example, we easily adjust locomotion to changes in wind speed and direction, load and grade. The first part of our study investigated whether or not the Treadport produces a sufficiently compelling sensory experience as to generate the same sort of recalibration as witnessed in these earlier studies. As with Rieser et al. [1995] (and different from Durgin et al. [2002]), we evaluated the effects of recalibration with a task conducted in the real world. 3.1
Method
Twenty-four subjects (12 male, 12 female) participated in the experiment. All were tested to ensure that they had normal or corrected-to-normal vision, and their eyeheight was measured. Viewing was binocular. This introduces an inherent sensory conflict that was present in all conditions. Each subject was given practice walking without vision in the real world for five minutes. This procedure helped to increase trust between the subject and the experimenter and allowed the subject to become more familiar with the experience of walking naturally without vision. Following the training session, subjects performed in pre-test, adaptation, and post-test phases of the experiment. Each subject participated in one of the three visual conditions: visually slower, visually faster or visually same. Gender was balanced among the conditions. The pre-test involved walking without vision to previously viewed targets on the floor in a real hallway. The targets were placed at distances of six, eight and ten meters, each presented three times for a total of nine trials. The subjects viewed a target on the floor, were instructed to create a “good image” of the target and the surrounding environment, and then to walk blindfolded with eyes closed to the target location. The adaptation phase involved walking on the Treadport for ten minutes while viewing an “endless hallway,” rendered to look similar to an actual hallway in our engineering building (Figure 2). Subjects then walked on the Treadport at a fixed speed of 1.0 m/s while viewing visuals presented in one of three ACM Transactions on Applied Perception, Vol. V, No. N, Month 20YY.
5
6
·
Betty J. Mohler et al.
Fig. 2. Realistic hallway used to provide a visual indication of self-motion for the first experiment. The images from all three screens are shown.
conditions: visually slower (the visual speed was 0.5× the walking speed), visually same (the visual speed was matched to the walking speed) or visually faster (the visual speed was 2.0× the walking speed). People walking on the Treadport often narrowly focus their attention straight ahead. Their sense of presence is increased if they are given reason to vary where they are looking over the field of view covered by the projection screens. Looking to the side may also play a role in the visual perception of the speed of self motion [Banton et al. 2005]. Therefore, subjects were asked to call out “left” or “right” when a double door was directly to the right or left of them. This task also provides a distraction from the mechanical aspects of the Treadport, which cannot be completely masked from the user. Immediately after the adaptation phase, subjects were guided back to the real world hallway without vision and performed the same blind walking task as in the pre-test. Posttest trials were presented with the same distances in the same order as the pre-test. No feedback about walking accuracy was given provided for any trial. 3.2
Results and Discussion
When exposed to the visually faster condition, subjects undershot the distances in the post-test trials by an average of 6% relative to the pre-test. Given the visually slower condition, they overshot the distances in the post-trials by an average of 11%. In the visually same condition, subjects overshot by an average of 3% (See Figure 3). Paired t-tests confirmed a significant difference between the percent change between the pre- and post-tests for each condition (visually slower: t(7) = 6.57, p < .01; visually same: t(7) = 4.34, p < .01; visually faster: t(7) = −4.92, p < .01). A univariate ANOVA comparing the effect of visual condition on the post-pretest difference showed that there was a significant effect of visual condition, F (2, 23) = 43.7, p < .01. Planned contrasts showed that the overshoot in the visually slower condition was reliably greater than the overshoot in the visually same condition (mean difference = 7.1, SE = 1.83, p < .01) and that the undershoot in the visually faster condition was reliably lower than the visually same condition (mean difference = −10.0, SE = 1.83, p < .01). Our results show that in a VE when the speed of visual motion differs from biomechanical walking speed, visually directed locomotion is recalibrated in the real world. The effects are similar to those found using real-world visual motion and ACM Transactions on Applied Perception, Vol. V, No. N, Month 20YY.
Calibration of Locomotion due to Visual Motion in a Treadmill-based Virtual Environment
·
15
Percent Change (Post−Pre)
10 Overshoot relative to pre−test
5
0
−5 Undershoot relative to pre−test
−10
−15
Visually Slower
Visually Same
Visually Faster
Fig. 3. Recalibration of blind walking to targets for three visual conditions using the realistic hallway.
treadmill walking. These results are consistent with the explanation that subjects spatially update their representation of the environment as a function of the relearned relationship between visual motion and locomotor activity. They differ from Withagen and Michaels [2002] in that we found an undershoot for the faster biomechanical walking speed condition. In our results, there was a larger effect for the visually slower condition compared to the visually faster condition. The asymmetry is consistent with the small but significant overshoot found in the visually-same condition. Rieser et al. [1995] also found an asymmetry between visually slower and visually faster in their experiment using a tractor-pulled treadmill. They suggested that this asymmetry might be due to an elevated eye-height. However, since we found a similar effect without a change in eye-height, the present study could contribute to alternative explanations. It is possible that subjects perceptually compressed distances while viewing the endless hallway in the VE. A systematic underestimation in distance judgments has been found in numerous VE studies using head-mounted displays [Loomis and Knapp 2003; Thompson et al. 2004]. Although we have not directly tested distance estimations in the treadmill VE, a compression of distance would be predicted to lead to an overshoot in blind walking, similar to the visually slower condition. A second possible explanation that would account for both Rieser et al.’s work and our own is the distinction between walking on a treadmill belt and walking on the real ground. On the treadmill, while the subject has information from their motor and visual systems that they are moving, they have conflicting vestibular and cognitive cues that they are staying in the same place [Pelah et al. 1997]. If subjects were to calibrate to the perception of reduced self-motion, then we would also expect to find error in blind walking in the direction of the visually slower condition. Durgin, Pelah et al. [2005] supports this explanation. They found that after subjects walk without vision on a stationary treadmill, they overshoot in a ACM Transactions on Applied Perception, Vol. V, No. N, Month 20YY.
7
8
·
Betty J. Mohler et al.
blind-walking task. However, because Rieser et al. [1995] used a moving treadmill and we used a stationary treadmill, Durgin, Pelah et al. [2005] is not conclusive on this topic. 4.
PERCEPTION OF SELF-MOTION OR MAGNITUDE OF OPTIC FLOW?
The role of “visual flow” in the control of locomotion is widely recognized (e.g., [Gibson 1950; 1958; Warren 1995]). Often, it is theorized that control of heading and speed is directly influenced by optic flow, which is the two-dimensional vector field indicating the angular velocity of environmental points in the optic array. The optic flow field is sufficient to indicate the direction of motion, at least for environments free of large, independently moving objects, and it is sufficient to indicate the time-to-collision [Lee 1976]. However, optic flow on its own does not provide enough information for the recovery of absolute speed or distance. There is extensive research investigating perception and the control of self-motion (see Warren and Wertheim [1990] for a representative overview). Unfortunately, when considering visual perception, many authors use the term visual flow, without clarifying whether they mean 2–D optic flow or more general visually determined information about the actual 3-D movement through the world. Recently, questions have been raised about whether the control of locomotion involves aspects of accurately scaled three-dimensional space perception that go beyond simple characterizations of patterns of optic flow [Loomis and Beall 1998; Durgin and Pelah 1999; Kelly et al. 2005; Frenz and Lappe 2005; Loomis et al. in press]. Most relevant to the results reported here, Kelly et al. [2005] found that perceived self-motion and not just optic flow is used by the postural control system. Also, Loomis et al. [in press] found that complex actions can be carried out without retinal optic flow. For a review on perception of self-motion based on retinal/extraretinal optic flow, see Lappe et al. [1999]. Durgin and Pelah [1999] investigated visuo-locomotor recalibration without vision and their results also imply that the perception of self-motion may influence the recalibration effect. A person’s perceived self-motion is dependent on both the pattern and magnitude of optic flow and the perceived distances between the observer and environmental locations generating visual flow. Our previously described experiment left open the question of whether the effect was caused by a change in the magnitude of the optic flow alone or also by a change in the perceived self-motion. To explore this issue, we modified the first experiment so that the visual information presented to subjects in different conditions specified different velocities of self-motion through the world, while maintaining similar magnitudes and patterns of optic flow. We used computer graphics to present a stylized hallway in the Treadport. The hallway consisted of textured walls and textureless floor and ceiling, so that visual flow information was only available from the walls. In each condition, subjects saw a rendering of a relatively small or large hallway, moving at a visual speed less or greater, respectively, than their biomechanical rate of walking. Because space was scaled by the same amount as the change in the velocity through the space, the magnitude of optic flow remained essentially constant. Changes in pre- to post-test perception-action coupling would therefore be evidence that changes in optic flow are not necessary for recalibration effects to occur and argue for the importance ACM Transactions on Applied Perception, Vol. V, No. N, Month 20YY.
Calibration of Locomotion due to Visual Motion in a Treadmill-based Virtual Environment
visually/perceptually slower visually/perceptually same visually/perceptually faster
Experiment 1 Realistic hallway 11.0% 3% -6%
·
Experiment 2: Perception of self-motion 10.1% (smaller hallway) — -3.2% (bigger hallway)
Table I. Experimental overview and percent change in blind-walking to targets due to recalibration. The percentages are the percent change from a pre-test to a post-test of blind-walking to targets where a positive number indicates overshooting and a negative number indicates undershooting after recalibration.
of visual cues beyond the magnitude and distribution pattern of optic flow. An overview of our two experiments and the results can be found in Table I. 4.1
Method
16 subjects (8 male and 8 female) were screened for normal or corrected-to-normal vision and given real-world practice walking without vision. Viewing was binocular for all conditions as in the first experiment. Pre- and post- test were the same as in the first experiment, subjects walked blindfolded to previously viewed targets at 6, 8, 10m before and after 10 minutes of walking on the treadmill. A visual attention task was used to keep subjects from focusing on the middle of the center screen, for the reasons described in Experiment 1. The visual attention task for this experiment was to call out a number that appeared on small rectangular posters when in the center of the left and right screen. These posters were randomly ordered in colors of red, green, blue and had random numbers placed on them from 1 to 6. The series of posters were the same for each subject. In the adaptation phase, subjects were exposed to one of two visual conditions, corresponding to two different sizes of hallways and two different speeds of movement down the hallways. Biomechanical walking speed was 1.2 m/s in both cases. Figures 4–9 show views of the two hallways and their associated optic flow fields. Gender was balanced in the two conditions. 4.2
Visual Stimulus
For a viewer translating forward over a level ground plane, the magnitude of optic flow associated with the floor is a function of speed, eye height, and position in the field of view, as specified in the following relationship [Warren 1990]: v (1) β˙ = cos α sin2 β h where v is speed, h is eye height, α specifies the meridian angle of the line of sight (0 is down), β specifies the eccentricity angle of the line of sight (0 corresponds to the direction of motion), and β˙ is the flow magnitude expressed as a rate of change of angular position. As is clear from Equation 1, the ratio between speed and eye height provides a uniform scaling of all flow values. The optic flow associated with movement parallel to a vertical hallway wall is specified in a similar manner: v β˙ = cos γ sin2 β d
(2)
ACM Transactions on Applied Perception, Vol. V, No. N, Month 20YY.
9
10
·
Betty J. Mohler et al.
Fig. 4. View of small hallway, with distance cues sufficient to indicate a speed of self-motion slower than biomechanical walking speed. The images displayed on all three Treadport screens are shown. One of the visual attention posters is apparent on the right screen.
Fig. 5.
Fig. 6.
Optic flow field associated with Figure 4.
Optic flow field associated with Figure 4 (center screen).
ACM Transactions on Applied Perception, Vol. V, No. N, Month 20YY.
Calibration of Locomotion due to Visual Motion in a Treadmill-based Virtual Environment
·
Fig. 7. View of large hallway, with distance cues sufficient to indicate a speed of self-motion faster than biomechanical walking speed. The images displayed on all three Treadport screens are shown. One of the visual attention posters is apparent on the left screen.
Fig. 8.
Fig. 9.
Optic flow field associated with Figure 7.
Optic flow field associated with Figure 7 (center screen). ACM Transactions on Applied Perception, Vol. V, No. N, Month 20YY.
11
12
·
Betty J. Mohler et al.
where d is the perpendicular distance to the wall, γ is the meridian angle of the line of sight (0 is towards the closest point on the wall), and v, β, and β˙ are as before. For the case of movement along walls, ratio between speed and distance to the wall provides a uniform scaling of all flow values. Equations 1 and 2 demonstrate that speed of self motion cannot be determined from optic flow alone, without additional information about distance to visible surfaces. For this experiment, the distance to the wall and distance to the floor are the relevant distances. We can take advantage of this property to create a display in which the magnitude of optic flow and the visually indicated speed of self motion are independently manipulated. This is done by introducing static visual cues for depth, thus allowing the determination of v given β˙ for one or more lines of sight to surface points with visible flow. Since there are non-visual sources of information for eye height [Wraga 1999], we eliminated texture on the hallway floors that would generate visible flow. In our displays, the only textured surfaces generating visible flow were the hallway walls. The desired indicated speed of self motion and scaling of flow magnitude was achieved by appropriate variations of the visually indicated distance to the walls. Providing visual information for environmental scale in a virtual environment through which it is possible to move for many meters is a difficult task. Binocular convergence is of limited utility over the distances involved, and in any event stereo displays were not available to us. We created our displays to highlight linear perspective and a ground plane, which when combined with eye height, is sufficient to provide information about absolute distance [Sedgwick 1986]. To increase the sense of standing on a ground plane without generating a visible optic flow pattern on the ground, we slightly darkened the corners between the walls and floor but otherwise left the floor untextured. When viewed on the printed page, Figures 4 and 7 appear to show tilted corridors, with Figure 4 being the more tilted of the two. This is an artifact of the perception of pictorial space [Rogers 1995]. In the near visually immersive Treadport, subjects have a subjective sense of a level floor in the appropriate location relative to their body. To better standardize the displays across subjects, hallway dimensions were normalized for eye height. For a person with an eye height of 1.7m, the smaller hallway was rendered with a width of 2.48m and a height of 2.27m, and moved at a speed of 0.80m/s. This corresponds to a visual speed 0.67× the walking speed. 2 The larger hallway was rendered with a width of 7.44m and a height of 6.27m, and moved at a speed of 2.40m/s, corresponds to a visual speed 2.0× the walking speed. Because the scale of the space was changed by the same amount as the change in velocity through the space, the optic flow for lines of sight parallel to the floor was identical in both cases. In principle, the average flow over the walls was also the same, though flow due to the walls moved upward/downward in the visual field for the larger/smaller hallway. However, the fixed height of the display screens (8’, 2.44m) had the effect of reducing the screen area exhibiting visible flow in the large hallway relative to the small hallway. Table II shows the percentage of the display screens corresponding to the textured hallway walls for the narrow hallway 2 The variability in how much of the wall area was visible on the projection screens (Table II) kept us from duplicating the 0.5× mismatch of the first experiment.
ACM Transactions on Applied Perception, Vol. V, No. N, Month 20YY.
Calibration of Locomotion due to Visual Motion in a Treadmill-based Virtual Environment
center screen side screens all screens
coverage (narrow hallway) 46.2% 98.9% 81.3%
coverage (wide hallway) 36.0% 64.6% 55.1%
·
relative change in flow magnitude (narrow/wide) 1.27 0.99 1.03
Table II. Percentage of screen area covered by the image of the hallway walls for the wide and narrow hallways for an eye height of 1.7m, together with the relative change in the average magnitude of optic flow over the hallway walls for the narrow hallway compared to the wide hallway. Note that the narrow hallway corresponds to the visually slower condition while the wide hallway corresponds to the visually faster condition.
15
10 Percent Error (Post−Pre)
5
0
−5
Slower
Faster Perceived Self−Motion (PSM)
Fig. 10. Recalibration of blind walking to targets for two visual conditions involving different visual cues for the speed of self-motion but similar magnitudes of optic flow.
and wide hallway conditions. The asymmetric clipping of the flow associated with the hallway walls caused by the limited height of the display screens also affected the average flow magnitude present in the stimuli. The last column of Table II shows the average flow magnitude for the display of the narrow hallway divided by the average flow magnitude for the display of the wide hallway. As indicated in the table, on the center screen the average flow magnitude was greater for the narrow hallway (visually slower condition) than for the wide hallway (visually faster condition). The average flow magnitudes on the side screens were essentially the same for both conditions. 4.3
Results and Discussion
For the visually slower condition, subjects increased the distance they walked by an average of 10.1% between the pre- and post- tests. For the visually faster condition, subjects decreased the distance they walked by an average of 3.2% (see Figure 10). Paired t-tests confirmed a significant difference between the percent change between the pre- and post-tests for the two conditions(slower perceived ACM Transactions on Applied Perception, Vol. V, No. N, Month 20YY.
13
14
·
Betty J. Mohler et al.
self-motion: t(7) = 7.32, p < .01; faster perceived self-motion: t(7) = 5.04, p < .01). A univariate ANOVA comparing the effect of perceived self-motion on the post-pretest difference showed that there was a significant effect of perceived selfmotion, F (1, 15) = 54.8, p < .01. The overshoot in the slower perceived self-motion condition was reliably greater than the undershoot in the faster perceived selfmotion condition (mean difference = 13.3, SE = 1.48). An unobstructed view of the two hallways would have resulted in an average magnitude and distribution pattern of optic flow that were similar in both cases, though in the larger hallway (faster visually indicated self motion) the portions of the display with visible flow are shifted upward. The finite extent of the projection screens limited the portion of the hallway walls that could actually be rendered. One result of this was that the average magnitude of optic flow was greater for the the smaller hallway (slower visually indicated self motion) condition, since less of the wall having smaller magnitude flow was visible. The data indicates that blind walking performance changed in a manner consistent with slower perceived speed of self motion in the narrow hallway. If subjects were responding to the magnitude of optic flow rather than the perceived speed of self motion, we would expect the results to be reversed from what was actually observed. As a result, the second experiment provides evidence that a recalibration occurred based on the perceived speed of self motion during the adaptation phase of the experiment, rather than changes in the magnitude of optic flow during adaptation. The results from this experiment are qualitatively similar to those obtained in the first experiment. Visual information indicating a speed of movement slower than the biomechanical rate of walking produced overshoots in the post-test blind walking task compared with pre-test distances. Visual information indicating a speed of movement faster than the biomechanical rate of walking produced posttest undershoots compared with pre-test performance. For a variety of reasons, quantitative comparisons are harder to make. As previously indicated, technical reasons precluded matching the ratio of visual to biomechanical motion in the visually slower conditions. Comparing visual conditions is further complicated by the fact that the subjective sense of self-motion generated by the realistic hallway, with its familiar size cues and more complex geometry, was more compelling than that generated by the simple hallway. Finally, biomechanical walking speeds were slightly different in the two experiments, and the strength of the recalibration effect may be influenced by walking speed [Durgin, Pelah et al. 2005]. Whether or not recalibration is affected by changes in the magnitude of optic flow in the absence of changes in the perceived speed of self-motion remains an open question. This situation can arise in practice when moving at the same speed through physical environments of different scale. Unfortunately, a carefully controlled experiment to investigate this issue is difficult to construct, since it is not clear that it is possible to objectively measure and equate perception of self-motion to create the reverse condition. The fact that there was some recalibration effect in the visually same condition of experiment 1 supports this lack of control. If the proposed condition were to be run, and some amount of recalibration resulted, it would be impossible to determine whether this effect was a result of differential optic flow alone or whether perception of self-motion actually differed as well. Using ACM Transactions on Applied Perception, Vol. V, No. N, Month 20YY.
Calibration of Locomotion due to Visual Motion in a Treadmill-based Virtual Environment
·
VEs for such a study is particularly problematic, since there is evidence of systematic misperception of both speed of self-motion [Thurrell et al. 1998; Banton 2005; Durgin, Gigone et al. 2005] and absolute egocentric distance [Loomis and Knapp 2003; Thompson et al. 2004]. 5.
SUMMARY
To serve either as interfaces to large scale virtual worlds or as devices for better understanding human perception, treadmill VEs should provide users with an accurate and integrated visual and proprioceptive sensation of walking. We have demonstrated that it is possible to construct a treadmill VE in which manipulations of perception-action couplings in VE transfer to behaviors in the real world, providing evidence that perception-action couplings involving locomotion are similar in the two environments. We have also shown that perception-action calibration is affected by changes in cues associated with the perception of self-motion that do not involve corresponding changes in the magnitude of optic flow. Acknowledgments This material is based upon work supported by the National Science Foundation under grants 0080999 and 0121084. REFERENCES Banton, T., Stefanucci, J., Durgin, F. H., Fass, A., and Proffitt, D. 2005. Perception of walking speed in virtual environments. Presence: Teleoperators and Virtual Environments 14, 4. Durgin, F. H., Fox, L. F., Lewis, J., and Walley, K. A. 2002. Perceptuomotor adaptation: More than meets the eye. Paper presented at the 43rd annual meeting of the Psychonomic Society. Durgin, F. H., Gigone, K., and Scott, R. 2005. The perception of visual speed while moving. Journal of Experimental Psychology: Human Perception and Performance 31, 2. Durgin, F. H. and Pelah, A. 1999. Visuomotor adaptation without vision. Experimental Brain Research 127, 12–18. Durgin, F. H., Pelah, A., Fox, L. F., Lewis, J., Kane, R., and Walley, K. A. 2005. Self-motion perception during locomotor recalibration: More than meets the eye. Journal of Experimental Psychology: Human Perception and Performance 31, 3. Frenz, H. and Lappe, M. 2005. Absolute travel distance from optic flow. Vision Research. Gibson, J. J. 1950. The Perception of the Visual World. Riverside Press, Cambridge, MA. Gibson, J. J. 1958. Visually controlled locomotion and visual orientation in animals. British Journal of Psychology 49, 182–194. Hollerbach, J., Xu, Y., Christensen, R., and Jacobsen, S. 2000. Design specifications for the second generation Sarcos Treadport locomotion interface. In Haptics Symposium, Proc. ASME Dynamic Systems and Control Division. Vol. DSC-Vol. 69-2. Orlando, FL, 1293–1298. Kay, B. A. and Warren, Jr., W. H. 2001. Coupling of posture and gait: Mode locking and parametric excitation. Biological Cybernetics 85, 89–106. Kelly, J. W., Loomis, J. M., and Beall, A. C. 2005. The importance of perceived relative motion in the control of posture. Experimental Brain Research 161, 285–292. Lappe, M., Bremmer, F., and van den Berg, A. V. 1999. Perception of self-motion from visual flow. Trends in Cognitive Sciences 3, 9 (September). Lee, D. 1976. A theory of visual control of braking based on information about time-to-collision. Perception 5, 437–459. Loomis, J. M. and Beall, A. C. 1998. Visually controlled locomotion: Its dependence on optic flow, three-dimensional space perception, and cognition. Ecological Psychology 10, 3-4, 271–285. ACM Transactions on Applied Perception, Vol. V, No. N, Month 20YY.
15
16
·
Betty J. Mohler et al.
Loomis, J. M., Beall, A. C., Macuga, K. L., Kelly, J. W., and R., S. in press. Visual control of action without retinal optic flow. Psychological Science. Loomis, J. M., Da Silva, J. A., Fujita, N., and Fukusima, S. S. 1992. Visual space perception and visually directed action. Journal of Experimental Psychology: HPP 18, 906–921. Loomis, J. M. and Knapp, J. M. 2003. Visual perception of egocentric distance in real and virtual environments. In Virtual and Adaptive Environments, L. Hettinger and M. Haas, Eds. Erlbaum, Hillsdale, NJ, 21–46. Pelah, A. and Barlow, H. B. 1996. Visual illusion from running. Nature 381, 283. Pelah, A., Durgin, F. H., Miller, C. M., Washington, T. A., and Nelson, M. 1997. Adaptation to running depends on runner’s frame of reference. Investigative Ophthalmology & Visual Science 38, S1007. Pelah, A., Secker, B., Bishop, A., and Askham, C. 1998. A wide-field simulator for studying the visuo-motor interactions in locomotion. Journal of Physiology, P13–P14. Pelah, A., Thurrell, A. E. I., and Berry, M. 2002. Reduction of perceived visual speed during walking: Evidence against the involvement of attentional or vestibular mechanisms. Journal of vision 2, 7, 628a. Rieser, J. J., Ashmead, D. H., Talor, C. R., and Youngquist, G. A. 1990. Visual perception and the guidance of locomotion without vision to previously seen targets. Perception 19, 675– 689. Rieser, J. J., Pick, Jr., H. L., Ashmead, D., and Garing, A. 1995. Calibration of human locomotion and models of perceptual-motor organization. Journal of Experimental Psychology: HPP 21, 480–497. Rogers, S. 1995. Perceiving pictorial space. In Perception of Space and Motion, W. Epstein and S. Rogers, Eds. Handbook of Perception and Cognition, vol. 5. Academic Press, Chapter 4, 119–163. Sedgwick, H. A. 1986. Space perception. In Handbook of Perception and Performance, K. R. Boff, L. Kaufman, and J. P. Thomas, Eds. Wiley-Interscience, 21–1–21–57. Stappers, P. J. 1996. Matching of proprioceptive to visual speed affected by non-kinematic parameters. Perceptual & Motor Skills 83, 1353–1354. Thompson, W. B., Willemsen, P., Gooch, A., Creem-Regehr, S. H., Loomis, J., and Beall, A. 2004. Does the quality of the computer graphics matter when judging distances in visually immersive environments? Presence: Teleoperators and Virtual Environments 13, 5, 560–571. Thurrell, A. E. I. and Pelah, A. 2002. Reduction of perceived visual speed during walking: Effect dependent upon stimulus similarity to the visual consequences of locomotion. Journal of Vision 1, 3, 307a. Thurrell, A. E. I., Pelah, A., and Distler, H. 1998. The influence on non-visual signals of walking on the perceived speed of optic flow. Perception 27s, 147. Usoh, M., Arthur, K., Whitton, M. C., Bastos, R., Steed, A., Slater, M., and Brooks, Jr., F. P. 1999. Walking > walking-in-place > flying, in virtual environments. In ACM SIGGRAPH. 359–364. Warren, R. 1990. Preliminary questions for the study of egomotion. In Perception & Control of Self-motion, R. Warren and A. H. Wertheim, Eds. Lawrence Erlbaum Association, Hillsdale, NJ. Warren, R. and Wertheim, A. H. 1990. Perception & Control of Self-motion. Lawrence Erlbaum Association, Hillsdale, NJ. Warren, Jr., W. H. 1995. Self-motion: Visual perception and visual control. In Perception of Space and Motion, W. Epstein and S. Rogers, Eds. Academic Press, New York, 263–325. Warren, Jr., W. H., Kay, B. A., and Yilmaz, E. H. 1996. Visual control of posture during walking:functional specificity. Journal of Experimental Psychology: HPP 22, 818–838. Withagen, R. and Michaels, C. F. 2002. The calibration of walking transfers to crawling: Are action systems calibrated? Ecological Psychology 14, 223–234. Witmer, B. and Sadowski, Jr., W. J. 1998. Nonvisually guided locomotion to a previously viewed target in real and virtual environments. Human Factors 40, 478–488. ACM Transactions on Applied Perception, Vol. V, No. N, Month 20YY.
Calibration of Locomotion due to Visual Motion in a Treadmill-based Virtual Environment
·
Wraga, M. 1999. Using eye height in different postures to scale the heights of objects. Journal of Experimental Psychology: Human Perception and Performance 25, 2, 518–530.
ACM Transactions on Applied Perception, Vol. V, No. N, Month 20YY.
17
