Manual And Virtual Rotation Page 1 MANUAL

Manual And Virtual Rotation Page 1 MANUAL AND VIRTUAL ROTATION

Manual and Virtual Rotation of a Three-Dimensional Object Roy A. Ruddle Dylan M. Jones Cardiff University

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 2 Abstract

An orientation-matching task, based on a mental rotation paradigm, was used to investigate how participants manually rotated a Shepard-Metzler object in the real world and in an immersive virtual environment (VE). Participants performed manual rotation quicker and more efficiently than virtual rotation but the general pattern of results was similar for both. The rate of rotation increased with the starting angle between the stimuli meaning that, in common with many motor tasks, an amplitude-based relationship such as Fitts’ Law is applicable. When rotation was inefficient (i.e., not by the shortest path) it was often because participants incorrectly perceived the orientation of one of the objects, and this happened more in the VE than in the real world. Thus, VEs allow objects to be manipulated naturally to a limited extent, indicating the need for timing scale factors to be used for applications such as method-time-motion studies of manufacturing operations.

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 3 Manual and Virtual Rotation of a Three-Dimensional Object

A goal of many virtual environment (VE; virtual reality) applications is to allow users to interact “naturally” with 3D computer-generated objects. If natural interaction can be achieved then not only would the interface exploit well-practiced motor skills, but real-world operations such as manual handling tasks, that involve human operators, could be realistically simulated in real time. This holds manifold advantages in manufacturing design. For example, it would allow accurate method-time-motion studies to be performed using VEs to optimize processes within each cell of a manufacturing facility, the lifting requirements of different tasks to be assessed using actual human motion, “what if” scenarios to be easily investigated, and provide a new mechanism for training production operators. One component of interaction is the free-form (unconstrained) rotation of objects. This is an everyday task, but one that is poorly understood in the cognitive domain. The present article reports the findings of three experiments that studied aspects of free-form manual (real world) and virtual object rotation, and had three primary goals. First, to understand the effect of axes of rotation and angles of separation on the manner in which people perform manual rotation. Second, to determine the extent to which people can rotate objects naturally in a VE and, third, to investigate some of the factors that cause the differences between manual and virtual rotation. First, however, we consider existing studies of manual and virtual rotation, and investigations of a related task, mental rotation. Background The first study of mental rotation was published approximately 30 years ago (R. Shepard & J. Metzler, 1971) and the primary finding, the linear relationship between reaction time and angular displacement for a handedness recognition task, has proved to be very robust. Although there are circumstances under which mental rotation does not appear to take

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 4 place (Cohen & Kubovy, 1993), the original finding has been repeated for both 2D and 3D objects, drawings of objects and the objects themselves, binocular and monocular viewing conditions, stimuli of different complexities, and “possible” and “impossible” shapes (e.g., Bethell-Fox & R. Shepard, 1988; Dror, Ivey, & Rogus, 1997; Kaushall & Parsons, 1981; S. Shepard & D. Metzler, 1988). Wohlschläger and Wohlschläger (1998, see also Wexler, Kosslyn, & Berthoz, 1998) put forward a common-processing hypothesis which suggested there is a single process that controls the rotation of objects via motor commands and mental changes in the visuospatial representation of objects. This hypothesis states that mental and manual rotation should be affected in the same way by the same factors, and this type of relationship is found in some other types of motor task, for example, both imagined and actual pointing follow Fitts’ law (Maruff et al., 1999). In the context of the mental rotation paradigm, the common processing hypothesis predicts that the time taken to perform manual rotation will increase linearly with the angle between two objects’ orientations, and will be affected by the orientation of the objects with respect to three different sets of axes (Parsons, 1995; see Figure 1): the major limb of the objects (O), the principal axes of the person (viewer; V), and the axis that defines the shortest angular path of rotation (R; this is unique for any pair of orientations, unless they differ by 180°). Alignment of all three produces an arrangement known as the OVR condition, whereas the general condition occurs when none are aligned. In between are conditions where either the rotation and object, or rotation and viewer, axes are aligned. Parsons showed that people are progressively less accurate at imagining rotational transformations as the alignment is changed from the OVR to the general condition. Manual and virtual rotation involve the comparison of the orientations of different objects and the execution of motor tasks to change those orientations. The purpose of the

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 5 present study is to compare manual and virtual rotation, and to explain the differences that occur in terms of the differences that exist between real and virtual environments in the perception of objects’ orientations and the interface used to perform motor tasks. A variety of object manipulation interfaces have been developed for use in VEs (for a review, see Hand, 1997). Studies have shown that efficient (i.e., straight line) translations are straightforward to perform, although accuracy is more difficult to achieve in depth than in the viewing plane (Boritz & Booth, 1997). Rotational movements are performed less efficiently (Zhai & Milgram, 1998) and, therefore, are of particular interest to researchers. However, general 3D rotations are performed more quickly if all three degrees-of-freedom (DOFs) can be controlled simultaneously using a single input device, rather than separately (Chen, Mountford, & Sellen, 1988; Hinckley, Tullio, Pausch, Proffitt, & Kassell, 1997). The two most common interfaces that allow simultaneous control are props and virtual hands. A prop is a physical object that contains a six DOF sensor and which a user holds in their hands. Hence, some haptic feedback is provided. The sensor continually measures position and orientation of the prop and, typically, movements of the prop produce corresponding changes in the position and orientation of the virtual object that is being manipulated. With a virtual hands interface a six DOF sensor is attached to each of the user’s hands and the user places their hands around the virtual object, as if they were holding it. A disadvantage of this type of interface is that clutching (the process of swapping hands when turning an object through a large angle) has to be specified explicitly (e.g., by holding down a button) instead of being automatic. For this reason, props are most likely to facilitate natural interaction and have been shown experimentally to allow rotations to be performed 20% quicker than with virtual hands (Zhai, Milgram, & Buxton, 1996). Props vary in terms of their haptic correspondence with the object that is being manipulated. Rotation becomes quicker when the prop has the same size, shape and inertial

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 6 characteristics as the object (Wang & MacKenzie, 1999a; see also Ware & Rose, 1998), and slower when orientation disparity is deliberately introduced (Wang & MacKenzie, 1999b). However, in manufacturing design applications, where the virtual objects do not yet exist in a physical form, “realistic” props are not a viable option. A common, practical alternative is to use a sphere-shaped prop, and this also minimises the haptic conflicts that occur when many different objects are being manipulated in a VE. Another factor that affects interfaces for object rotation is the number of hands that are used. Small rotations (e.g., up to 45°) can be performed quickly using one hand (Wang & MacKenzie, 1999a; 1999b; 2000) but two hands are typically preferred by users (Mapes & Moshell, 1995) and aid clutching when large rotational movements are being performed. The orientation of a hand-held object can be perceived both haptically and visually. In contrast, only visual information can be used to perceive the orientation of a virtual object that is manipulated using an interface such as sphere-shaped prop. Lighting conditions and surface shading affect the way objects are perceived (e.g., Mamassian & Kersten, 1996; Tarr, Kersten, & Bülthoff, 1998) and, although studies of object recognition are often performed using computer-generated (i.e., virtual) images rather than physical objects, and Johnston and Curran (1996) found similar effects when they compared real and virtual (Phong-shaded) stimuli, the former contain more subtle variations in appearance which may aid orientation perception and the discrimination of the different parts of an object. To determine fully the extent to which a person can rotate an object naturally in a VE, measures must be made of the time taken, the path through which the object is moved, and the movements made by the person. The term “mental rotation” was coined because the linear relationship found by Shepard and Metzler (1971) was what would have occurred if participants had performed the task by rotating their mental image of the objects at a constant angular velocity, along a path that increased linearly in length with the angular separation

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 7 between the two stimuli. Rotation about the shortest path satisfies this linear relationship, but other geometrical bases of rotation, for example, spin-precession (simultaneous rotation about two different axes (e.g., axes in the object’s and viewer’s reference frame)), and rotation-by-dimensions (a sequence of rotation axes; Parsons, 1987), do not. To date, however, most studies of mental, manual and virtual rotation have only measured rotation time, and used the shortest path to infer the rotation rate. Exceptions are Wohlschläger and Wohlschläger (1998) who analyzed the threshold at which participants were as likely to turn the object in the opposite direction to the shortest path as follow the shortest path (the object was turned using a knob), and Zhai & Milgram (1998) who compared the cumulative angle participants turned a virtual object through with the minimum possible angle. Human body movements have not been studied in manual or virtual rotation tasks and, although shape and orientation affect the way objects are grasped (Mamassian, 1997) and biomechanical limitations affect the way in which objects are manipulated, these factors are outside the scope of the present study. We used the simultaneous presentation mental rotation paradigm to study how participants rotated a 3D object in the real world (Experiment 1) and in an immersive VE (Experiments 2 and 3). All the experiments used three different initial angular separations between pairs of objects (90º (practice trials), and 45º and 135º (test trials)) and two combinations of axis alignment (OVR and general; see Figure 1). In all of the trials the stimuli had the same handedness. The studies were designed to investigate the paths through which participants rotated manual and virtual objects, as well as how long they took. Experiment 1 Method Participants. Six men and six women, aged 19 to 24 years, took part in the experiment. All the participants were either undergraduates or graduates, who volunteered for Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 8 the experiment and were paid an honorarium for their participation. A repeated measures, within-participants design was used (see below). Materials. Two wooden models of the Shepard-Metzler object used by Parsons (1995) were constructed. The wood had a 50 x 50 mm cross section, so the longest side of each object was 250 mm. Each object’s orientation was measured by a Polhemus Fastrak magnetic sensor that transmitted data via the MR Toolkit (Green, 1995). Six orientation pairs for each angle/axis were pre-computed using random rotations that satisfied the OVR and general alignment conditions, as appropriate. For the OVR condition the principal axis of the object coincided with the axis of the shortest path of rotation, and there were two pairs of orientations in which these axes were aligned with each of the viewer’s axes (X, Y and Z; see Figure 1). For the general condition the shortest rotation axis was always in a direction that bisected the principal axes of the viewer (e.g., {x, y, z} = {0.58, 0.58, 0.58} or {-0.58, 0.58, 0.58}) and in one orientation of each pair the principal axis of the object bisected the principal axes of the viewer in another direction (e.g., {x, y, z} = {-0.58, 0.58, 0.58} or {0.58, 0.58, -0.58}). For both the OVR and the general condition the rotation of the object about its principal axis was calculated randomly. Procedure. Participants were run individually and performed the trials in a standing position because this allowed greater freedom of movement than when sitting down. One object (the target object) was mounted on a tripod 0.8 m in front of a participant and 0.8 m below their eye level, and they held the other (mobile) object in their hands. After familiarizing themselves with the shape of the object, each participant performed 12 practice trials, had a short rest, and then performed 24 test trials. At the start of each trial the participant was blindfolded. The experimenter then used the VE software of Experiment 2 to set the target object at a pre-computed orientation. Next, the experimenter placed the mobile object in the participant’s hands and used the VE

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 9 software to set the object at another pre-computed orientation. Then the experimenter pressed a key to blank the VE display and begin recording orientation data for the mobile object, and removed the participant’s blindfold. They rotated the mobile object “as fast and as accurately as possible” until they judged that it matched the orientation of the target, and said “OK”. The experimenter pressed a key that caused the data recording to be stopped and the angular difference between the two finishing world-referenced orientations (the error) to be displayed as a text message on a computer monitor for the participant to see. A world (scene-based) reference frame was used because that has been shown to be dominant when participants make orientation-matching judgements (Hinton & Parsons, 1988). The purpose of this message was to help ensure each participant placed a similar emphasis on finishing accuracy in the trials. Error feedback proved to be a more practical way of achieving this than terminating each trial automatically once participants had held the mobile object within a certain angular tolerance of the target’s orientation for a given amount of time. Every participant used the same pre-computed orientation pairs but the order of presentation was random, as was the orientation within each pair that was chosen for the target and mobile objects. Results Preliminary Data Analysis. The orientation of the mobile object was recorded at a rate of 12.5 Hz. Inspection of these data showed that participants quickly rotated the object until it was in approximately the correct orientation (the rotation stage) and then sometimes spent a considerable time trying to get the orientation exactly right (fine tuning). In fact the task was “impossible” because two identical objects can only have an identical appearance if they overlap with one another in space. Our interest lay in the large manipulations that were made so the data for the rotation stage of each trial were extracted using the following two techniques that identified the beginning and end of the rotation stage, respectively. The Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 10 techniques were implemented in software but, for clarity, are described as if they were performed by hand. Both started by plotting a curve that showed how the angle between the mobile and target objects varied during a trial. For each data point, this angle was the absolute value of the angular separation between the objects, calculated for rotation about the shortest angular path between the objects at that moment in time. The data at the start of each trial contained a period of time when participants were still blindfolded. The beginning of the rotation stage was identified by fitting a cubic Bezier spline to the curve and calculating the time at which the gradient of the spline first exceeded 10 degrees/sec. The end of the rotation stage was defined as the time at which the mobile object first came within 20º of the target and then remained within 20º for the remainder of the trial. Clearly, the two parameters that were used (10 deg/sec and 20º amplitude) are subjective but graphical inspection of the data for every trial indicated that they were appropriate. A trial was discarded if the spline indicated that the rotation stage started at the beginning of the trial (an impossibility because the participant was still wearing the blindfold), the orientation of mobile object at the beginning of the rotation stage differed by more than 10º from it’s “official” starting orientation, or the mobile object never satisfied the 20º criterion. Overall, 9% of the trials were discarded. The amplitude criterion meant that it was possible for a participant to complete each trial by rotating the mobile object through much less than the nominal angle (45º or 135º). However, inspection of the data showed that the mean angular difference between mobile object’s orientation at the start and end of the rotation stage of the non-discarded trials was only 6º less than the nominal angles. The data reported below are for the rotation stage of the non-discarded trials. All the data were analyzed using analyses of variance (ANOVAs) that treated the angle (45º or 135º)

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 11 and axis alignment (OVR or general) as repeated measures. Only significant interactions are reported. Mean data for these analyses are shown in Table 1, along with the mean time taken in the fine tuning stage. Overall, the mean time for the initiation stage was 1.7 s. Rotation time. The distribution of participants’ time data was normalized using a logarithmic transformation. A repeated measures ANOVA showed that, as expected, participants took less time to complete the rotation stage in the 45º trials than in the 135º trials, F(1, 11) = 63.78, MSE = 0.02, p < .01, and they also took less time to complete the OVR trials than the general trials, F(1, 11) = 28.45, MSE = 0.02, p < .01. Mean rotation rate. The distribution of these data was normalized using a logarithmic transformation. A repeated measures ANOVA showed that participants mean rate of rotation was slower in the 45º trials than in the 135º trials, F(1, 11) = 38.35, MSE = 0.01, p < .01, but there was no effect of axis combination, F(1, 11) = 3.00, MSE < 0.01, p = .11. This indicates that the principal reason participants took longer to complete the general trials than the OVR trials was because they rotated the mobile object further, rather than more slowly. Rotation efficiency. One of the primary goals of this study was to provide information about how participants rotated the mobile object, as well as the length of time they took. Two measures of participants’ rotation efficiency were used: the percentage extra angle of rotation (PE-angle), and the object’s root mean square (RMS) angular deviation from the shortest path. The PE-angle metric was suggested by Zhai & Milgram (1998) and a similar technique has been used to measure wayfinding efficiency in navigation studies (e.g., Ruddle, Payne, & Jones, 1999). It is calculated as PE-angle = 100 * (Ac - Am) / Am, where Ac is the cumulative angle of rotation during the rotation stage of a trial and Am is the angle between the starting and finishing orientations of the mobile object. The RMS deviation metric is calculated as RMS = √(∑(ai * ai) / n), where i is in the range 1 to n, n is the number of time steps in the rotation stage, and ai is the angular separation of the mobile object’s orientation from the

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 12 shortest path of rotation between the mobile object’s orientation at the start of the rotation stage and the target’s orientation (calculated using an iterative algorithm). Neither the PEangle nor the RMS deviation metric is suitable on its own because a trial with a large PEangle might not deviate far from the shortest path, and a trial with a low RMS error could still contain a lot of back and forth movement. The distributions of participants’ PE-angle and RMS deviation data were both normalized using a logarithmic transformation. For the former, a repeated measures ANOVA showed no effect of angle, F(1, 11) = 2.19, MSE = 0.08, p = .17, but there was an effect of axis combination, F(1, 11) = 36.47, MSE = 0.09, p < .01. For the RMS deviation, a repeated measures ANOVA showed main effects of angle, F(1, 11) = 79.43, MSE = 0.02, p < .01, and axis combination, F(1, 11) = 51.07, MSE = 0.03, p < .01. Taken together, these data show that the OVR trials were very efficient. Participants rotated the mobile object approximately 25% further than they had to and deviated by only a small amount from the shortest path. By contrast, the 135º-general condition was particularly inefficient. To provide an explanation each trial was classified according to whether the mobile object was initially rotated towards or away from the target, with the latter referred to as incorrect direction of rotation (IDOR). The amount of rotation required before this classification was performed only had a small effect on the percentage of trials in each category and didn’t change the overall pattern of the data. Using a 20º rotation criterion, a repeated measures ANOVA showed main effects of angle, F(1, 11) = 8.63, MSE = 402.46, p = .01, and axis combination, F(1, 11) = 6.43, MSE = 335.41, p = .03, and there was also a significant interaction, F(1, 11) = 8.11, MSE = 407.63, p = .02. IDOR trials occurred more than three times more frequently in the 135º-general condition than in any other condition and manual inspection showed that, after the initial incorrect rotation, participants quite often paused with the short and long stubs of the object swapped over (180º error) before correcting

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 13 themselves. Referring back to Table 1, the frequency of the IDOR trials and the 180º errors explain why the RMS deviation was so large in the 135º-general condition. Discussion The results of this experiment are interpreted in the context of the common processing hypothesis (Wohlschläger & Wohlschläger, 1998). The time and rotation rate data show that participants took longer to complete the trials when the axes were not aligned (the general condition) and the rate of rotation increased with the angle between the stimuli. The first finding is in agreement with mental rotation studies. The second finding is in agreement to studies of simple motor tasks but contrary to mental rotation. Thus manual and mental rotation may share some common processes but there is also at least one notable difference. The manner in which participants rotated the mobile object is indicated by the efficiency data. These show that participants used a near-shortest path when the experimental design was “friendly” (the OVR condition) but a very inefficient path in the 135º-general condition. With all three types of rotation strategy that were suggested by Parsons (1987) (the optimum axis (shortest path), spin-precession (simultaneous rotation about two different axes (e.g., axes in the object’s and viewer’s reference frame), and rotation-by-dimensions (a sequence of rotation axes)) the angle between the mobile and target object would have decreased throughout the trial. The present experiment is not able to test for particular rotation strategies but the data do indicate that the angle between the objects sometimes initially increased (the IDOR trials) and often oscillated. This suggests that the primary causes of rotation inefficiency were errors in participants’ perceptions of the objects orientations or execution of the rotational responses, rather than following a deliberate, but non-optimal, path.

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 14 Finally, the metrics used to analyze the data from Experiment 1 provided stable measures of rotation performance. In turn, this establishes a baseline for manual rotation of one particular object that can be used to compare rotation tasks that are performed in VEs. Experiment 2 Experiment 2 recreated the first experiment in an immersive VE (a VE shown using a head-mounted display (HMD)). As well as viewing a virtual, rather than physical, object two additional differences between the two experiments should be noted. First, the interface used in Experiment 2 was a spherical prop. A two-handed, prop-type interface was chosen because it allows more rapid rotation than virtual hands, and the process of clutching becomes automatic. It would have been possible to use a prop that was the same size and shape as the Shepard-Metzler object but in most VE applications, as has already been noted, users manipulate many different objects. Therefore, a spherical shape is a more practical choice because it has neutral orientation disparity. Second, the virtual objects were scaled by a factor of 0.3 compared with the physical objects so that the target and mobile objects could be seen simultaneously in the HMD, but it should be emphasised that participants never saw the physical objects. Method Participants. Six men and six women, aged 19 to 37 years, took part in the experiment. All the participants were either undergraduates or graduates, who volunteered for the experiment and were paid an honorarium for their participation. None had participated in Experiment 1. Materials. The VE software was designed and programmed by the authors, and ran on a Silicon Graphics Maximum IMPACT workstation. The HMD was a Virtual Research VR4 (247 x 230 pixel resolution; 48º x 36º field of view) and head-tracking was performed using a Fastrak sensor. Images were displayed in stereo in the HMD and the inter-pupilary distance Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 15 was adjusted for each participant. Participants rotated the mobile object by rotating a hockey ball prop (a 72 mm diameter sphere that contained a Fastrak sensor) and there was 1:1 correspondence between rotations of the object and prop. The design of the application meant that the maximum lag in the system was the same as the graphics frame rate (80 ms). At the fastest of the mean rotation rates in Experiment 1 (135-OVR), this corresponds to an angular lag of 7°. The virtual objects were displayed one above the other in the HMD at a center-tocenter spacing of 0.11 m, corresponding to a visual angle of 18°. A pilot investigation showed that this was preferable to displaying them side by side. The mid-point between the upper (target) and lower (mobile) object was horizontally in front of the participant’s eyes. This meant the HMD was balanced and minimized neck strain. Procedure. People are used to manipulating objects in the real world but none of the participants had previously used a prop interface for a VE, so the experiment was divided into two phases. In the first phase a participant was trained to rotate virtual objects using the prop. In Phase 2 the participant performed the orientation-matching task of Experiment 1. A participant started Phase 1 by rotating a virtual object for a few minutes to familiarize themselves with using the prop and the HMD. Then they did thirty trials in which they had to rotate the prop so a mobile virtual object continuously matched the orientation of a spinning target. The trials were split into five blocks, and within each there were three different shapes of object (an aircraft, a cow and a canon), three rates of spin (30, 60 and 90 degrees/sec) and spinning axes that were either constant or changed continuously. The axes always lay in the participant’s XY, XZ, or YZ plane (the orientation of participants’ (viewer’s) axes are shown in Figure 1), which meant they were different to the shortest path axes used in the orientation-matching task. In each trial the target object was stationary for 3 s and then spun for 30 s. As a whole, this phase trained the participant to use the prop,

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 16 without familiarizing them with the Shepard-Metzler object used in Phase 2, or specifically training them to perform rotations using either a constant axis or a constantly-varying axis. The virtual objects used for the orientation-matching task (Phase 2) were the same shape as the physical objects used in Experiment 1, but were scaled by a factor of 0.3 (see above). The objects were texture mapped with the pattern of wood. The task used the same practice (90º) and test (45º and 135º) orientation pairs as Experiment 1 and, as before, they were presented in a random order. At the start of each trial a sphere was displayed in the position where the objects would appear. When the participant was looking at the sphere the experimenter pressed a key, which caused the sphere to be replaced by the two objects in one set of the pre-computed orientation pairs. The participant rotated the mobile object “as fast and as accurately as possible” until they judged that it matched the orientation of the target, and said “OK”. Then the experimenter pressed a key that terminated the trial and caused the angular error message to be displayed in the HMD. As precautionary measure, symptoms of VE sickness were monitored for 1 hr at the end of the experiment, using the Short Symptom Checklist developed by Cobb, Nichols, Ramsey, and Wilson (1999). These data are not reported here. Results For consistency, the rotation stage was extracted from the data for each trial using the same two techniques that were used in Experiment 1. The rotation stage data were then analyzed using the same type of data transformation (logarithmic) and repeated measures ANOVAs as were used in Experiment 1. Mean data for all of these analyses except the interface training are shown in Table 2, along with the mean time taken in the fine tuning stage. Interface training. Participants’ mean orientation error during the interface training was analyzed using an ANOVA that treated the trial block and rotation rate as repeated Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 17 measures. This ANOVA showed main effects of trial block, F(4, 11) = 8.78, MSE = 329.22, p < .01, and rotation rate, F(2, 11) = 63.55, MSE = 290.14, p < .01 (see Figure 2). The magnitude of the mean errors is an indication of the difficulty of the task, particularly with rotation at 90 degrees/sec. The important things to note are the improvement that took place in participants’ performance, and the plateauing-out of the errors in the final two blocks. Rotation time and rate. A repeated measures ANOVA showed that participants took less time to complete the rotation stage in the 45º trials than in the 135º trials, F(1, 11) = 59.55, MSE = 0.03, p < .01, and there was a reduced effect of axis combination, F(1, 11) = 4.27, MSE = 0.03, p = .06. Analysis of the mean rotation rate data showed that participants rotated the object slower in the 45º trials than in the 135º trials, F(1, 11) = 130.17, MSE < 0.01, p < .01, but there was no effect of axis combination, F(1, 11) = 1.18, MSE < 0.01, p = .30. Rotation efficiency. As in Experiment 1, the efficiency of each trial was analyzed using the PE-angle, RMS deviation, and IDOR data. For the PE-angle data, a repeated measures ANOVA showed reduced effects of angle, F(1, 11) = 4.20, MSE = 0.05, p = .07, and axis combination, F(1, 11) = 3.52, MSE = 0.09, p = .09. A repeated measures ANOVA of the RMS deviation data showed main effects of angle, F(1, 11) = 50.92, MSE = 0.03, p < .01, and axis combination, F(1, 11) = 14.93, MSE = 0.02, p < .01, and a significant interaction, F(1, 11) = 11.35, MSE = 0.01, p < .01. For the IDOR data, a repeated measures ANOVA showed a main effect of angle, F(1, 11) = 13.65, MSE = 349.30, p < .01, but not of axis combination, F(1, 11) = 0.17, MSE = 407.00, p = .68. However, there was a significant interaction, F(1, 11) = 9.18, MSE = 249.55, p = .01. Comparison with Experiment 1. The general pattern of results was similar in the two experiments, but that for some of the data there was also a considerable difference in performance. To investigate real-world vs. VE differences the data were reanalyzed using

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 18 ANOVAs that treated the angle and axis combination as repeated measures (as before) and the experiment as a between-participants factor. Participants completed the rotation stage significantly faster in the real world than in the VE, F(1, 22) = 18.91, MSE = 0.07, p < .01, and the rotation rates were slightly greater in the real world, F(1, 22) = 3.02, MSE = 0.04, p = .10. Participants’ PE-angle was significantly lower in the real world than in the VE, F(1, 22) = 9.14, MSE = 0.35, p < .01, and there was a significant interaction between experiment and axis combination, F(1, 22) = 9.23, MSE = 0.09, p < .01. There were similar effects for the RMS data. Participants’ RMS deviation was significantly lower in the real world experiment than in the VE experiment, F(1, 22) = 13.00, MSE = 0.07, p < .01, and there was a significant interaction between experiment and axis combination, F(1, 22) = 7.75, MSE = 0.03, p < .01. There was no difference between the two experiments for the IDOR data, F(1, 22) = 1.35, MSE = 790.21, p = .26. Discussion Although the general pattern of results in the two experiments was similar, participants’ rotations took 80% more time in the VE experiment than in the real world experiment. The difference was because virtual rotation was less efficient than manual rotation (participants rotated the object through a greater angle in the former), rather than a significant change in the rate of rotation, and the change in efficiency was particularly notable in the OVR condition. All the participants performed clutching while rotating the prop. Factors that are likely to have contributed to the reduced efficiency of virtual rotation can be divided into those that affected participants’ ability to: (a) perceive the objects’ orientation, or (b) efficiently perform the motor task of rotation. Within the perception category there are four factors. First, the objects’ position (horizontally in front of participants’ eyes) may have made it more difficult for participants to discriminate between Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 19 the short and long stubs of the objects and, therefore, to determine the objects’ orientations. However, an earlier VE study found similar rotation performance when objects were positioned in front of participants’ eyes or at waist level (Ruddle, Huddart, & Jones, 1999). Second, in the real world experiment participants were instructed to hold the mobile object with one hand on each stub. This meant that participants could identify the two stubs and, hence, obtain information about the object’s orientation both visually and haptically. In the VE experiment, participants were only provided with the visual information. Third, informal observation suggests that participants used different strategies to match the orientations in the two experiments. In the VE experiment participants often matched the appearances of particular surfaces on the objects, using the VE’s lighting conditions, whereas in the real world experiment participants matched the overall orientation of the objects’ three parts. Similarities can be drawn between this and mental rotation, where the effects of stimulus complexity disappeared with continued practice, suggesting that participants rotated an object piece by piece when it was unfamiliar but as a whole once it was well learned (Bethell-Fox & Shepard, 1988). However, it should be emphasized that participants peformed equal amounts of practice in Experiments 1 and 2 of the present study. Fourth, in Experiment 2 participants may have deliberately rotated the mobile object to help determine its orientation. In other words, participants may have made pragmatic actions (Kirsh & Maglio, 1994) to help overcome their difficulty in perceiving the mobile object’s orientation. In the motor task category it was noticeable that there were occasions during the spinning object training when participants had difficulty rotating the mobile object in the direction they intended. Although participants improved significantly during the training there are likely to be some inherent difficulties rotating virtual objects. One is the conflict between the shape and inertia of the prop and the virtual object. Although this could be overcome a key objective of many VE applications, particularly those concerned with

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 20 manufacturing design, is to allow people to interact with virtual objects that do not exist physically. A second is that in the real world experiment participants were instructed to hold the object with one hand on each stub, so the axis that defined the shortest path of rotation in the OVR trials was also the axis that connected participants’ hands. In the VE experiment participants tended to rotate the prop by moving their fingers. The first two experiments demonstrated that people can perform virtual rotation in a similar manner to manual rotation, but questions remain as to the cause of the performance decrease that was observed. In Experiment 3 participants performed the orientation-matching task under three conditions that differed either in the difficulty with which the objects’ orientations could be perceived, or the amount of time for which the objects were visible prior to rotation commencing. Experiment 3 The three perception conditions used were: (a) immediate response using the woodtextured object, (b) immediate response using the three-colored object, and (c) delayed response using the wood-textured object. These are referred to as the textured, colored and observe conditions, respectively. The textured condition was the same as in Experiment 2. In the colored condition the short and long stubs of the objects were different colors, allowing the relative orientation of the mobile and target objects to be easily discriminated, and meaning that orientation matching was predicted to take place more quickly than in the textured condition. In the observe condition, participants had 10 s in which to perceive the objects’ orientation and plan their rotation response. This was also predicted to lead to quicker orientation matching than in the textured condition, but no prediction was made between the colored and observe conditions.

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 21 Method Participants. Three men and nine women, aged 19 to 26 years, took part in the experiment. All the participants were either undergraduates or graduates, who volunteered for the experiment and were paid an honorarium for their participation. None had participated in the other experiments. Materials. The experiment used the same hardware, software, and prop interface as Experiment 2. A second version of the Shepard-Metzler object was created. This was the same size and shape as the wood-textured version used in Experiment 2 but, instead of being textured, the three limbs of the object were each a different color (magenta, cyan or yellow). Procedure. The experiment was divided into two phases. Phase 1 was identical to Experiment 2. A participant was trained to use the prop interface, using the spinning object trials. In Phase 2 the participant performed the orientation-matching task under the three perception conditions. The textured condition was similar to Experiment 2 but participants were instructed to respond immediately the objects became visible. The colored condition was identical to the textured condition except that the three-colored version was used for the target and mobile objects. In the observe condition participants looked at the objects for 10 s and then performed the rotation. During the observation period the prop was de-coupled from the mobile object so participants could not change its orientation. The end of the 10 s observation period was indicated by a short tone. The order in which participants performed the three conditions was balanced using a Latin square design. For each condition they performed practice trials (90º angular difference) and then test trials (45º and 135º angular differences). Each set of trials used a different set of pre-computed orientation pairs. Within each condition the pairs were presented in a random order, and within each pair the two orientations were allocated at

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 22 random to the target and mobile objects. The participant performed 12 practice trials for their first condition (as in Experiments 1 and 2) and 6 practice trials for the other conditions. As in Experiment 2, symptoms of VE sickness were monitored for 1 hr at the end of the experiment, but these data are not reported here. Results In the textured and colored conditions participants were instructed to respond immediately the objects became visible. This meant that there was a greater emphasis on speed of response than in Experiments 1 and 2. Data recording started at the same time as the objects became visible and that time was taken as the beginning of the rotation stage. The end of the rotation stage was defined using the same technique that was used in Experiments 1 and 2. Overall, the mean lengths of the fine-tuning stages were 3.9 s (textured), 3.6 s (colored) and 1.0 s (observe). The rotation stage data were analyzed using the same type of data transformation (logarithmic) as were used in the other experiments, and ANOVAs that treated the perception condition (textured vs. colored vs. observe), angle (45º vs. 135º) and axis combination (OVR vs. general) as repeated measures. Rotation time and rate. For the time data, a repeated measures ANOVA showed main effects of perception condition, F(2, 11) = 30.53, MSE = 0.01, p < .01, angle, F(1, 11) = 130.50, MSE = 0.01, p < .01, and axis combination, F(1, 11) = 38.63, MSE = 0.01, p < .01, and there was also a significant three-way interaction, F(2, 22) = 3.62, MSE = 0.01, p = .04 (see Figure 3). Trial completion time became faster as perception of the orientations became easier, and within each perception condition the pattern of the results were similar to the other experiments. For the rate data, repeated measures ANOVA showed main effects of perception condition, F(2, 11) = 5.79, MSE = 0.02, p < .01, and angle, F(1, 11) = 562.34, MSE < 0.01, p < .01, but not of axis combination, F(1, 11) = 2.72, MSE < 0.01, p .13. Means for the perception conditions were 83 deg/sec (textured), 85 deg/sec (colored) and 71 deg/sec Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 23 (observe), and were 59 deg/sec and 100 deg/sec for the 45º and 135º angles, respectively. The lack of an effect of axis combination is in line with the findings of the other experiments but there were significant interactions between axis combination and angle, F(1, 11) = 8.54, MSE < 0.01, p = .01, and angle and perception condition, F(2, 22) = 5.48, MSE = 0.01, p = .01. Rotation efficiency. The efficiency of each trial was analyzed using the PE-angle, RMS deviation, and IDOR data. For the PE-angle data, a repeated measures ANOVA showed main effects of perception condition, F(2, 11) = 48.10, MSE = 0.10, p < .01, and axis combination, F(1, 11) = 45.23, MSE = 0.03, p < .01, but not of angle, F(1, 11) = 0.02, MSE = 0.03, p = .88. However, there were significant interactions between axis combination and angle, F(1, 11) = 5.57, MSE = 0.06, p = .04, and perception condition and angle, F(1, 11) = 3.38, MSE = 0.05, p = .05, and a significant three-way interaction, F(2, 22) = 5.52, MSE = 0.03, p = .01 (see Figure 4). A repeated measures ANOVA of the RMS deviation data showed main effects of perception condition, F(2, 11) = 21.81, MSE = 0.04, p < .01, angle, F(1, 11) = 192.79, MSE = 0.02, p < .01, and axis combination, F(1, 11) = 80.26, MSE = 0.01, p < .01, and there was a significant interaction between axis combination and angle, F(1, 11) = 5.57, MSE = 0.02, p = .04 (see Figure 5). A repeated measures ANOVA of the IDOR data showed main effects of perception condition, F(2, 11) = 35.53, MSE = 367.80, p < .01, angle, F(1, 11) = 20.29, MSE = 269.04, p < .01, and axis combination, F(1, 11) = 9.79, MSE = 391.79, p < .01. There was a significant interaction between axis combination and angle, F(1, 11) = 29.19, MSE = 106.80, p < .01, and a significant three-way interaction, F(2, 22) = 5.04, MSE = 176.30, p = .02 (see Figure 6). Discussion As expected, in all five types of data reported above there was a main effect of perception condition. The increase in ease with which the orientations of the objects could be determined in the colored condition, compared with the textured condition, produced a Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 24 modest (14%) reduction in rotation time and a corresponding increase in rotation efficiency. However, there was no overall change in the percentage of IDOR trials between these two conditions, indicating that using color to identify the short and long stubs of the object had no initial effect on the ease with which participants could determine the shortest path of rotation. There was a large difference in performance between the observe condition and the two instantaneous response conditions. During the 10 s of observation participants were able to determine the relative orientation of the target and mobile objects so that their initial motor response was almost always in the correct direction. In turn, this brought about a decrease in rotation time and a corresponding change in efficiency of participants’ rotations. Despite this, a sizeable amount of inefficiency remained, probably caused by difficulties in executing the motor task of virtual rotation. Although rotation time was least in the observe condition, the rate of rotation was slower than in the other two conditions. However, this was probably caused by the emphasis on speed of response in those latter conditions. General Discussion The aim of the present study was to provide information about the manner in which people manually rotate objects in the real world, and the extent to which they can perform that rotation “naturally” in an immersive VE. While it is acknowledged that objects have many properties (e.g., size, weight, and shape) and tasks have many constraints (e.g., precision, or maintaining something “this way up”) that affect rotation, the scope of this study was restricted to a small abstract shape that was manipulated freely and spontaneously. The general pattern of the data was similar in all three experiments and three main findings should be highlighted. First, in common with simple (e.g., one-dimensional) motor tasks but in contrast to mental rotation, the mean rate at which rotations were performed increased with the angle between the stimuli. Second, under friendly experimental conditions (the OVR axis combination) participants performed rotation by using a near-shortest path.

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 25 Third, when there were inefficiencies in participants’ rotations they were caused by errors in the perception of the objects’ orientations or the motor response for rotation, rather than a deliberate but non-optimal path. Thus, some support was found for the common processing hypothesis (Wohlschläger & Wohlschläger, 1998) but there are also clear differences between manual and mental rotation. When performed under similar conditions (Experiments 1 and 2) virtual rotation was almost twice as inefficient as manual rotation (see the PE-angle data, above). Under easier experimental conditions (the observe condition of Experiment 3), the time required to perform virtual rotation, and the efficiency with which it was performed, became similar to that rotating an object manually and responding instantaneously. This shows that a large amount of the difference between manual and virtual rotation that was observed in the first two experiments was caused by the difficulty participants had perceiving the orientation of the objects in the VE, and the difference was present despite the fact that the images were displayed in stereo and standard rendering techniques (directional lighting and Phong shading) were used. Of course, if manual rotation were studied using the observe condition then further improvement would be likely. Therefore, some residual inefficiency remains in the motor task of virtual rotation but this is to be expected given the differences in the shape and inertia of the virtual objects and the prop. The three-colored object was used to explicitly identify the short and long stubs of the objects and, therefore, remove one source of error that occurred in the first two experiments. It was particularly effective in increasing the efficiency of trials performed using the most difficult combination of angle and axis (135-general), but only to the level of performance that occurred in manual rotation. Substantial questions remain about the cause of the perceptual problems that participants encountered during virtual rotation and further research is required to determine the extent to which they were caused by the optical quality of the

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 26 HMD (substantially less than that of a computer monitor) or the limitations of computer graphics algorithms in general (ray tracing is not possible in real-time and radiosity cannot be applied to scenes in which objects or the viewer moves). From an applied perspective, this study shows that a spherical prop interface only allows virtual objects to be manipulated in a natural manner to a limited extent, even though all three rotational degrees of freedom can be varied simultaneously. For example, virtual object manipulation times would have to be reduced by a scale factor to provide reliable data for method-time-motion studies of manufacturing operations, although the scale factor would become closer to one as the operations became more repetitive and the positions and orientations of objects became easier to predict. Finally, three additional points need to be made. First, the prop-type interface used in the present study is just one of many different types of interface that are used in VEs. Further investigations are required using alternative interfaces and rotation tasks that have other types of constraint. Second, the fine-tuning stage of the trials took substantially longer than the rotation stage. This means that the unexpectedly long rotation times recorded in some VE studies (e.g., Hinckley et al., 1997; Ware & Rose, 1998) were probably caused by the trial completion criterion that were used or an increased emphasis on accuracy, rather than any inherent difficulty in performing virtual object rotation. Third, the results help set out a manifesto for follow-up research in this understudied area. The data processing techniques and metrics used in the present study are well-suited for quantifying the efficiency of freeform object rotations performed in real and virtual environments, and other versions of the mental rotation paradigm such as successive presentation (Cohen & Kubovy, 1993) could be used to investigate the fine tuning stage of rotation, and to distinguish between observation time that can be used to perceive objects’ orientation or plan rotational paths.

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 27 Acknowledgements This work was supported by grant GR/L95496 from the Engineering and Physical Sciences Research Council. We gratefully acknowledge Raymond Nickerson and the anonymous reviewers for their helpful suggestions.

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 28 References Bethell-Fox, C. E., & Shepard, R. N. (1988). Mental rotation: Effects of stimulus complexity and familiarity. Journal of Experimental Psychology: Human Perception and Performance, 14, 12-23. Boritz, J., & Booth, K. S. (1997). A study of interactive 3D point location in a computer simulated virtual environment. Proceedings of the ACM Virtual Reality Science & Technology Conference (VRST ‘97, pp. 181-187), New York: ACM. Chen, M., Mountford, S., J., & Sellen, A. (1988). A study in interactive 3-D rotation using 2D control devices. Computer Graphics, 22, 121-129. Cobb, S., V., G., Nichols, S., Ramsey, A., Wilson, J., R. (1999). Virtual reality-induced symptoms and effects (VRISE). Presence: Teleoperators and Virtual Environments, 8, 169-186. Cohen, D., & Kubovy, M. (1993). Mental rotation, mental representation, and flat slopes. Cognitive Psychology, 25, 351-382. Dror, I. E., Ivey, C., & Rogus, C. (1997). Visual mental rotation of possible and impossible objects. Psychonomic Bulletin and Review, 4, 242-247. Fitts, P. M. (1954). The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 47, 381-391. Green, M., (1995). The MR Toolkit Version 1.4 [Computer program]. Department of Computing Science, University of Alberta, Canada. Hand, C. (1997). A survey of 3D interaction techniques. Computer Graphics Forum, 16, 269281. Hinckley, K., Tullio, J., Pausch, R., Proffitt, D., & Kassell, N. (1997). Usability analysis of 3D rotation techniques. Proceedings of ACM User Interface Software & Technology (UIST ‘97, pp. 1-10). New York: ACM.

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 29 Hinton, G. E., & Parsons, L. M. (1988). Scene-based and viewer-centered representations for comparing shapes. Cognition, 30, 1-35. Johnston, A., & Curran, W. (1996). Investigating shape-from-shading illusions using solid objects. Vision Research, 36, 2827-2835. Kaushall, P., & Parsons, L. M. (1981). Optical information and practice in the discrimination of 3-D mirror-reflected objects. Perception, 10, 545-562. Kirsch, D., & Maglio, P. (1994). On distinguishing epistemic from pragmatic action. Cognitive Science, 18, 513-549. Mamassian, P. (1997). Prehension of objects oriented in three-dimensional space. Experimental Brain Research, 114, 235-245. Mamassian, P., & Kersten, D. (1996). Illumination, shading and the perception of local orientation. Vision Research, 36, 2351-2367. Mapes, D. P., & Moshell, J. M. (1995). A two-handed interface for object manipulation in virtual environments. Presence: Teleoperators and Virtual Environments, 4, 403-416. Maruff, P., Wilson, P. H., De Fazio, J., Cerritelli, B., Hedt, A., & Currie, J. (1999). Asymmetries between dominant and non-dominant hands in real and imagined motor task performance. Neuropsychologia, 37, 379-384. Parsons, L. M. (1987). Imagined spatial transformation of one’s body. Journal of Experimental Psychology: General, 116, 172-191. Parsons, L. M. (1995). Inability to reason about an object’s orientation using an axis and angle of rotation. Journal of Experimental Psychology: Human Perception and Performance, 21, 1259-1277. Ruddle, R. A., Huddart, S. A., & Jones, D. M. (1999). Interaction in immersive virtual environments: Rotating objects with an instrumented prop. Proceedings of the Human

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 30 Factors and Ergonomics Society 43rd Annual Meeting (pp. 1214-1218). Santa Monica, CA: Human Factors Society. Ruddle, R. A., Payne, S. J., & Jones, D. M. (1999). The effects of maps on navigation and search strategies in very-large-scale virtual environments, Journal of Experimental Psychology: Applied, 5, 54-75 Shepard, R. N., & Metzler, J. (1971). Mental rotation of three-dimensional objects. Science, 171, 701-703. Shepard, S., & Metzler, D. (1988). Mental rotation: Effects of dimensionality of objects and type of task. Journal of Experimental Psychology: Human Perception and Performance, 14, 3-11. Tarr, M. J., Kersten, D., & Bülthoff, H. H. (1998). Why the visual recognition system might encode the effects of illumination. Vision Research, 38, 2259-2275. Wang, Y., & MacKenzie, C. L. (1999a). Object manipulation in virtual environments: Relative size matters. Proceedings of the Computer Human Interfaces Conference (CHI’99, pp. 48-55). New York: ACM. Wang, Y., & MacKenzie, C. L. (1999b). Effects of orientation disparity between haptic and graphic displays of objects in virtual environments. Proceedings of INTERACT’99 (pp. 391-398). IOS. Wang, Y., & MacKenzie, C. L. (2000). The role of contextual haptic and visual constraints on object manipulation in virtual environments. Proceedings of the Computer Human Interfaces Conference (CHI’00, pp. 532-5395). New York: ACM Ware, C. (1990). Using hand position for virtual object placement. The Visual Computer, 6, 245-253. Ware, C., & Rose, R. (1998). Real handles, virtual images. Proceedings of the Computer Human Interfaces Conference (CHI’98, pp. 235-236). New York: ACM.

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 31 Wexler, M., Kosslyn, S. M., & Berthoz, A. (1998). Motor processes in mental rotation. Cognition, 68, 77-94. Wohlschläger, A., & Wohlschläger, A. (1998). Mental and manual rotation. Journal of Experimental Psychology: Human Perception and Performance, 24, 397-412. Zhai, S., & Milgram, P. (1998). Quantifying coordination in multiple DOF movement and its application to evaluating 6 DOF input devices. Proceedings of the Computer Human Interfaces Conference (CHI’98, pp. 320-327). New York: ACM. Zhai, S., Milgram, P., & Buxton, W. (1996). The influence of muscle groups on performance of multiple degree-of-freedom input. Proceedings of the Computer Human Interfaces Conference (CHI’96, pp. 308-315). New York: ACM.

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 32 Table 1 Mean Rotation Time and Efficiency Data for the Angles and Axis Alignments in Experiment 1.

45º

135º

OVR General

OVR General

Fine tuning time (s) 10.4

10.5

9.4

9.4

1.0

1.5

1.9

3.8

56.3

54.9

91.1

81.6

25.1

56.4

21.8

112.8

9.6

13.6

14.1

38.1

12.9

9.7

13.3

43.3

Rotation stage time (s) Rotation rate (deg/s) Percentage extra angle (PE-angle) RMS deviation from shortest path (deg) % IDOR trials

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 33 Table 2 Mean Rotation Time and Efficiency Data for the Angles and Axis Alignments in Experiment 2.

45º

135º

OVR General

OVR General

Fine tuning time (s)

8.3

8.3

9.7

7.0

Rotation stage time (s)

1.9

2.2

4.0

5.6

Rotation rate (deg/s)

51.3

46.6

70.2

70.6

Percentage extra angle (PE-angle)

79.4

92.0

87.1

157.0

RMS deviation from shortest path (deg)

15.0

17.4

25.4

47.5

% IDOR trials

22.2

10.8

28.3

44.6

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 34 Figure Captions Figure 1. Examples of the axis combinations used in the experiments. The right hand images show the stimuli as they appeared in the HMD in Experiments 2 and 3, with the target object on top and the mobile object below. The left hand images show the orientation of the target object and the axis of the shortest path of rotation (R) between the objects, relative to the principal axes of the viewer (Vx, Vy and Vz are rightwards, forwards and upwards, respectively). O is the axis of the major limb of the object. From the top the images are for: rotation about the X axis (OVR condition), Y axis (OVR condition), Z axis (OVR condition), and the general condition. In the general condition the vector R is {0.58, 0.58, 0.58} and O is {-0.58, 0.58, 0.58}.

Figure 2. Mean orientation error during spinning object interface training in Experiment 2 (and standard error of the mean).

Figure 3. Mean rotation time in Experiment 3 (and standard error of the mean).

Figure 4. Percentage extra angle (PE-angle) the mobile object was rotated through in Experiment 3 (and standard error of the mean).

Figure 5.

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 35 Mean root mean square (RMS) deviation of mobile object from the shortest path in Experiment 3 (and standard error of the mean).

Figure 6. Mean percentage of trials in Experiment 3 where initial rotation was away from the target (IDOR) (and standard error of the mean).

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 36 OVR-X 135°

Vz

Vy VxOR

Vz OVR-Y 45°

VyOR Vx VzOR OVR-Z 45°

Vy Vx

Vz

O

General 135° R Vy Vx

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 37

Mean orientation error (deg)

90 deg/sec 60 deg/sec 30 deg/sec 125 100 75 50 25 0 1

2

3

4

5

Trial block

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 38

7 6 Time (s)

5 4 3 2 1 0 Textured

Colored

Observe

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 39

300 % extra angle

250 200 150 100 50 0 Textured

Colored

Observe

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

RMS angular deviation (degrees)

Manual And Virtual Rotation Page 40

60 50 40 30 20 10 0 Textured

Colored

Observe

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.

Manual And Virtual Rotation Page 41

70 % IDOR trials

60 50 40 30 20 10 0 Textured

Colored

Observe

Ruddle, R. A., & Jones, D. M. (2001). Manual and virtual rotation of a three-dimensional object. Journal of Experimental Psychology: Applied, 7, 286-296.