Dynamic Three-dimensional Information Visualization for Quantitative Information in Augmented Reality Systems Whung Yee Lai *, Henry Been-Lirn Duh † School of Mechanical and Production Engineering Nanyang Technological University, Singapore Abstract Quantitative information visualization such as statistical graphics is concerned with the visual representation of quantitative and categorical data for statistical analysis. With improvements in graphics display technology, it is now possible to make use of motion to stimulate recognition of patterns and structure embedded in quantitative data. Past studies have shown that the judged final position of a moving target is often displaced in the direction of the anticipated future motion of the target. Termed as representational momentum, these memory distortions have a strong relationship with the target’s velocity. This study investigated human performance in visualizing dynamic quantitative information in augmented reality environments. Statistical results showed that the differences in speed and percentage change affect subject’s accuracy in perceiving quantitative information significantly. On the other hand, the differences in display devices (Head-mounteddisplay and Liquid-crystal-display) did not indicate significant effects on subject’s performance. Our results also showed that as the speed increases, the errors made in judging the final position of the moving bar also increases. CR Categories: H1.2 [User/Machine]: human factors; H5.2 [User Interface]: GUI Keywords: augmented reality, information visualization, dynamic information visualization
1
possible now to make use of motion to stimulate recognition of patterns and structure in information. In fact our human visual system is very sensitive to perceiving motion. Quantitative information can be presented dynamically by showing a series of static images at a proper rate to evoke apparent motion. At each frame, the object is place at different positions, when displayed at a proper rate, the object will be perceived as moving continuously. The proper rate depends on factors such as the alternation rate and the separation between the two positions of the object. If the display rate is too fast (e.g. blades of a fan) or too slow (e.g. clock hour hands), the object will not be perceived as moving. Thus there is a range of speed where people will actually perceive continuous motion in order to perceive a clear and distinct motion, the duration between each frame must decrease as the separation between the two positions of the object is increased (Korte’s third law). Bartram [1997] mentioned that motion might provide a great potential for effective information visualization because it is perceptually efficient, interpretatively rich and it has not been over-coded. It is still unclear whether people’s perception of quantitative information will be the same using different speeds and different display devices. When visualizing dynamic information, the user will need to continuously track the motion of the stimulus that is displayed to them. In the real world, the mental representation of a moving stimulus cannot be stopped instantaneously due to the physical momentum of the real-world objects. Instead, it continues for some time such that the judged final position is ahead of the actual position. Hubbard and Bharucha [1988] found that this phenomenon also exists in the virtual environment where the judged final position of a moving target is often displaced in the direction of the anticipated future motion of the target. Termed as representational momentum, it refers to the distortion in visual memory that is induced by implied motions of a pattern or its elements [Freyd and Finke, 1984].
Introduction
Quantitative information visualization is the depiction of quantitative and categorical data using graphical representations to facilitate discovery, decision-making, explanation and other cognitive skills. Visualization helps people to grasp the content of a picture much faster than scanning and understanding texts. A successful information visualization system should be able to convert abstract information into a visual representation while preserving the underlying meaning and at the same time providing new insights to the user. The advances in statistical computation and improvements in graphic display technology have provided much better tools for visualization as compared to those in half a century ago. It is † e-mail:
[email protected] *e-mail:
[email protected]
Copyright © 2004 by the Association for Computing Machinery, Inc. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Permissions Dept, ACM Inc., fax +1 (212) 869-0481 or e-mail
[email protected]. © 2004 ACM 1-58113-884-9/04/0006 $5.00
Short Paper
148
Finke, Freyd and Shyi [1986] mentioned that factors that influence physical momentum should also influence representational momentum because representational momentum is resulted from an internalization of the laws of physical momentum by the representational system. Since physical momentum is defined as the product of an object’s velocity and the mass thus both of these factors should also influence representational momentum [Hubbard, 1997]. Previous researches have reported strong relationships between target velocity and representational momentum [Freyd and Finke, 1985; Hubbard, 1996; Hubbard and Bharucha, 1988]. Some of the basic characteristics of representational momentum have been uncovered in several studies. For example, the visual memory distortions increase in proportions to the implied velocity similar to the way physical momentum increases in proportion to
an object’s true velocity [Finke et al., 1986]. Finke et al mentioned that these memory distortions arise because there is a tendency to mentally extrapolate the implied motions beyond the end of the inducing sequence. Due to the increase in representational momentum, observer will find it harder to stop the extrapolate process at the end of the inducing sequence just as increasing an object’s physical momentum makes it harder to stop the object at some point along its actual path of motion. As a result, a person’s memory for the last–observed display is shifted forward, along the direction of the implied motions, by an amount proportional to the implied velocity [Finke and Shyi, 1988].
2
Method
2.1 Participants Twelve undergraduates (6 males and 6 females) were recruited from the School of Engineering at Nanyang Technological University. All subjects have taken first year Engineering Graphics course. All subjects have normal or corrected-to-normal vision.
2.2 Apparatus A green color dynamic bar chart created from 3D Studio Max (Discreet, Inc.) was displayed to the subject in a HMD-iglasses3D (i-O Display systems, Inc.) or 15” desktop LCD monitor (NEC, Corp.). The AR environment was created using the ARToolkit (Kato, et. al, 1999). The bar chart was displayed at a resolution of 800 x 600 pixels in the HMD and LCD display devices. The camera angle in the AR environment was approximately 41° (see Figure 1).
Finke and Shyi [1988] mentioned “cognitive resistance”. Just like how a car’s stopping distance is dependent on both its physical momentum and on how effectively the brakes are applied, the amount of representational momentum and how effectively the tendency to continue the extrapolation is resisted will also determine the stopping distance along representational pathways. It is assumed that cognitive resistance is used to dissipate representational momentum similar to the way a car’s brakes dissipate physical momentum. [Finke and Shyi, 1988]. Thus the greater the representational momentum, the faster the effect of cognitive resistance.
Chin rest
Web cam
There are three different categories of visual display devices that can be used in augmented environment systems namely the headmounted-displays (HMDs), desktop monitors and large screen projection systems. These displays vary along many dimensions, such as resolution, field-of-view (FOV), level of immersion, and so on. Generally, there is no guideline as in which display device is more appropriate for a particular application. Most HMDs offer limited FOV as compared to a normal human FOV. Narrow FOV has been shown to degrade human performance on naiviation, manipulation, and spatial awareness tasks and visual search tasks. [see Duh et al., 2002]. It causes longer task completion times, disrupted eye and head movement coordination, and misperception of size, space, and ego-center [Alfano and George, 1990].
Position of AR object
Humans perform better in certain tasks in a visually immersive environments than in a traditional desktop displays as they are able to built a mental frame-of-reference for the space [Randy et al., 1997]. However people do not make very accurate spatial judgments in visually immersive environment [Sinai et al., 1999]. Factors like limited FOV, limits on sharpness and resolution may have contributed to the misperception of distance [William et al., 2002].
Headmounteddisplay
Table
Subject’s location
a) HMD display
LCD Monitor
Web cam Chin rest
The objective of this paper is to investigate the effect of speed on people’s perception of dynamic quantitative information in AR environments. Subject’s accuracy in judging percentage change at five different speeds in a dynamic bar chart presented in HMD and Liquid-crystal-display (LCD) will be compared.
Position of AR object
Table
Subject’s location
b) LCD display Figure 1: Experimental setting
149
(F (1,20) = 0.04, p > 0.05). All interaction effects are not significant.
2.3 Procedures In both displays (HMD and LCD), there would be an increase in height of the bar chart. (20%, 40%, 60%, 80% or 100% increase). For each stimulus, the bar chart would be vibrating to and fro in one of the five percentage changes. The task of the subject was to judge the percentage change in the height of the dynamic bar chart using the HMD and LCD display. For each display, 50 trials were randomly presented to each subject. They composed of five different speeds (10, 40, 120, 160 Hz) by five different percentage changes (20, 40, 60, 80, 100 %). Each stimulus was displayed to the subject twice randomly. Therefore for two display devices, there were a total of 100 trials. Subjects were given 5 minutes of rest time after every 25 trials. Displays (HMD/LCD) were between-subjects factors. The experimenter would give a brief explanation of the experiment procedures and apparatus to the subjects. Subjects were given 5 minutes practice on the tasks before the actual experiment. (see Figure 2 for experimental stimulus)
For both HMD and LCD displays, subjects performed well at the slowest speed (10 Hz) with all the five percentage changes (20 %, 40 %, 60 %, 80 %, 100 %) having less than 15 % errors. As the speed increases, the error for all the five percentage changes also increases. Among the five different speeds, subjects perform the worst at the fastest speed (160 Hz). This is especially true for the 20 % change in height of the bar where subjects recorded more than 50 % errors. Subjects’ performance in HMD display did not differ very much as compared to the LCD display. This is justified by the result from the ANOVA, which shows that the differences in display are insignificant. For the two displays, the most significant error occurred at 160 Hz for all the five percentage changes in height of the bar chart. Subjects achieved the most accurate results at 10 Hz under HMD and LCD displays.
Absolute errors
AR HMD
10
20%
5
40%
0
60% 80%
-5
100%
-10 -15 -20 10
40 80 120 Speed (Hz)
160
Figure 2: Stimulus showed in the experimental conditions
AR LCD
3
Absolute errors
During the actual experiment, subjects only opened their eyes when the stimuli showed up. Although there was no time limit for the experiment, the subjects reported the answer only when they had at least 80% confidence of the answer. The time taken to answer the question was recorded after every trial. At the end of the experiment, subjects completed a simple questionnaire regarding their experience during the experiment.
Results
10
20%
5
40%
0
60% 80%
-5 t
-10
100%
-15 -20
The results are shown in Figure 3. The chart shows the error for the AR conditions using the HMD and LCD display devices at the five different speeds: 10, 40, 80, 120, 160 Hz. The results demonstrated that subjects had a higher tendency to overestimate at lower speed and underestimate at higher speed. The results from both HMD and LCD display conditions exhibit a similar trend, which is subject’s judgments at lower speed, is mostly higher than judgments at higher speed.
10
40 80 120 Speed (Hz)
160
Figure 3: Experimental results
4
Conclusion
The two display devices – HMD and LCD used in this study have different dimensions. Although the resolution in both displays has been set to 800 x 600 pixels, the FOV, level of immersion in the HMD still differ from the LCD monitor. However the experimental results have shown that the type of displays did not affect the accuracy when visualizing a dynamic bar chart.
A two (display) x five (speed) x five (percentage change) mixed ANOVA was performed to evaluate the effects of these independent variables on the accuracy of the subject’s judgment. ANOVA results indicated that the differences in the speed and percentage change are significant (Significant effects of speed, F (4,550) = 36.02, p < 0.01; percentage change, F (5,550) = 770.64, p < 0.01). However the differences in display are not significant
150
The results from this study did support the hypothesis that speed does affect human performance. Hubbard and Bharucha [1988] mentioned that the judged final position of a moving target is often displaced ahead in the direction of the anticipated position. Termed as representational momentum, this observed shift in memory will increased with velocity [Finke et al., 1986]. In this study, the results also reflected this phenomenon in that as the speed increases, the errors made in judging the final position of the moving bar also increases. However, subjects did not overestimate as the speed increases. Instead the increase in representational momentum of the dynamic bar has caused subjects to underestimate the percentage increase of the bar chart as the speed increases. It seems that at a slower speed, subjects tend to apply cognitive resistance at a later stage to dissipate the representational momentum. On the other hand, at a higher speed, cognitive resistance is applied earlier in order to stop the increased in representational momentum effectively. This may have resulted in subjects underestimating the percentage increase as the speed increases.
Experimental Psychology: General, 115, 175-188. Finke, R.A., & Shyi, G.C.-W. 1988. Mental extrapolation and representation momentum for complex implied motion. Journal of Experimental Psychology: Learning, Memory and Cognition, 14, 112120. Freyd, J.J., & Finke, R.A. 1984. Representational momentum. Journal of Experimental Psychology: Learning, Memory and Cognition, 10, 126132. Freyd, J.J., & Finke, R.A. 1985. A velocity effect for representational momentum. Bulletin of the Psychonomic Society, 23, 443-446. Hubbard, T.L. 1996. Representational momentum, centripetal force and curvilinear impetus. Journal of Experimental Psychology: Learning, Memory and Cognition, 22, 1049-1060. Hubbard, T.L. 1997. Target size and displacement along the axis of implied gravitational attraction: Effects of implied weight and evidence of representational gravity. Journal of Experimental Psychology: Learning, Memory and Cognition, 23, 1484-1493.
Overall, subject performed the worst at the fastest speed of 160 Hz. They made errors of up to 50% for both HMD and LCD displays for the 20% increase in height of the dynamic bar chart. On the other hand, they made less than 10 % errors at the slowest speed of 10 Hz.
Hubbard, T.L., & Bharucha, J.J. 1988. Judged displacement in apparent vertical and horizontal motion. Perception & Psychophysics, 44, 211211. Kato, H., Billinghurst M., Blanding B., May R., 1999. ARToolkit, Technical Report, Hiroshima City University
In view of these results, it can be seen that the speed to introduce to a dynamic bar chart will affect performance of people. Using the wrong speed for the information visualization system does not only provide new insights to the user but most importantly the visual representation had actually distorted the underlying meaning of the actual data. With errors reaching more than 20% for certain speeds, the detrimental effects can seriously render the information visualization system less usable.
Lampton, D. R., Knerr, B. W., Goldberg, S. L., Bliss, J. P., Moshell, J. M. and Blau, B. S. 1994. The Virtual Environment Performance Assessment Battery (VEPAB): Development and Evaluation, Presence: Teleoperators and Virtual Environments, 3(2): 145-157. Randy, P., Dennis P., and George, W. 1997. Quantifying immersion in virtual reality. Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, 13-18. Sinai, M.J., Krebs, W.K., Darken, R.P., Rowland, J. H., and McCarley, J.S. 1999. Egocentric distance perception in a virtual environment using a perceptual matching task. Proceedings of the 43rd Annual Meeting Human Factors and Ergonomics Society, Houston, TX, 43, 1256-1260.
More research needs to be done to determine how the introduction of more dynamic bar charts instead of only one affects human’s perception and performance. AR may be use as a tool for information visualization however more study is needed in areas such as whether human performance will improve if users are allowed to manipulate the bar chart.
William, B. T., Peter, W., Amy, A. G., Sarah H. C., Jack, M. L., and Andrew, C. B. (2002). Does the quality of the computer graphics matter when judging distances in visually immersive environments? Technical Report UUCS-02-015, University of Utah.
Acknowledgments Thanks for Dr. Parker giving advices and comments in the early draft of this paper. This research is supported by NTU SDS grant.
References Alfano, P.L & George, F.M. 1990. Restricting the field of view: perceptual and performance effects. Perceptual and Motor Skills, 70(1), 35-45. Bartram, L. 1997. Can motion increase user interface bandwidth? Proceedings of IEEE Conference on Systems, Man and Cybernetics 1997, 1686-1692. Duh, H.B.L., Lin, J.J.W., Kenyon, R.V., Parker, D.E., Furness, T.A. 2002. Effects of characteristics of image quality on balance in an immersive environment. Presence: Teleoperators and Virtual Environments, 11(3), 324-332 (SCI). Finke, R.A., Freyd, J.J., & Shyi, G.C.-W. 1986 Implied velocity and acceleration induce transformation of visual memory. Journal of
151
