Shared Viewpoint Control for Information Visualization Stephen Hughes University of Pittsburgh Pittsburgh, PA 15260
[email protected] Abstract—Visualizations often rely on either animated guided tours or user-self exploration to control the viewpoint. This paper introduces a framework that supports a six level continuum of control between the viewer and the computer system in order to facilitate the extraction of information from the display. Furthermore, by dynamically adjusting the control along this continuum, the system can adapt to the changing information needs of the viewer.
Introduction Visualizations are artifacts of encoding raw data into inherently visual objects, allowing viewers to quickly gain insight to complex relationships that might otherwise be difficult to perceive. Until recently, visualizations were used to convey preformulated, explicit messages; the process of a graphic designer was laborious and the creation was static. The viewer was therefore limited to the role of a passive observer. Advances in computing power now permit the real-time construction of imagery that is complex, detailed and perhaps even life-like. As the complexity of the images increased, the ability to guide the viewer’s attention became necessary. Researchers turned to the art of cinematography to come up with methods for dynamically creating animations that steered the viewpoint through intricate 3D displays[1, 2]. While these tools can be useful to highlight important features to a naïve viewer, some tasks require self-exploration. Some interactive visualizations allow the user to manipulate the viewpoint in order to survey the environment. 3D viewpoint motion control has been the subject of intense scrutiny [3] and has yielded several stable metaphors including: World-inHand, Eyeball-in-hand, Walking, and Flying [4]. The ability to control the viewpoint allows the viewer to take a more active role in visualization; instead of simply observing, the viewers can engage in active sensemaking. Unfortunately, giving control of the viewpoint to the user can be detrimental: information-rich displays can distract or disorient viewers [5]. Control of the viewpoint in 3D visualizations does not need to be an all-or-nothing scenario. Beckhaus et. al. [6] propose the idea of allowing the viewpoint control to be toggled between the viewer an a guided tour. This paper extends that idea and suggests that a continuum of shared viewpoint control can support viewers with different needs. We begin by introducing Attentive Navigation, a technique for viewpoint manipulation that serves as the
basis shared control. We discuss six levels of shared control and how they can be realized using the Attentive Navigation technique. Finally, we explore some circumstances where the level of control can be dynamically manipulated to support the needs of the viewer, ranging from targeted information seeking to learning general knowledge from a display.
Attentive Navigation Attentive Navigation is one technique that allows intermediate control of the viewpoint [7]. A detailed description of the Attentive Navigation is beyond the scope of this paper, however a brief description here can provide the essentials for discussing an implementation for shared viewpoint control. The approach is based on work developed by Hanson et. al. to facilitate motion through a 3D virtual environment using a 2D controller [8, 9]. Hanson et. al. divide the navigation space of a display into two components: a Constraint Space and a Camera Model Field (CMF). The constraint space is the set of positional values that a viewpoint can assume. The CMF is a collection of vectors that provide “ideal” information for every point on the constraint surface. Practically speaking, the CMF is not explicitly represented; as the number of points that comprise the constraint surface can be arbitrarily large. Hanson and Wernert proposed sampling the CMF at a fixed resolution and then interpolating using a bicubic Catmull-Rom spline[8]. This ensures smooth transitions between the viewpoints and also keeps the CMF at a manageable size. The semantics of this “ideal” vectors can vary depending on the implementation. Attentive Navigation proposes storing two vectors at each sampled point: the ideal gaze direction and the ideal ‘next motion’ direction. The ideal gaze direction will be used to provide a suggested orientation for the viewpoint; the direction that the virtual camera should be facing given a certain
location. The ideal motion vector can be used to suggest the next logical positional value as the virtual camera moves through the viewpoints. Thus, Attentive Navigation effectively serves as a recommendation engine capable of instructing the viewer to the ideal viewing orientation as well as the suggested motion from ones current position.
Shared Viewpoint control The shared viewpoint control can be implemented at several levels depending on how the information stored in the CMF is used. Specifically, six levels are discussed in order of decreasing computer guidance. Complete Guidance: As described previously, this method offers no options to the viewer; they merely observe as the computer determines the appropriate viewpoint changes required to understand the visualization. This is accomplished programmatically by first using the motion vectors from the CMF to move the viewer to a new location. The CMF is given the current location and responds with the “ideal” next step. After the position has been updated, a similar operation is done with the gaze CMF to align the orientation of the camera with the ideal viewing location. This process can be repeated, leading to continuous motion through the display, as demonstrated in Figure 1.
Fig. 1. Complete Control of the Camera Motion can be achieved by using the CMF to direct the “ideal next step. The solid line shows that in this example, the camera will continuously cycle through the visualization.
Restricted: This method allows partial control of the viewpoint to be assumed by the viewer. Either the orientation or the location is determined by the viewer, while the other is extracted from the CMF. If the viewer controls the orientation and the system determines the motion, the situation is analogous to riding in a tour bus. The bus follows a predetermined path (designed to pass interesting features) while riders are free to choose where they wish to look. The inverse of this method allows the viewer to control the motion, leaving the orientation to the
CMF. While this does not have the familiar analogy of the previous example, it can still generate a very powerful effect. As the person moves through the environment, their gaze will be automatically redirected to focus on objects of interest. As they move past these objects, their gaze can remain fixated on the object as shown in figure 2. Their attention will be focused on that object until they near another interesting feature or zone of interest. By navigating in this manner, the viewer gets a preview of all the salient features in the environment.
Fig. 2. Restricted control where the motion is controlled by the viewer and the orientation is controlled by the system results in automatic gaze redirection to fixate on significant objects.
Facilitated: This approach extends the viewer’s control further. As before, the CMF is automatically involved to suggest a direction of motion or gaze. However, now the viewer has the option of overriding the suggestion. This allows the viewer to explore beyond the limitations imposed by the previous approaches. In keeping with the spirit of a shared viewpoint control continuum, the degree to which (and the ease with which) the viewer can override the suggestions can be limited. For example, a system may want to offer facilitated control, but wanted to keep the viewer from straying too far from the plan. In this case, the gaze could be limited to within 45° of the ideal. Requested: The viewer is granted full autonomy for this level of control. However, the CMF values are retained for assistance as needed. The viewer is able to explore the scene on their own will, but at any time they recognize that they need help, they are able to ask the computer to intervene and show them the ideals. When the computer hears a request, it can align the viewer with the ideals, literally pointing them in the right direction. This is equivalent to asking for a hint or stopping to ask for directions. Annotated: Again the role of the computer suggestions has been relegated to the background. In this method, the CMF vectors are used to power additional imagery that supplements the standard visualization. Highlighting or arrows may be used to signify critical elements of the display, as shown in figure 3. At this point, the computer is not capable of directly intervening in the control of the
Fig. 3 The CMF vectors are used to project an arrow in the display, indicating the direction of an object of interest.
viewpoint; it is up to the viewer to control the viewpoint in a manner that is consistent with what they perceive the machine is suggesting. Free: The upper end of the viewpoint control spectrum has the viewer in complete control, with the machine offering no assistance.
Appropriate Levels of Control The power to adjust the amount of control is not necessarily a design level decision. The level of assistance that is appropriate is determined largely by the information needs of the viewer. The interaction with a person seeking specific information is likely to be very different from someone who wishes to learn about the environment at large. Therefore collaboration with the system may certainly vary from user to user and even from use to use. Fuzzy Searches To find specific information requires that the viewer be able to express a query to the computer. The degree to which the system can interpret and process the viewer’s information needs will correspond to the amount of control that it can assert. If the viewer can express precisely what they are looking for, e.g. “Show me all the paintings by Cezanne”, the system can take full control and lead the viewer directly to the points of interest in the environment. Some searches, however, may contain more abstract queries, or rely on fuzzy data, e.g. “I am interested in finding paintings that were inspired by religious themes”. In these cases, the control assumed by the system could be relaxed. The system might adopt a facilitation approach, indicating that it has a reasonable basis for making a recommendation, but offering the viewer the option of exploring some boundary conditions as well. As a final example, the system might be incapable of matching the viewer’s request with the knowledgebase that it is using to make its decisions.
Under these circumstances, the system probably should not take away any control from the viewer, as it is incapable of providing any real assistance. While the support provided by the system may be initially marginal, it may be able to increase its assistance. By monitoring the ways that the viewer responds to the display, the system may be able to better infer the viewer’s information needs, implicitly refining the query. Accordingly, the system can alter the way it reacts, perhaps subtly influencing the viewpoint to draw attention to regions of potential interest. This kind of adaptive navigation support has proven effective for hypertext systems and it is likely that a similar approach can also be applied to visual environments. In addition to slight modifications, the system may want to more actively engage the viewer. With shared control, a system might manipulate the viewpoint in order to ‘ask’ questions. By directing attention to an object of questionable relevance, a system can effectively ask, “Is this important to you?” By gauging the viewer’s response, the system can update its internal model of the viewer’s information needs. Sharing control of the viewpoint can therefore serve to both increase the flexibility of an exploration as well as allow the viewer to collaborate with the system to refine a search. Structured Learning Another task for visualizations require learners to gain a global understanding of a set of concepts. Visual simulations can augment a user’s ability to obtain a skill in managing a process or manipulating controls by providing a sense of presence. Naive simulations often rely on learning from mistakes over numerous repetitions in order to foster understanding – a technique known as “drill-andpractice”. An alternative approach, known as “scaffolding”, describes a collaborative process that allows a student to complete tasks that might ordinarily be too complex for them. This concept is built from the work of Vygotsky, who proposed that instructors should manage learners’ experiences by giving them portions of the task that are within their ‘Zone of Proximal Development’ [10]. Over time the students take a more active role in the completion of the task until they are able to complete it independently. It has been argued that this kind of pedagogy has been successfully built into other online tutoring systems [11], so again, it might be applied to reasoning with visualizations and simulations as well. The scaffolding process can be achieved by refining the amount of control that the viewer exerts, ranging from passive to complete. Initially, the viewer can watch as the system uses complete control to point out the important features. As the viewer begins to understand what is significant, they can attempt to interact with the display with less intervention from the system. When the viewer has obtained a good understanding of the organization in the environment, they might only need the hints provided
by requested control. Ultimately, when the viewer has achieved the desired learning, they can freely navigate without the assistance of the system. Therefore, it is possible to provide a more structured approach to learning from a simulation by progressively granting more control to the viewer.
being presented. This motivates the need for the support structure to be dynamic, a characteristic that should readily extend to our framework. Ultimately, rules to support this adaptation need to be investigated and is the focus of ongoing work.
References Additional Considerations The ability to dynamically adjust the level of viewpoint control requires two additional considerations: a) the ability to make alterations to the CMF on the fly and b) an inference monitor that determines when such alterations need to be made. A CMF generation tool is currently being developed to assist with the first task. The initial prototype consists of a simple drag-and-drop interface that allows markup of a projected 2D map of the environment. This tool simplifies the task, however, since it requires human interaction, it is not sufficient for creating dynamic, customized CMFs. Ongoing work is being done to establish a batch process to automate this tool and the author does not see any major technical obstacles to achieving this. Another drawback to the current implementation is that it relies on a 2D projection. This immediately restricts the navigation space to 2 degrees of freedom for position and 1 degree of freedom for orientation. This is consistent with a “walking” metaphor that is common to a large number of virtual environments, but it is still not acceptable as a general solution. Fortunately, this restriction is strictly a function of the interface that is used for the mark-up, and is not indicative of the technique itself. Again, once the batch tool can be completed, this constraint should evaporate. This tool shows promise for how the CMF can be generated on the fly, but rules for when and how to make updates are still needed. The development of these inference rules is still the focus of ongoing work but the author believes that strong analogies can be drawn from what has already been learned from the field of adaptive hypermedia.
Conclusions The ability to interact with visualizations has greatly increased people’s ability to reason about complex data sets. However, one of the critical elements of this interaction, finding an appropriate point of view, is often left to the viewer. We have presented a framework for providing the assistance to the viewer, according to their needs. Moreover, we have noted that these needs may change as the viewer is better able to express their goals, or as they become more knowledgeable about the data
1. He, L.w., M.F. Cohen, and D.H. Salesin. The virtual cinematographer: A paradigm for automatic real-time camera control and directing. in SIGGRAPH 96. 1996: Addison Wesley. 2. Drucker, S.M. and D. Zeltzer. Intelligent Camera Control in a Virtual Environment. in Proceedings of Graphics Interface '94. 1994. Banff, Alberta, Canada. 3. Bowman, D., D. Koller, and L. Hodges. Travel in Immersive Virtual Environments: An Evaluation of Viewpoint Motion Control Techniques. in Virtual Reality Annual International Symposium. 1997. 4. Ware, C., Information Visualization: Perception for Design. 2000, San Fancisco: Morgan Kaufmann. 5. Goerger, S., et al. Spatial Knowledge Acquisition from Maps and Virtual Envrionments in Complex Architectural Spaces. in 16th Applied Behavioral Sciences Symposium. 1998. U.S. Air Force Academy, Colorado Springs, CO. 6. Beckhaus, S., F. Ritter, and T. Strothotte, Guided Exploration with Dynamic Potential Fields: The CubicalPath System. Computer Graphics Forum, 2001. 20(4): p. 201-210. 7. Hughes, S. and M. Lewis. Attentive Camera Navigation in Virtual Environments. in IEEE International Conference on Systems, Man & Cybernetics. 2000. 8. Hanson, A. and E. Wernert. Constrained 3D Navigation with 2D controllers. in Visualization '97. 1997: IEEE Computer Society Press. 9. Hanson, A., E. Wernert, and S. Hughes, Constrained navigation environments, in Scientific Visualization: Dagstuhl '97 Proceedings, H. Hagen, G.M. Nielson, and F. Post, Editors. 1999, IEEE Computer Society Press. p. 95-104. 10.Vygotsky, L.S., Mind in Society. 1978, Cambridge: Harvard University Press. 11.Hübscher, R. and S. Puntambekar. Adaptive Navigation for Learners in Hypermedia is Scaffoleded Navigation. in Adaptive Hypermedia and Adaptive Web-Based Systems. 2002. Malaga, Spain: SpringerVerlag.
