VI S UA LIZ AT10 N B LACK BOAR D
The Responsive Workbench Wolfgang Krueger and Bernd Froehlich German National Research Center for Computer Science
The standard metaphor for humancomputer interaction arose from the daily experience of a white-collar office worker. For the last 20 years, desktop systems have been enhanced more and more, providing tools such as line and raster graphics, window-icon-mousepointer graphical user interfaces, and advanced multimedia extensions. With the advent of immersive virtual environments, the user finally arrived in a 3D space. Walkthrough experiences. manipulation of virtual objects, and meetings with synthesized collaborators have been proposed as special humancomputer interfaces for the scientific visualization process. Another approach to the design problem for future human-computer interfaces relies on the early ideas of Myron Kruegerl-nonimmersive interactive multimedia environments. Basically, they rigorously center the user’s point of view. Application-oriented visualization environments have been proposed and built to support a specific problem-solving process. The computer acts as an intelligent server in the background, providing necessary information across multisensory interaction channels.?.’ We developed the Responsive Workbench concept as an alternative to the multimedia and virtual reality systems of the past decade. Analyzing the daily working situations of such different computer users as scientists, architects, pilots, physicians, and service people in travel agencies and at ticket counters, we recognized that almost nobody wants simulations of their working worlds in a desktop environment. Generally, users want to focus on their tasks rather than on operating the computer. Future computer systems should use and adapt to the rich human living and working environments, becoming part of a responsive environment.
Virtual working environment The Responsive Workbench is a virtual working environment that locates virtual objects and control tools on a real “workbench” (see Figure 1).
12
Figure 1. Scheme of the Responsive Workbench.
The objects-computer-generated stcreo imagcs-are projected onto the sui-face of the workbench. This setting corresponds to the actual work situation in an architect’s office, in surgery, and s o forth. A human guide uses the virtual working environment while scveral collaborators watch events through stereo shutter glasses. In Figure 2, for example, two architects use the Responsive Workbench to collaborate on a design. The participants operate within a nonimmersive virtual environment. Depending o n the application, the virtual workbench can integrate various input and output modules, such as motion, gesture. and voice recognition systems. This characterizes the general trend away from the classical humanmachine interface. Several guides can work together in similar environments either locally or by using broadband communication networks. A responsive environment. consisting of powerful graphics workstations, tracking systems, cameras. projectors, and microphones, replaces the traditional multimedia desktop workstation.
Realizing the possibilities Implementing the Responsive Workbench required close attention to several important elements: the user interface, feedback speed, and real-time rendering. Application-h uman interface The most important and natural manipulation tool for virtual environments is the user’s hand. Our environment depends on the real hand. not a computer-generated representation. The user wears a dataglove with a Polhemus sensor mounted on the back. Gesture recognition and collision detection algorithms, based on glove and Polhemus data, compute the user’s interaction with the virtual world objects. On top of this basic level we implemented operations on objects’ topology and geometry, such as removing and adding vertices of objects, tweaking of vertices, and choosing and moving around objects. Another approach for interaction with virtual world objects-natural language-fits well into our responsive environment. The user issues com-
IEEE Computcr Graphics and Applications
Figure 3. A virtual patient on the Responsive Workbench.
reduces the realistic appearance of virtual objects. Therefore, the system reads the latest available Polhemus data before the culling process starts, defining the new viewing frustra. Hand tracking and speech recognition are not as real-time critical. since a delay of two or three frames does not seem noticeable compared to delayed head-movement response. Fast sound feedback coupled to collision detection yields a more realistic feeling for user interaction. Rendering We implemented the Responsive Workbench project on a Silicon Graphics Onyx RE2 workstation. Rendering is done using Iris Performer, which efficiently uses multiple processors to achieve real-time graphics performance. Performer supports parallel processes for application computations, culling, and rendering.
Applications So far we have embedded two kinds of applications in this new type of environment: medical and architectural. The first application involves surgery planning and nonsequential medical training based on a model of a patient. Figure 3 shows thc model, called the transparent woman. in a teacheristu-
May 1994
13
Figure 4. Zooming in and varying transparency.
dent scenario. The patient's skin can become transparent, making the arrangement of the bones visible (see Figure 4). Now the surgeon or student can pick a bone with the dataglove and examine its joints, or take a close look at the bone itself (see Figure 5 ) . Even the beating heart can be removed and examined (see Figure 6). The application extends to surgery planning as well. using real data sets obtained from computed tomography (CT) or magnetic resonance imaging (MRI) measurements. The second application involves the design and discussion process in architecture, landscape, and environmental planning. Figure 2 shows an architectural model on the workbench, in this case the area around our company's buildings. In front of the table, two architects discuss the model, moving buildings or other objects such as trees in the virtual world. Additionally. users can set light sources with the dataglove to simulate different times of day. For this environment, the concept of active objects appears essential, for example, cars driving around, pedestrians walking along the street, and so on. We solved the problem of generating an animation path for each object by adding a Polhemus sensor that can be moved around in the virtual world like an object to be animated. The Polhemus generates the position, orientation. and velocity data for the animation path.
Figure 5. Picking a bone with the dataglove to take a closer look at it.
Future work We designed the Responsive Workbench to demonstrate the power of future cooperative responsive environments. Further applications on this virtual workbench will include an adapted "virtual windtunnel" for automobile design, and the simulation of air and ground traffic in airports. Generally, the issue is to discuss and analyze their specific tasks and work situations with other users, who are not necessarily accustomed to working around a workbench. As with other virtual environments.
14
IEEE Computer Graphics & Applications
COMPUTERANIMATION '94 May L 5-28, 1994 Geneva, SWITZERLAND
The proceedings contains the latest interdisciplinary results from reseachers in computer vision. image synthesis. psychology. artificial life. and physics that give life to pictures and animate our world.
Sections:Phq sics-Based Animation. Facial Ani in a t i o n. Be h a \ I oral Ani mat i o n. Animation Systems and Techniques. Rendering of Animation Sequences. Multimediv and Animation. 216pages ColorPlates Softcover Catalog# 6240-02 $70 00 -Members $35 00
UNDERSTANDINGCOMPLEX DATA WITH COMPUTER ANIMATION Avideo companion to the IBM Journalof Research and Development Figure 6. Examining the heart.
technical problems occur in the areas of real-time rendering of complex scenes and latency. Special prediction filters will lower the time delay to less than 0. I second. All our experiments show that tactile feedback via the dataglove is 3 most desirable.
Acknowledgments We thank our colleagues and students Manfred Berndtgen. Christian Bohn. Heinrich Schueth. Thomas Sikora, Josef Speier, Wolfgang Strauss. Gerold Wesche. and Juergen Ziehm for their extraordinary involvement in software and hardware management and modeling.
References 1. M.W. Krueger. Artificicrl Rmliry I I , Addison-Wesley. Reading. Mass.. 1991. 2. J. Nielsen. " N o n c o m m a n d U s e r Interfaces." Coi7lt?z.ACM. Vol. 36. NO. 3. April 1993% pp. 83-99.
3. A. Marcus. "Human Communications Issues in Advanced IJl's." C ' o t i i u z . ACM. Vol. 36. No. 4. April 1993. p p . 101-109.
Submissions Sought We invite submissions to the Visualization Blackboard.Contact the publicationsoffice for an author's guide: Carole Danner Editorial Secretary IEEE Computer Society 10662 Los Vaqueros Circle Los Alamitos, CA 90720 or contact managing editor Nancy Hays by e-mail at n.haysQ compmail.com. Send proposals and submissions to Lawrence1. Rosenblum Visualization Blackboard Advanced Information Technology Branch Code 5580 Naval Research Laboratory Washington, DC 20375-5320
[email protected]
The combination of rendering. imaging, and animation tools is the subject of this special issue containing 23 papers o n imaging multidimensionaI data. interactive interpretation. and processing requirements. The video shows how interactive imaging and animation habe enabled eight users to fully utilire their data in areas ranging from the perfomiancc of microscopic transistors, to "learning" proces.; of a neural network. to the analysis ofcrime statistics in the US.
Sections: Simulation of ZD Fluid Dynamics. Visualizing Neural Network Processes. Interactive Visualization for Mechanical Analysis, Visualizing Multivariate Structure with VISUALSIPxPI, Interpretation of Multivariable Data in 3D. Interactive Analysis of4D Vector Fields. 3D Molecular Dynamics. Visualization of Fluid Flow. 79minutes 1991 Videotape# 5103-23 (VHS) $79 00- Members$59 00
IEEE COMPUTER SOCIETY PRESS 10662 Los Vaqueros Circle Los Alamitos, CA 90720-1264
v Call toll-free: 1-800-CS-BOOKS v v Fax: (714) 821-4641 v v E-Mail:
[email protected]
