MMT Medical. Prof. PAUL SHARKEY ... (Paul Neumann et al) [2]. It was hoped that .... [2] Neumann, P., Parshall, R., Sadler, L. (1994) The Virtual Eye. NSF, DoE.
Virtual Reality and Interactive 3D as effective tools for Medical Training GEORGE WEBB ALEX NORCLIFFE PETER CANNINGS MMT Medical Prof. PAUL SHARKEY Dr. DAVE ROBERTS University of Reading
Abstract. CAVE-like displays allow a user to walk in to a virtual environment, and use natural movement to change the viewpoint of virtual objects which they can manipulate with a hand held device. This maps well to many surgical procedures offering strong potential for training and planning. These devices may be ne tworked together allowing geographically remote users to share the interactive experience. This maps to the strong need for distance training and planning of surgeons. Our paper shows how the properties of a CAVE-Like facility can be maximised in order to provide an ideal environment for medical training. The implementation of a large 3D-eye is described. The resulting application is that of an eye that can be manipulated and examined by trainee medics under the guidance of a medical expert. The progression and effects of different ailments can be illustrated and corrective procedures, demonstrated.
Background In 1994 the Virtual Reality Room at SIGGRAPH [1] showcased technologies that made use of advanced 3D displays. Some of the applications on display focused on medical training and simulation. One that was particularly relevant to this paper was the 3D Virtual Eye (Paul Neumann et al) [2]. It was hoped that the application would allow medical students to study the eye's geometry and allow them to simulate common pre-surgical procedures. In 1997 a project called The Virtual Temporal Bone was set out with similar aims (Alan Millman et all) [3]. Other projects, such as The Virtual Laboratory, and related papers, also make use of immersive virtual environments yet they take a slightly different approach. They make use of real data collected from various sources (including MRI and CT scanners). They then try to predict the effects that different operating techniques have upon this data. An example of this is the effect that different vascular reconstruction techniques have on blood flow. In contrast to the work described in this paper “visual realism was not the primary goal” (Bellman and Sloot, 2001) [4]. Our work builds upon that of previous projects through the support for distance collaborative training and planning. This is combined with greater realism and naturalness
of interaction. We have linked CAVE-Like devices in UK and Austria to support shared presentation and interaction, both with the medical model and virtually embodied users.
Methods We discuss both the reparation and collaborative use of our model. For the creation of accurate 3D models a number of graphical sources and sometimes plastic representations are used. The actual modeling itself is done using Maya, a market leading 3D package. In order to maintain real- time performance there is always a fine balance between the quality of finished models and the number of polygons used to create them. Collaborative use of our application is achieved through linking display devices varying from immersive CAVElike facilities to desktop computers. These devices are linked using the DIVE Collaborative Virtual Environment (CVE) (Carlson and Hagsand 1993) [5]. Support for immersive displays uses the Spelunk extension to DIVE (Steed, 2001) [6]. VRML is used as an interchange language between the modeling software and the CVE. The two images below show the virtual eye in different states of development. The first image shows the eye being used in an immersive display. Up to five people can fit into the display at a time though many others could be connected through a network. The cornea and lense sections of the eye are not displayed due to problems with transparency that we were experiencing at the time. The second image shows a higher quality view of the eye that was not rendered in real-time. The advantage of this is that it allows for the rendering techniques of bump mapping and ray-tracing. Such techniques help to provide more realistic looking images.
1. The virtual eye being demonstrated in a CAVE- 2. A high quality render of the inside of the virtual eye Like facility
Results Results to date include using the 3D eye as focus for interaction between immersive users in the UK and Austria. The eye is about 4ft tall and users can easily get inside it as well as pick it up, turn it around or even look in or out of it through the pupil. The inside of the eye is coated with photo-realistic textures whereas the outside of the eye uses realistic procedural textures created within Maya. Interactivity was provided at each end of the network, thus allowing for collaboration and instruction. Remote users are shown as virtual avatars whereas users grouped within a single cave are visible to each other. Natural communication is supported through audio and free gesturing. So far the users can toggle
between viewing a whole eye or a cross-section of the eye and the effects of different diseases can be demonstrated. The main advantages of viewing the eye in the immersive display include the ability to walk around the eye (or inside it) as well as the ability to manipulate the eye in a much more natural way. This is achieved by the tracking of the the movements of the user’s head and hand.
Conclusion The importing and networking of the 3D eye into desktop and immersive displays has been successful. Further work focuses on naturalness of behavior and interaction as well as the incorporation of other media. Interactive behavior may include tissue and fluid modeling and the feeling of touch. Other media under investigation include haptic devices and video. The inclusion of animated simulations are also being developed, these could be of the progression of different diseases (such as glaucoma) or of the surgical procedures that are needed to combat such illnesses. Where surgical procedures are simulated, they can then be practiced by trainee medics and feedback can be given relating to accuracy. The CAVELike facility provides an excellent environment in which to practice such techniques, it allows the users to walk around, select and touch the different models in the display. It also allows the models to be viewed at many times their normal size resulting in a very high level of detail.
Novelty/Discussion The prime novelty of this work lies in the sharing of natural interaction with a life-like medical model. This, we hope, will provide unprecedented levels of realism for distance collaboration in medicine, both for training and planning.
Acknowledge ments Thanks to Oliver Otto and Robin Wolff for their expertise in CAVE-Like facilities.
References [1] Petrovich L., Tanaka, K., Morse, D., Ingle, N., Morie, J., Stapleton, C., Brown, M."SIGGRAPH '94 Visual Proceedings" Computer Graphics Annual Conference Series 1994, Orlando, FL, 07/01/94-07/01/94. [2] Neumann, P., Parshall, R., Sadler, L. (1994) The Virtual Eye. NSF, DoE. [3] Millman, A., Mason, T., Rasmussen, M., Evenhouse, R., Warren, L., Lowenthal, N., McElroy, J., Seelaus, R., Challender, J., Applebaum, E., Panko W, (1997) The Virtual Temporal Bone (ongoing) [4] Belleman, R. G., Sloot, P.M.A. (2001) Simulated vascular reconstruction in a virtual operating theatre. CARS 2001 Conference (CARS2001), Berlin, Germany, June. [5] Carlsson, C., & Hagsand, O. (1993). DIVE - A platform for multi-user virtual environments. Computers & Graphics, 17(6), 663-669. [6] Steed, A., Mortensen, J., & E., F. & Frecon E., (2001). Spelunking: Experiences using the DIVE System on CAVE-like Platforms.
