ARCHITECTURAL DESIGN ISSUES FOR WEB-BASED VIRTUAL REALITY TRAINING SYSTEMS Nuha H. El-Khalili and Ken W. Brodlie School of Computer Studies, University of Leeds, Leeds LS2 9JT, United Kingdom E-mail:
[email protected],
[email protected]
KEYWORDS: Simulators, Health care, VRML, Java, Physically-based modeling.
In the surgical eld the demand for new training methods has increased in order to cope with the development of surgical technology. Virtual reality technology o ers great potential in the training simulation eld. This paper presents a surgical training system which utilises both virtual reality technology and the advantage of world wide accessibility o ered by the web. The requirements of such a system are presented and an architectural design to achieve these requirements is proposed.
1. INTRODUCTION
are given in the last section.
Virtual Reality (VR) o ers great potential as a technology for computer- based training and simulation. The realistic visual environment that VR o ers, together with its intuitive forms of interaction, makes it attractive for a variety of training applications. However, VR is typically associated with dedicated powerful graphical workstations, and specialised input/ output devices. Cost is therefore a deterrent. Meanwhile the explosion of the World Wide Web (WWW) as an environment for the exchange of information has had a key in uence on software development: there has been a growth of web based applications, where the user interface is provided via a web browser (from anywhere on the Internet), and the application resides on a server under control of the application provider. In this paper we propose an architecture for a webbased VR training system. In order to ensure successful transfer of skills in a training system, the system must mimic the real procedure, within the limits of the technology being used. Hence, issues such as responsiveness, quality and accuracy of the virtual environment, and realism of user interface all need to be considered. Furthermore, we will present the special requirements of a webbased application. In our web- based VR training system, we have embedded the application within the client-server nature of the web environment. The paper presents the di erent options of implementing this theme and the feasibility of each option. This work is part of a project carried out in conjunction with St. James's University Hospital in Leeds to build a surgical training system for endovascular procedures used to treat abdominal aortic aneurysms (AAA). The next section of the paper will introduce the application motivating this work. Section three will discuss the requirements of a web-based VR training system. Section four presents the proposed architectural design and some implementation issues. Conclusion and future work
2. APPLICATION As surgical technology advances, the traditional training methods become inecient and unsuitable. The demand for new surgical training methods arises from an even heavier demand of increasing the health care quality. In an attempt towards answering this demand, we are investigating the development of a surgical training system for endovascular procedures, which are used to treat AAA. In this section we give a general medical background of the required system. Then we outline our solution.
2.1. Abdominal Aortic Aneurysms
Aortic aneurysm is a disease which causes a balloonlike swelling in the walls of the aortic artery. The disease can rupture or compress the blood vessel causing internal bleeding or lack of blood supply to other organs. Figure 1 shows a picture of a diseased artery.
Figure 1 Abdominal Aortic Aneurysm.
An aortic aneurysm can be treated by performing an open surgery that places a metallic clip which excludes the aneurysm cavity from the blood circulation. However, with the advancement of surgical techniques, it is now possible to perform this operation using endoscopy technology- an endovascular operation. This consequently means that an endoscope- a tubular device which allows the exploration of hollow anatomical regions- is introduced into the body from a small incision through the femoral artery. The endoscope is navigated through the arteries to the location
of the aneurysm and a metal stent is deployed to exclude the aneurysm from the blood circulation [ Gla]. Figure 2 shows the stent deployment process in an AAA endovascular procedure.
Figure 2 Stent deployment procedure.
The advantage of using endoscopy rather than the traditional open surgery method comes from the nature of the endoscopic technique, being a minimally invasive method. Hence, the recovery time of the patient is less and the cost is reduced.
2.2. Training
The same factors which make endoscopy a successful surgical technique in terms of health care, make it a dif cult technique in terms of learning and mastering. The attraction of endoscopic techniques is the fact that the surgeon does not have to aggressively cut tissues to expose the area of interest. On the contrary all work is done from a distance through a small hole in the body. This means that the surgeon can no longer see or feel the operational area. The direct vision sensation available in an open surgery has been replaced by a video monitor. Similarly, the direct touch and feel sensation has been migrated to the end of a long endoscope. Hence, for endoscopic procedures, the surgeon needs to aquire two new skills not required in an open surgery procedure. The rst is eye-hand coordination to allow them to navigate the endoscope and receive the visual feedback of their movement through the monitor. The second is to develop a feel sensation at the far end of the endoscope rather than the direct touch/ feel sensation. Currently, training surgeons on endoscopic procedures is done by one-to-one mentoring with an expert. Animals, cadavers and plastic models have been used also to give a preliminary but not accurate model of the real situation. These techniques are very expensive and time consuming. In addition, training on real patients is hazard prone. What
we need is a systematic safe method of training surgeons on endovascular procedures.
2.3. Outline of the Solution
Computer- based simulation is an arti cial environment, mainly computer generated, which represents a real situation in such a manner to allow the transfer of training skills [ Jac93]. Using a computer- based simulation to create a surgical training system seems to be the solution to our problem. It will provide an environment where the surgeon can repeatedly practice the procedure without endangering patients. It can also form a rich environment which exposes them to a variety of cases and possible complications. This solution also lends itself to a systematic way of training and assessing the performance of the trainee. In addition a computer- based simulation system would be a cost e ective solution. One of the major issues in simulation is the delity of the system. Fidelity refers to the combination of accuracy and realism. The need for accuracy or realism is determined by the type of the tasks performed in the simulation system. Tasks are classi ed into four classes according to the amount of mental e ort required in performing the task. Procedural, psychomotor, cognitive and perceptual is an ascending order of the task types from a low to high mental requirement [ Jac93]. Analysing the tasks involved in the endovascular procedure, we can divide the whole process into two main tasks. The rst task is navigation of the endoscope in the blood vessels. This task is a psychomotor task, which requires the user to perform a motor action based on mental interpretation of the current situation and the goal. Highly tortuous vessels and diving into narrow branches are examples of diculties encountered during this task. The second main task is the stent deployment. This task involves two types of sub-tasks. The rst are procedural-type sub-tasks which include the measurement of the aneurysm nick, checking the deployment position, then the actual deployment. The second sub-task is a perceptual task, where the surgeon decides whether to deploy the stent or not, and where to deploy it. This decision is taken based on the previous procedural task. During this task the surgeon has to make sure enough distance exists for the stent to deploy and that it will not block branches. According to Jackson [ Jac93] a procedural task requires accuracy, while psychomotor and perceptual tasks demand realism. Hence, for an endoscopic simulation system both realism and accuracy are a necessity to guarantee positive transfer of training skills through the simulator. Recently, virtual reality (VR) has demonstrated its potential in the computer- based simulation eld. The reason for that is that it o ers the most important factors needed in a simulator: realism and interaction. The philosophy behind VR technology is to give the illusion of immersion in an environment mainly computer-generated and maybe augmented with reality. This sense of immersion is thought to be accomplished by creating a closed loop of interaction
between the user and the world such that it is done in an intuitive and realistic manner and as fast as possible. Hence come the special input/ output peripherals introduced in VR applications and the demand for high computational power machines. Nevertheless, VR has not reached maturity in terms of providing natural and accurate ways of interaction. With these two limitations of VR technology in mind, we should design our simulator to accomodate and maximise the utility of current VR technology. Taking our solution one step further, it would be useful to make the endoscopy simlator accessible from di erent parts of the world, such that trainees can download it and practice some cases or even pre-practice a speci c case, which they will perform the next morning. Furthermore, one way of providing the assessment feature of the simulator, is to allow for an expert/ student scenario, where the expert can give guidelines and carry out evaluation. A web-based application will provide world-wide accessibility and multi-user collaboration. Hence, our target is to build a web-based VR training system for endovascular procedures used to treat AAA.
2.4. Other Endovascular Training Systems
There are other attempts to answer the demand of training systems for endovascular operations. On the high end scale of surgical simulators lies High Techsplanations (HT) medical simulator. HT took a general approach to tackle this area by developing a toolkit that can be used to develop any surgical simulator without the need for programming knowledge. Their toolkit (Teleos) consists of two applications. The rst, the simulator designer, is used to specify the required anatomy and the tools, and to describe the properties of the simulator. The second part- the simulator engine- is used to interact with the developed simulator. The toolkit supports input/ output devices that can track the tool position and the orientation and provide tactile feedback. This toolkit was used to develop an interventional radiology simulator, which gives the physician the choice of treating vessel blockage using either angioplasty, or drug treatment. Also, the application simulated some procedural complications due to user error [ Meg96]. The drawback of this solution is its cost. It requires high performance machines and special input/ output devices. Work done by CieMed has produced a simulator for interventional radiology procedures called daVinci. DaVinci is based on models from the visible human database. It simulates the behaviour of the endoscope tip using nite element methods. The simulator produces uoroscopic images to show the current position of the endoscope [ Cie]. Another training system for endovascular procedures used to treat abdominal aortic aneurysm is being developed by Utrecht University. The system delivers uoroscopic and digital subtracted images (DSA) similar to the ones used in the real procedure from pre-operative CT data of the patient. It also simulates deformation of the anatomy due to breathing using 2D warping of the uoroscopic images [ vWZea97].
The accessbility feature of our web-based training system should ensure its wide usefulness for a greater number of users.
3. REQUIREMENTS When we are dealing with a surgical simulation system accuracy is a must. There are two aspects of accuracy that should be considered. First, accuracy of the modelled world. In the endovascular simulator we are considering, this means the accuracy of the vessel models in terms of dimensions and details. An aortic model which misses the representation of a renal branch is of no use to the surgeon when positioning the stent. Furthermore, a model which is not as accurate as the real case, may have a wrong aneurysm nick size and lumen size, which would make it impossible to perform the task. Accuracy of the endoscope model would include its behaviour and features. Endoscopic tools do not only di er in their diameter and length, but also in sti ness, torsion and sliding friction. Hence the modelling of such a device should take these features into consideration to be able to achieve the correct behaviour. The other aspect of system accuracy is to be able to handle extreme and unexpected situations correctly. An endoscope model which behaves accurately, but looks like a collection of jagged lines is not a convincing model. Here the importance of the realism factor comes into play. Realism is a very subjective issue, but it can be divided into three parts. Visual realism refers to the plausibility of the environment representation. Issues such as shading, texture mapping and jaggedness are all factors that a ect the visual realism of a model. Behaviour realism is another side of the system's realism. Objects that suddenly appear and disappear is an example of ungraceful behaviour which a ects our belief in the modelled environment. Finally natural ways of interaction can a ect our perception of the system realism. Not only do methods of interaction need to be intuitive, but also they need to form a closed loop. It is essential that the users of the system receive some form of feedback to their action. Even more, the form of feedback needs preferably to be as close as possible to the real situation. Again, a natural way of communicating with the system that takes a few hours to respond is not acceptable. Hence, the response time of the input/output model is essential. Response time of the input/output model is one factor which a ects the responsiveness of the system. Other factors like the delay in the computational or rendering modules are important too. In a web-based system the network delay adds another factor to system responsiveness, especially in the case where the client needs to frequently go back to the host machine for data or update. This issue will be further discussed later. All the above factors seem to encourage high end machines with high graphical performance, special input/output peripherals and dedicated network lines. But
realistically, this situation cannot be a orded by many users. Looking at world wide web users, we nd a vast range of machines from a low end PC to a high end dedicated graphical workstation. Input/ Output devices range from a 2D mouse to a customised force feedback sensor tool which looks like a real endoscope. This situation calls for two more requirements for a simulator system. The system needs to be platform independent and scalable. Hence, anything that is platform dependent or requires special performance con gurations should be either avoided or kept on the host machine. In addition, the client side should be adaptable to all ranges of computation power and input/output devices.
4. PROPOSED SOLUTION 4.1. Design
Now that we have speci ed the requirements of the endovascular simulator we need to design the system under the client-server theme of the web. There are two possible con gurations of this system. In one case- option A- the server job is merely to provide the simulator. Once the client has downloaded the application no further communication is needed with the server. All the computation and rendering work is done on the client side. This con guration is favourable because it eliminates the network latency from the list of system responsiveness features. It also relieves the server from any extra work. Hence, the performance of the server will be independent of the number of users requesting the application. The second option- option B- is to distribute the simulation process between the client and the server. One advantage of this con guration is that any module that requires high computational power and/or special libraries is kept on the server where it can be handled eciently. Meanwhile modules that are likely to be user-dependent are downloaded to the client. The second option seems to successfully achieve our scalability and platform independence requirements. However, a major drawback to this method is that it requires a continuous connection between the server and the client, hence introducing network latency to the system. On a dedicated network, this would not have been a problem, but because we are considering the internet as our network, we need to look closely to this issue. When applying option B of the client-server con guration to the endovascular simulator, it is intuitive to have the input/output interface and the 3D modelling of the blood vessels and the endoscope on the client side. Meanwhile, the actual computation of the endoscope behaviour as a response to a user action is done on the server side (we assume that the vessels are rigid). Thus, it is the behaviour of the objects that is calculated on the server and is exchanged over the network. It seems sensible to reduce the amount of information exchanged over the net which would consequently reduce the network lag. In [ Roe] Roehl classi es objects in a virtual world
according to their behaviour into static, animated, Newtonian and intelligent objects. The behaviour of objects are also divided into four levels. The top level is restricted to intelligent objects, where a decision is taken concerning a number of options. The second level is to perform a task. This task is broken down into a set of actions in the next level. The lowest level is the change of object attributes (e.g. position, velocity) to accomplish the action. Taking into account the need to reduce the size of the exchanged information, we should tend to draw the client-server interface somewhere among the top levels. In our application, the blood vessels can be described as a static object, which makes it easily placed at the client side. Meanwhile, the endoscope is a Newtonian object. The rst level of its behaviour is actually the user interaction, which obviously has to be on the client side. These user interactions are translated into a set of commands which form the second layer of behaviour. It seems feasible to draw the server-client line at that level. In this case the client will send commands such as 'push/ pull endoscope' in conjunction with some attributes needed such as the applied force. In this case we were able to minimise the exchanged data. However, looking at the other direction of data exchangethat is from the server to the client- it appears that the server will have to send data of the lowest level- new attributes of the object. How severe is this disadvantage will depend on the size and type of these attributes. As will be discussed later in the next section option A is not a feasible solution under the current available techonology. Hence, for our endovascular training system we have implemented option B of the client-server con guration.
4.2. Implementation
The endovascular simulator consists of the modeling, input/ output interface and the simulation modules. Modeling of the blood vessels and the endoscope was implemented in VRML 2.0. Since VRML 2.0 [ VRM] is the emerging ISO standard of representing 3D worlds on the web, utilising it will achieve platform independence. Currently, this is not strictly the case, since there are many di erent browsers which do not all implement all the features of VRML speci cation. Another advantage of VRML 2.0 is that it allows the presentation of dynamic worlds. Objects in the virtual world can acquire simple or complex behaviours. One way of representing animated behaviour in VRML is using engines and interpolators. Script nodes can be used to represent more complicated bahaviour. Script nodes can be written in JavaScript or Java. This can be used to implement option A of the client- server con guration, where the whole process runs on the client side. Another option is to use the External Authoring Interface (EAI) to connect the VRML world to a Java applet which is embedded in the application's HTML page and runs alongside. Using this interface the Java applet can send `eventIns' to the VRML world a ecting the state of its objects and receive
`eventOuts' of the latest updates in the world [ EAI]. This method was used to implement option B of the con guration. The EAI is not formally part of the VRML ISO standard, but there is an intention to incorporate it later. The Newtonian behaviour of the endoscope is calculated based on physically- based modelling using the Lagrangian equation of motion. The endoscope is represented by a collection of mass and spring elements [ Wit, GOT97]. External forces applied by the user (push, pull and twist) and internal forces are computed. There are two types of internal forces that a ect the endoscope behaviour. Spring forces are generated when the intermediate springs experience compression or expansion. Meanwhile, collision forces a ect the part of the endoscope which collides with the blood vessel walls. VRML 2.0 only detects collision between the avatar- the representation of the virtual viewer in the world- and the objects of the virtual world. However, we need to be able to detect collision between the endoscope model and the vessel model. Thus we have used the V-COLLIDE C library to build `Oriented Bounding Boxes' (OOB) trees for both models. The OOB trees are traversed to check for object collision. The OOB structure can still be used if the object undergoes some transformations [ Hea]. However, since the endoscope model experiences deformation rather than transformation, the OOB tree for the endoscope should be rebuilt every time the model changes. However, because the endoscope model was kept simple, the overhead of doing that is tolerable. The Runge-Kutta integration method is used to integrate the equations of motion over time. Under option A of the design, a Java numerical library would be needed to implement the simulation. Because such a library does not exist yet (at least not one including ecient and robust ODE solvers) and because the suitability of Java to perform numerical calculations is still questionable (debatable), this option is not feasible for implementation at the current time. Alternatively, following option B, the NAG C numerical library is used to solve the di erential equations. Native methods are used to allow the linking of libraries in other languages and to exchange data between the Java code and the other languages' code. Because of security features, Java applets are prohibited from including native methods [ jav]. To get around this restriction, the serverwhich is a Java application- will include the native methods which agrees with option B that we have adopted. Figure 3 shows the architectural design of the endovascular training system.
Figure 3 Architectural design of the system.
On the client side, the input/ output interface consists of two VRML worlds and the Java applet running the EAI. One VRML world shows an endoscopic view of the blood vessel- i.e placing the view point inside the vessel. This local view should assist the surgeon when positioning the stent. The second VRML world shows a global view of the vessel with the endoscope model at its current position. The Java applet is currently used to apply forces to the endoscope. The user can push or pull the endoscope with a certain force value using a scroll bar scale. We plan to change to a more intuitive tool embedded in the global VRML world. In addition, the Java applet serves as the communicator between the two VRML worlds. As the endoscope position changes as a result of user interaction or interaction with the blood vessel, the global view VRML world will update the endoscope position. Consequently, the viewpoint position of the local VRML world will change to show the endoscopic view from the current endoscope position. Figure 4 shows the client interface.
Figure 4 Client Interface. 4.3. Results
A prototype of the endovascular simulator was implemented on an SGI O2 machine running IRIX 6.3. The user interface to the networked application was provided via Netscape 3.01 with CosmoPlayer as VRML plug-in. A spiral CT (Computer Tomography) data set was obtained
from St. James's University Hospital. The data was interpolated to give 2mm slice spacing. Segmentation was applied to obtain the geometry of the aorta. The segmentation was performed using a 3D deformable model which employs a self optimising mesh structure to adapt the resolution of the model to t the requirements of the image data and model the branching structure of the vessel [ BE96]. When running the simulator, there is an initial preprocessing step where the OOB trees for both the vessel and the endoscope are built (the number of triangles in the vessel is approximately 3000, the number in the endoscope 24). Every calculation step, the VRML time sensor will send a time step update message that will cause the client side to read the current applied user force, then send the force with a time step command to the server. On the server side, internal and external forces acting on the endoscope are accumulated. This step includes the calculation of damped spring forces and forces from collision between the endoscope and the vessel. Ordinary di erential equations are used to calculate the next position of the endoscope. Then the OOB structure of the endoscope is rebuilt according to the new positions. To be able to achieve real time computation, while preserving the realism of the endoscope model, we reduce the endoscope model to a 1D curve when calculating its behaviour, while the full 3D indexed face set model is used in collision detection and rendering. The server calculation loop takes from 5 to 10 msec to perform.
5. CONCLUSION We have presented an architectural design for an endovascular training system running on the World Wide Web. The architecture has taken into account the special needs of a web-based application (scalability and accessibility). We have argued two options of the client-server con guration and suggested keeping the simulation module of the training system on the server side to achieve platform independence and scalability. The simulation module of the endoscope is currently responding to user pull/push interactions and collision detection with the blood vessels. This needs to be extended to handle the user twist interactions. Physical properties of the di erent endoscope types (e.g. torsion and bending) needs to be incorporated into the model to be able to simulate various endoscope types. In addition, intuitive user interactions need to be developed on the client side to allow the user to manipulate the endoscope. Finally, some form of feedback needs to be added to respond to the user applied force to close the loop of interaction. ACKNOWLEDGMENT The authorswould like to thank David Kessel from St. James's Hospital, Andy Bulpitt and Stuart Lovegrove for their cooperation.
References [BE96]
[Cie] [EAI]
Andrew J. Bulpitt and Nicholas D. E ord. An ecient 3d deformable model with a self- optimising mesh. Image and Vision Computing, 14:573{580, 1996. Ciemed: Davinci: Medical simulators. http:// beast.cbmv.jhu.edu:8000/ daVinci/daVinci.shtml.
projects/
External authoring interface, working group. http://www.vrml.org/ WorkingGroups/vrml-eai/. [Gla] D. Glauser. Endovascular treatment technique for cerebral aneurysm. http://dmtwww.ep .ch/ imt/ robchir/ aneurysms.html. [GOT97] U. Gudukbay, B. Ozguc, and Y. Tokad. A spring force formulation for elastically deformable models. Computer and Graphics, 21(3), 1997. [Hea] T. Hudson and et al. V-collide: Accelerated collision detection for vrml. http://www.cs.unc.edu/ geom/ collide.html. [Jac93] P. Jackson. Application of virtual reality in training simulation. In J. Gray K. Warwick and D. Roberts, editors, Virtual Reality in Engineering, pages 121{136. 1993. [jav] The java tutorial: Object-oriented programming for the internet. http://java.sun.com/ docs/books/ tutorial/index.html. [Meg96] D. Meglan. Making surgical simulation real. Computer Graphics, Nov, 1996. http://www.ht.com/ htweb/ Test.html. [Roe] Bernie Roehl. Some thoughts on behaviour in vr systems. http://ece.uwaterloo.ca/ broehl/ behav.html. [VRM] The virtual reality modeling language speci cation. version 2.0. http://vag.vrml.org/ VRML2.0/ FINAL/ spec/index.html. [vWZea97] Teo van Walsam, Karel Zuiderveld, and et al. Ct-based simulation of uoroscopy and dsa for endovascular surgery training. In Proceedings CVRMed- MRCAS'97, volume 1205. SpringVerlag, 1997. [Wit] Andrew Witkin. An introduction to physically based modeling. http://www.cs.cmu.edu/ afs/cs/ user/ bara / www/pbm/ pbm.html.
