The Virtual Surgery Patient: Development of a digital, threedimensional model of human anatomy designed for surgical education. John Qualter, MS Advanced Educational Systems, NYU School of Medicine Email:
[email protected] Mary Ann Hopkins, M.Phil., M.D. Department of Surgery, NYU School of Medicine Email:
[email protected]
Marc M. Triola, M.D. Advanced Educational Systems, NYU School of Medicine Email:
[email protected] Miro Kirov Advanced Educational Systems, NYU School of Medicine
Matthew J. Weiner, M.D. Department of Surgery, NYU School of Medicine Martin S. Nachbar, M.D. Advanced Educational Systems, NYU School of Medicine
Abstract Teaching medical students about the anatomical principles of surgical procedures is a challenging task. In an effort to design a new tool that allows a third year medical student to explore specific surgical problems, the New York University School of Medicine Department of Surgery and the New York University School of Medicine Advanced Educational Systems lab collaborated to develop a Virtual Surgery Patient (VSP). The VSP is a realistic set of threedimensional models of human anatomy. Unlike previous applications, our objective was to create lightweight 3-D models with the inherent ability to deform smoothly upon animation, with an architecture that allowed for enhanced texturing as well as polygon reduction. We developed a technique for using reconstructed volume data from the Visible Human Project to create surface models that were easy to manipulate, avoiding artifacts introduced when resurrecting volumes from cadaveric sliced stacks. Surface shading can be used to give the models the appearance of living tissue, as well as allow for enhancements sometimes necessary to achieve an educational goal.
1. Introduction The Advanced Educational Systems Department and the Department of Surgery at New York University School of Medicine are currently developing a Virtual Surgery Patient (VSP). This project is designed to teach medical students and residents the principles and practices of surgery in a novel way. The VSP consists of a 3-D dataset of handcrafted, de novo models of human anatomy and a companion library of simulated events that can be applied to the models. The use of digitized anatomic data along with refinement by anatomists and surgeons provide for the accuracy of the VSP's construction. The VSP’s didactic utility is not limited to the visualization of static gross anatomy. Animation of organs and structures are used to demonstrate surgical and physical exam maneuvers, adding new depth to the ability of students
Proceedings of the 17th IEEE Symposium on Computer-Based Medical Systems (CBMS’04) 1063-7125/04 $ 20.00 © 2004 IEEE
to visualize anatomy. The dataset can be manipulated extensively and rendered to create high quality digital simulations of anatomic mechanics and surgical procedures.
2. Methods Under the direction of surgeons and anatomists, two experienced medical illustrators used Maya® animation software to create 3-D models with flexible polygon counts of organs in the human abdomen and thorax. The fundamental data reference used to resurrect the 3-D anatomy were the digitized photographic images of the Visible Human Project (1), acquired from the National Library of Medicine. The axial view of this cryosectioned cadaver allowed illustrators to define the correct shape of a particular organ by tracing its contours on each photograph in a selected stack. This method is commonly used for 3-D reconstruction of anatomy. Previous approaches to 3-D reconstruction have used custom or commercial applications to interpolate the contours of each slice and produce a volumetric voxel model of a particular anatomical structure. These programs are sometimes capable of also producing a polygonal surface model from the volumetric information. These models are generally very complex, making manipulation and animation deformation needed for a high-end visual rendering a cumbersome task. McInerney et. al note that this technique can result in datasets that “usually suffer from sampling artifacts, spatial aliasing, and noise, are essentially, blocks of ‘granite’ with meaningful embedded structure.”(2) Recent applications from a variety of software vendors have fine-tuned volumetric imaging, so that the end visualization is remarkably better than images generated from older render engines. However, the 3-D surface models, or “point-clouds” derived from these heavy datasets, are still large and complex due to their high numbers of polygons, which are the basic elements that bind the mesh of the model. We needed an approach that would generate models with a lower polygon count. Anatomic models created with lower polygon counts are much more desirable for our multimedia purposes as they can easily be deformed or animated to simulate motion, or physiologic processes. Also, our real-time 3-D graphics engines require lower polygon counts to properly display the 3-D geometry on computers not equipped with high-end video cards. To accomplish the task of acquiring versatile anatomic models with controllable polygon counts, Maya® was deployed for resurfacing the geometry of the datasets extracted from the Visible Human Project. The first step in the creation of the simplified models is to import the point-cloud data from the Visible Human Project into the Maya® workspace for analysis. Surface curves are used to trace the structure of the complex point-cloud. The curves acted as guidelines for the creation of the new lightweight surface model. Unlike the imported point-cloud, this resurfaced model conveniently deforms as needed, and can easily handle painted texture maps applied via a shader system. (Figure 1)
Proceedings of the 17th IEEE Symposium on Computer-Based Medical Systems (CBMS’04) 1063-7125/04 $ 20.00 © 2004 IEEE
Figure 1.
Process of resurfacing adrenal glands within Maya®. The high complexity point-cloud data from the Visible Human Project is on the left and the resulting low polygon anatomic model is on the right (black background).
This approach is advantageous because the shader system has attributes that can be adjusted to better achieve the educational objectives of the animation. For example, coloring geometry in a unique way to highlight a particular organ, or enhancing visual representation by making interfering structures transparent. Cutting et. al note transparency allows the underlying anatomy to be seen during surgical repair while maintaining a view of the overlaying tissue relationships. (3) Using this approach we can artistically apply textures to the models to make them appear more like living tissue, rather than the cadaveric tissue seen in the Visible Human Project and many photographic atlases. This new dataset of surface models can be further manipulated by adding animation handles, or deformers. The current version of the VSP has deformers built into the model that simulate common surgical maneuvers. For example, mobilization of the left colon or packing the small bowel away, are functions in the VSP. (Figure 2) An animator assigned to a project involving these procedures can rapidly assemble parts of a surgery simulation, greatly reducing the production time and process. As the VSP continues to evolve, new animation handles will be added, creating an extensible tool kit for simulating a diverse range of surgical maneuvers.
Figure 2. A render of the thorax and abdomen showing manipulation of the small bowel.
3. Results The new polygon surfaces are true to the reconstructed form, and unlike the original pointcloud models, they are lightweight, and can be easily exported to a variety of computer graphics file formats. (Figure 3) In the instance of resurfacing the adrenal glands, the original point-cloud mesh was composed of 12,440 polygonal faces, where as the new model was
Proceedings of the 17th IEEE Symposium on Computer-Based Medical Systems (CBMS’04) 1063-7125/04 $ 20.00 © 2004 IEEE
reduced down to 1,150 faces. This allows animators to work with the application of their choosing. The models can be easily adjusted to bandwidth and CPU/GPU limitations by altering the detail (polygon count) of the structures. However, the real significance of these new surface models is the ability to apply deformations for animation, and to add detailed textures that greatly enhanced the visual presentation of the final product. Figure 3. Screen shot of lightweight surface models in medical animator workspace.
Figure 4. Rendered frames from an animated colon surgery
The VSP has proven to be a valuable, reusable asset in the production of rich media designed for education. (Figure 4) The reusability, functionality, and flexible architecture of the VSP will continue to make it the backbone for future surgery simulations at our institution. Without such an application, a great deal of time spent working on a simulation project would be concentrated on modeling and animating the appropriate anatomy. With the VSP, surgeons,
Proceedings of the 17th IEEE Symposium on Computer-Based Medical Systems (CBMS’04) 1063-7125/04 $ 20.00 © 2004 IEEE
computer animators, and educators can focus their attention on the specifics of the animation, and determine the best way to communicate a particular procedure to a desired audience.
4. References [1] The Visible Human Project [online]. 2001 [cited 2003 Dec 21]. Available from: URL: http://www.nlm.nih.gov/research/visible/visible_human.html [2] Deformable organisms for automatic medical image analysis. Med Image Anal. 2002 Sep; 6(3):251-66. McInerney T, Hamarneh G, Shenton M, Terzopoulos D. [3] Use of three-dimensional computer graphic animation to illustrate cleft lip and palate surgery. Comput Aided Surg. 2002; 7(6):326-31. Cutting C, Oliker A, Haring J, Dayan J, Smith D.
Proceedings of the 17th IEEE Symposium on Computer-Based Medical Systems (CBMS’04) 1063-7125/04 $ 20.00 © 2004 IEEE
