Photorealistic Rendering of the Visible Human Karl Heinz Hohne, Thomas Schiemann, Ulf Tiede Institute of Mathematics and Computer Science in Medicine (IMDM), University Hospital Hamburg-Eppendorf, Germany e-mail:
[email protected]
Abstract. In principle the Visible Human data sets are an ideal basis for building electronic at-
lases. While it is easy to construct such atlases by just o ering the possibility of browsing through the 2D slices, the construction of realistic 3D models is a huge project. As one rather easy way to establish three-dimensionality, we have registered the Visible Human data to the already existing 3D atlas VOXEL-MAN/brain. The real challenge, however, is the construction of a realistic 3D model. This requires segmentation methods which are taylored for the subsequent visualization procedure. We have developed an interactive classi cation method that describes objects as ellipsoids in RGB-space. Super-sampling and texture mapping based on this classi cation deliver photorealistic perspective views.
Introduction In our previous work we have developed a framework for generating volume based 3D interactive atlases. These atlases are based on a two layer model [7]: Images volumes, usually obtained from radiology, and congruent label volumes for di erent domains of knowledge like morphology or physiology form the spatial part of the model. It is linked to a semantic network containing descriptive knowledge about the objects. For extraction of the model's contents there is a large set of visualization, exploration, and simulation tools. With this framework the VOXEL-MAN atlases of brain and skull have been completed [1] and atlases of other parts of the body are under development [3]. Their pictorial basis are radiologic cross-sectional images. While this has the advantage that anatomy can be linked nicely to radiological imaging, anatomical detail is subject to improvement. The high resolution data sets of the Visible Human project [8] are therefore an ideal basis for this purpose. Typically the Visible Human images are presented in stacks of orthogonal slices, which can be browsed through. Compared to such simple systems, creation of a real 3D model requires detailed segmentation, which is a huge project due to the vast amount of data and anatomical detail. In our rst approach we overcome this problem by registration of the Visible Human with the existing volume based atlas VOXELMAN/brain. This procedure introduces anatomical detail from the Visible Human into the radiological environment of VOXEL-MAN/brain, and on the other hand the detailed labelling of VOXEL-MAN can be correlated with the Visible Human. While this procedure gives a new dimension of functionality to the existing atlas, 3D display of the Visible Human is not possible in this way. Thus we have done rst steps in segmentation of some portions of the huge data set, which can thus be visualized in high quality with greatly improved realism. This procedure will be described in the second part.
Registration of the Visible Human to VOXEL-MAN Registration of the anatomical Visible Human images to the radiological images of the VOXEL-MAN atlases is established by a linear mapping function, which is speci ed in an interactive environment [5]. The speci cation uses two di erent steps of interaction:
Fig. 1. Correlation of slices from the Visible Human and VOXEL-MAN/brain after registration has been per-
formed: The VOXEL-MAN slice (left) shows semantic regions of the volume model (morphological objects on the right and blood supply areas on the left). The Visible Human slice adds pictorial anatomical detail to the context.
1. A set of landmarks (points and planes) is speci ed for a coarse mapping. For registration of the head region we use the bicommisurial coordinate system of Talairach [9]. For other body regions this can be generalized to any pair of characteristic points on an orthogonal plane. 2. For a re ned registration, surface representations of structures from VOXEL-MAN are rst transformed according to the speci ed landmarks, and can then interactively be registered on the Visible Human images in any linear manner (rotation, translation, scaling): Wire frames can be modi ed on perspective images and contours are manipulated on slices. Due to the linear mapping function and the fact, that two di erent individuals have to be registered, this procedure cannot create a registration with a precision in the order of a few voxels. Nevertheless the procedure results in highly correlated image volumes, enabling a combination of the advantages, which each of them has: The VOXEL-MAN volume has been segmented in great detail o ering information about morphology, blood supply, and function, while the anatomical images of the Visible Human present anatomical realism, which cannot be obtained from radiological procedures. Thus arbitrary slices of each of the volumes can be viewed side by side with one of them showing anatomical realism and the other one giving knowledge from the volume model (Fig. 1). In addition the Visible Human slices can also be shown in the 3D context of the perspective images.
Fig. 2. Frontal view of the left kidney: Several cut planes are speci ed in order to unveil the left kidney and the major vascular structures. Ribs, blood vessels and some muscles are excluded from cutting. The cut planes show the colors of the anatomical volume
Segmentation for high quality rendering High quality rendering of volume data requires supersampling of the data volume on basis of an analytical description of the segmentation result. In case of scalar image volumes this is a labelled volume in combination with threshold ranges for every object. In case of the anatomical images of the Visible Human the threshold range has to be generalized to a region in RGB feature space. In order to keep computing times during rendering as low as possible, the selected description should be easy to formalize. We have thus decided to classify the regions by ellipsoids. The segmentation is adapted from an interactive method, which has proven to be successful for many purposes with scalar images like CT or MRI [2, 4]. The procedure is based on thresholding followed by binary mathematical morphology and connected component labelling. While the latter two steps remain unchanged for color images, the thresholding step is generalized to speci cation of an ellipsoid in RGBspace:
Fig. 3. Reconstruction of muscular and vascular structures of the head. 1. The users outlines a (usually small) region of the object, which shall be segmented. This results in a set of RGB-triples (RGB )i . 2. The median rgb-triple RGB and the median d of all distances d(RGB ? (RGB )i ) are computed. 3. For determination of the ellipsoid only those triples are further regarded, which are closer to RGB than an interactively speci ed multiple of the median d : The center is taken from the median of the considered sample set, the axes directions are computed from covariance analysis and their length is determined such, that all considered triples are inside the ellipsoid. A more detailed description of the method can be found in [6]. Instead of this basic ellipsoidal speci cation, the segmentation step could also be performed with much more sophisticated classi cation algorithms. However, these do usually not result in easy to formalize descriptions, which we need during ray-casting. The proposed segmentation procedure is successful for all major constituents of the body like brain or abdominal organs. For a detailed subdivision of these structures, e.g. into gyri of the brain, manual segmentation is still needed due to a lack of image properties for this purpose.
Figure 2 shows a rendering of structures of the upper abdomen, which have all been segmented with the described semi-automatic procedure. The bone has been segmented from the CT images, which come along with the anatomical images and which have undergone further registration [5]. Except for bone and vascular structures, all organs are visualized according to their colors in the anatomical images. Bone colors are weighted with an arti cial color in order to enhance bone appearance. Arteries and veins are visualized in pure arti cial colors (red/blue) as usual on anatomical images. Figures 3 and 4 show views of the head and the right shoulder. The chosen visualization techniques are the same as described for the abdominal image. Further 3D images of the head and the abdomen are shown in [6] and [10].
Fig. 4. View of the right shoulder with bones, musculature and some major veins.
Conclusions We have proposed two di erent approaches for use of the Visible Human images for electronic anatomy atlases. The registration of the Visible Human with VOXEL-MAN/brain o ers a new dimension for interpretation of anatomy obtained from radiology. This functionality is ready for use without limitations. In our second approach the described rst steps in segmentation for high resolution volume rendering show, that the Visible Human data set represents a new quality of anatomical imaging, when used in a state-of-the-art visualization system. Nevertheless, a huge amount of work still has to be done for detailed segmentation. But unlike clinical imaging, which requires new segmentation for every case, segmentation of the Visible Human has to be done only once. If this work is completed, the results will surely have great impact on many applications of visualization such as education and simulation.
Latest results of ongoing research can be found at http://www.uke.uni-hamburg.de/Institutes/IMDM/IDV/VisibleHuman.html
Acknowledgements We are grateful to all members of our department, who have supported this work. Special thanks to Rainer Schubert for anatomical and Uwe Pichlmeier for statistical support.
References 1. Hohne, K. H. (Ed.): VOXEL-MAN, Part 1: Brain and Skull, Version 1.0. Springer-Verlag Electronic Media, Heidelberg, 1995. (ISBN 3-540-14517-6). 2. Hohne, K. H., Hanson, W. A.: Interactive 3D-segmentation of MRI and CT volumes using morphological operations. J. Comput. Assist. Tomogr. 16, 2 (1992), 285{294. 3. Hohne, K. H., P esser, B., Pommert, A., Riemer, M., Schiemann, T., Schubert, R., Tiede, U.: A new representation of knowledge concerning human anatomy and function. Nature Med. 1, 6 (1995), 506{511. 4. Schiemann, T., Bomans, M., Tiede, U., Hohne, K. H.: Interactive 3D-segmentation. In Robb, R. A. (Ed.): Visualization in Biomedical Computing II, Proc. SPIE 1808. Chapel Hill, NC, 1992, 376{383. 5. Schiemann, T., Hohne, K. H., Koch, C., Pommert, A., Riemer, M., Schubert, R., Tiede, U.: Interpretation of tomographic images using automatic atlas lookup. In Robb, R. A. (Ed.): Visualization in Biomedical Computing 1994, Proc. SPIE 2359. Rochester, MN, 1994, 457{465. 6. Schiemann, T., Nuthmann, J., Tiede, U., Hohne, K. H.: Segmentation of the Visible Human for high quality volume based visualization. In Hohne, K. H., Kikinis, R. (Eds.): Visualization in Biomedical Computing, Proc. VBC '96, Lecture Notes in Computer Science 1131, Springer-Verlag, Berlin, 1996, 13{22. 7. Schubert, R., Hohne, K. H., Pommert, A., Riemer, M., Schiemann, T., Tiede, U.: Spatial knowledge representation for visualization of human anatomy and function. In Barrett, H. H., Gmitro, A. F. (Eds.): Information Processing in Medical Imaging, Proc. IPMI '93, Lecture Notes in Computer Science 687, Springer-Verlag, Berlin, 1993, 168{181. 8. Spitzer, V., Ackerman, M. J., Scherzinger, A. L., Whitlock, D.: The Visible Human male: A technical report. J. Am. Med. Inf. Ass. 3, 2 (1996), 118{130. 9. Talairach, J., Tornoux, P.: Co-Planar Stereotactic Atlas of the Human Brain. Georg Thieme Verlag, Stuttgart, 1988. 10. Tiede, U., Schiemann, T., Hohne, K. H.: Visualizing the Visible Human. IEEE Comput. Graphics Appl. 16, 1 (1996), 7{9.
