A Geometric Modeling Tool for Visualization of

A Geometric Modeling Tool for Visualization of Human Anatomical Structures 1 Jean Hsuy, David M. Chelbergy and Charles F. Babbsz ySchool of Electrical Engineering zBiomedical Engineering Center

Purdue University West Lafayette, IN 47907

Abstract A geometric modeling system for creating and visualizing models of human anatomical structures has been developed. Many structures in the human body such as blood vessels, bones, the breast, and the chest cavity can be naturally modeled using generalized cylinder and quadric primitives. These models can be accurately and eciently rendered to form simulated transmitted x-ray and re ected visible light images of human anatomy. Stereoscopic images can be easily generated to allow 3D viewing. Representation of organs by geometric models allows easy modi cation of their size, shape, and grouping relations. Rotation and viewing in arbitrary direction, and anatomical manipulations such as removal of organs to reveal details of surrounding structures, can be accomplished. Some potential applications and sample images generated by our system are presented.

1 Introduction Medical imaging and visualization have been playing an ever increasing role in the diagnosis and followup care of patients. In order to train medical personnel and to evaluate new medical display technologies, large numbers of images of known normal and abnormal states are necessary. Collection of such a large number of images from routine clinical material is not necessarily a simple task. In particular, the presence or absence of abnormalities in an image (\ground truth") has to be determined by biopsy or 1 This work was supported in part by a Purdue University Trask Fund Grant, a Digital Equipment Corporation Faculty Incentives for Excellence Grant and an Indiana Elks Fellowship. Address all correspondence to David M. Chelberg at [email protected].

by the mutual agreement of a group of board-certi ed radiologists. The lack of knowledge concerning ground truth of images is also a problem in quantifying the accuracy of results in various medical image processing research. For research purposes and for selected aspects of pre-clinical and clinical training, we propose the use of an image simulation system as an inexpensive and reproducible source of simulated x-ray images with known ground truth. The use of an image simulation system gives full control over the image formation process (e.g. source location, image resolution etc.). Stereoscopic images can be easily generated to allow viewing in three-dimensions. Furthermore, both simulated x-ray and visible light images can be generated and compared. Visible light images are especially useful for visualization of reconstruction results and for educational purposes. Most research work in the area of computer graphics in medicine [1, 2] has concentrated on volumetric [3, 4] and surface rendering from 3D data sets [5, 6]. For surface rendering, the tissue-bone or airskin boundaries have to be extracted from 3D data sets [7]. In general, surface segmentation is a dicult problem [8]. Also, surface representation does not lend itself to easy speci cation of the size or shape of anatomical structures. Volumetric rendering of 3D data sets provides accurate rendition of the models but it is computationally intensive. In addition, volumes of interest in 3D data sets cannot be easily manipulated. Although editing tools [9] for 3D data sets are helpful, editing 3D data sets is still a relatively tedious and time consuming process. We are developing a system for modeling human anatomical structures using two types of geometric primitives: quadrics and generalized cylinders [10]. A ray tracing technique is used to generate simulated x-

ray and visible light images from mathematically de ned geometric models of anatomical structures [11]. Geometric model renderers are typically more ecient than voxel-based volumetric renderers since computations are done on the object level rather than the voxel level. In the next section, the advantages of geometric modeling and how anatomical structures are modeled in our system are discussed. A brief description of our geometric model renderer is given in section 3. Although simulated images cannot replace real images for diagnosis purposes, we show that many useful bene ts can be derived when they are used for initial training and evaluation purposes. Some potential applications and sample images generated by our system can be found in section 4.

2 Geometric modeling of anatomical structures To visualize and understand the structure or behavior of an object, a simpli ed representation of the object (\model") is often used. Various components of an object are geometrically de ned and combined to form a complex and yet coherent structure. For example, each individual rib may be separately de ned, and then the set is combined as an entity to form the rib cage. Geometric modeling allows quantitative interaction with the data and makes modi cation and manipulation of an object easy. A discussion on the usefulness of geometric modeling of biological material can be found in [12]. Many geometric modeling representations have been proposed and their application to biomedical visualization and simulation is discussed in [13]. Each representation has its pros and cons and the choice of a particular representation is dictated by its intended use [14]. In our system, the Constructive Solid Geometry (CSG) method is used to represent organs by a combination of quadric and generalized cylinder primitives. Many biological structures, including human organs, are elongated in one direction or are symmetrical. Generalized cylinders can be used to naturally represent such structures. Small organ parts can also be modeled eciently using quadric objects. These primitives are easy to render and their representations are compact and easily modi able. Modi ability is an important criteria for modeling human organs since the shape of an organ changes dynamically during ful llment of a speci c function (e.g. a pumping heart) and intrinsically as a result of growth or aging. The use of geometrical models gives the ability

to easily simulate biological variation and widely vary a standard anatomy by manipulation of the geometry of parts e.g. tall people, short people etc. Standard anatomy can also be varied by adding ranges of abnormality e.g. noticeable tumor, obscure tumor etc. The use of geometric models allows easy modi cation of the size, shape, and grouping of particular structures. Rotation and viewing from an arbitrary direction can be easily accomplished. Organ manipulations such as the removal of overlying layers to reveal details of subjacent organs, are simple. In the following sections, a description of quadric and generalized cylinder primitives is given. We also brie y describe the CSG method and how it is used for sculpting anatomical structures.

2.1 Quadrics The implicit surface equation of the form: Ax2 + By2 + Cz 2 + Dx + Ey + Fz + Gxy + Hyz + Ixz + J = 0 de nes the family of quadric surfaces. Examples of quadric surfaces include the sphere, cone, cylinder, ellipsoid etc. Quadrics can be used to represent small anatomical parts and abnormalities such as tumors and microcalci cations.

2.2 Generalized cylinders A generalized cylinder [10] is a volumetric solid generated by sweeping an arbitrarily shaped closed cross section along an arbitrary three-dimensional space curve (axis) (see Figure 1). The Frenet frame [15, 16] oers a natural method to orient the cross section relative to the axis. The cross section may be varied (deformed) as it is swept along the curve. This deformation function is usually called the sweeping rule. The cross sction function is given by c(u; v) = (cn (u; v); cb(u; v)) = s(cs(v); u) where vi  v  vf and s is the sweeping rule that speci es how a particular cross-sectional contour, cs(v), changes along the axis of the generalized cylinder. The axis is given by a(u) = (ax (u); ay (u); az (u)) where ui  u  uf : Blood vessels, bones, the liver and the stomach are examples of anatomy which can be naturally represented using generalized cylinders.

2.3 Sculpting process Many anatomical structures are complex and cannot be accurately represented using a single primitive. Constructive Solid Geometry (CSG) [17, 18, 19], a

general mechanism by which volume de ning primitives can be assembled into more complex shapes, is useful for modeling complex structures. Simple primitives are combined using boolean set operators such as Union, Dierence, and Intersection. These operators tell whether a given point is in the combined solid or not. For x-ray imaging of objects, it is also necessary to know the density associated with a given object point. We have thus expanded the set of boolean operators (see Table 1).

Object 1 Object 2 Union Intersect Difference Overlap Replace-by Line style represents the density of the segment. Density of object 1: Density of object 2: Density of object 1 + Density of object 2:

Table 1: Constructive solid geometry operations for the x-ray ray tracer For the Union and Dierence operations, the density of object 1 is used as the density for the combined object. For the Intersect operation, the density of the combined object is the sum of the densities of object 1 and object 2. The Overlap and Replace-by operations are specially de ned for the case of simulated x-ray imaging. They are similar to the Union operation except that for overlapped portions of an object, the Overlap operator adds the densities of object 1 and object 2 while the Replace-by operator uses the density of object 2. Using these operators, the user can sculpt complex anatomical structures from simpler ones. The current implementation requires the speci cation of anatom-

ical structures in the form of a text le. We are currently developing a user interface that allows the user to choose from the various primitives, modify the primitives interactively, and combine them using any of the de ned boolean operators. The user can then see an approximate rendition of a combined object onscreen instead of visualizing it in his/her mind. The addition of a more sophisticated user interface will increase the user-friendliness of the system. An alternate method for specifying anatomical structures is to use reconstruction results in the form of quadrics and generalized cylinders [20]. Automatic approximation of data sets by the allowable primitive objects can also be performed [21]. However, this is a signi cant area of research in itself and is beyond the scope of this paper.

3 Geometric model renderer To generate realistic images from geometric models of organs, a ray tracing technique [17, 22] is used. A detailed description of the implementation of our geometric model renderer can be found in [11]. We will brie y describe the basic concepts in this section. The ray tracing problem can be stated as follows: Given a scene description, an eye position (x-ray source), and an image plane, generate the 2D visible light (x-ray) image that is observed when the scene is viewed from the eye (source) position. The image plane may be thought of as a rectangular grid of the desired resolution. A ray is cast from the eye (source) to a grid location on the image plane (see Figure 2). Intersection points between the ray and objects in the scene are found and used to compute the value for the grid location. The image is fully rendered when the value of each grid location on the image plane has been determined. The ray tracing technique was originally proposed as a means to generate realistic visible light images. We have extended it to generate simulated x-ray images. We will brie y discuss the similarities and dierences between traditional visible light ray tracing and x-ray ray tracing. At the heart of every ray tracing algorithm is the computation of intersection points between a ray and each allowable primitive object. Details of the computation of intersection points between a ray and quadrics can be found in [17, 22]. Intersection points between a ray and a generalized cylinder can be computed using the method described in [11]. The ray-

object intersection portion of the ray tracer is essentially the same for both the visible light ray tracer and the x-ray ray tracer. One major dierence is that for single primitive object scenes, the eciency of the visible light ray tracer can be enhanced by computing only the rst intersection point of a ray with objects in a scene ( rst-hit speed up). For scenes with objects combined by the Constructive Solid Geometry (CSG) method, all the intersection points must be computed. For the x-ray ray tracer, since x-rays can pass through opaque objects (such as the various anatomical structures), the ray-object intersection routine must compute and return an ordered list of all entering and exiting intersections. Another major dierence between the visible light ray tracer and the x-ray ray tracer is how the value for each pixel location in the image plane is computed from the ray-object intersection points. For visible light ray tracing, shading is computed based on illumination source(s) and the position, orientation, and characteristics of the rst object that is hit by the ray. In our implementation, Phong's illumination model is used. For x-ray ray tracing, shading is determined by the intensity of the x-ray that passes through objects in the scene and to reach the image plane. Beer's law for absorption of photons by radiodense materials is used to compute the amount of photon attenuation [23]. More details on how the resultant visible light and x-ray images are generated from the computed intersection points can be found in [11].

4 Sample applications The bene ts of geometric modeling, and how geometric models of anatomical structures are de ned and rendered to form simulated x-ray and visible light images, have been described. In this section, we discuss some potential applications. These include: initial training of medical personnel, evaluation of new medical display technologies, generation of test images for various medical image processing research, and visualization of reconstruction results. Sample images are presented to demonstrate the applicability of geometric modeling techniques to these problem domains. Other possible applications we have yet to explore include surgical planning and training, and immersive virtual reality displays based on geometric models.

4.1 Medical training Medical training involves the study of many images. Ideally, images of a range of normal and dis-

eased organs and images from people of dierent age groups, build and gender have to be studied because the images can vary widely. Such a large collection of images is not easily obtainable. Using geometric modeling techniques, a standard anatomy can be de ned for which the size parameters can be easily modi ed to account for dierences in age, build and gender. Ranges of abnormalities can also be de ned. The user has full control over the location, size and shape of all structures and abnormalities in the image. A simulated x-ray image of a simpli ed knee joint, requiring only 6 primitives, is shown in Figure 3. The femur is represented by a generalized cylinder with subtraction of an ellipsoid. A single generalized cylinder is used to represent the tibia. The bula is represented by a generalized cylinder with an ellipsoid representing the head of the bula. The patella is approximated with an ellipsoid. Many human organs can likewise be represented by a small number of generalized cylinders and quadric primitives. In the training of medical personnel, the ability to view in dierent directions is an important aid to understanding the structure of the object. For example, in Figure 3(a), the patella cannot be easily seen. If however, the viewpoint is rotated by 90 degrees (Figure 3(b)), the patella can be clearly seen. Such fundamental aspects of transmission images can be presented to trainees using geometrically de ned anatomical models. In more advanced training, using computer generated images, subtle abnormalities can be added by the instructor to test if the trainee can locate and identify the abnormalities. Figure 4 shows a chest image of a hypothetical patient with occupational lung disease. A variety of abnormalities are present in the image. These include a spiculate primary carcinoma in the right upper lobe, laterally, right hilar adenopathy, calci ed granulomas of histoplasmosis, diuse interstitial brosis, and a patchy left lower lobe in ltrate. This geometric model of a chest is de ned using a total of 1401 primitives. The subtleness of the abnormalities can be controlled by the instructor according to the progress of the trainee. Answer keys can also be generated by highlighting the abnormalities in the images. It is important to note that although the sample images in this paper clearly show the stated anatomy, for actual clinical training, images of higher delity may be required. The goal of our system is to provide a rich set of primitives and tools to enable an anatomist to develop yet more realistic models of the human anatomy.

4.2 Evaluation of new medical display technologies The evaluation of new medical display technologies by the receiver operating characteristics (ROC) [24] methodology requires the rating of many images whose normal/abnormal states are known. Geometric modeling and rendering can provide an inexpensive source of images for this purpose. All abnormalities are userde ned and precisely known. For subjects participating in an ROC experiment, feedback can also be provided since the state (normal/abnormal) of every image is known. Figure 5 shows two sample simulated mammograms. The left image is a test pattern that has been used in ROC studies to compare observer performances when using stereoscopic versus monoscopic displays [25, 26, 27]. The right image shows a more realistic simulated mammogram. It is generated by using approximately 1000 spheres as compared to less than 100 spheres used in the left image. As the images clearly illustrate, more primitives can be used when images of higher complexity are required.

5 Conclusion Geometric modeling approach allows convenient modeling and rendering of human anatomical structures. The basic principles behind our geometric modeling tool and some potential applications have been discussed. Our system provides an attractive alternative source of images for training and evaluation purposes.

References

4.4 Visualization of reconstruction results

[1] C. Barillot, B. Gibaud, O. Lis, L. L. Min, A. Bouliou, G. L. Certen, R. Collorec, and J. L. Coatrieux, \Computer graphics in medicine: A survey," CRC Critical Reviews in Biomedical Engineering, vol. 15, no. 4, pp. 269{307, 1988. [2] E. J. Farrell and R. A. Zappulla, \Threedimensional data visualization and biomedical applications," CRC Critical Reviews in Biomedical Engineering, vol. 16, no. 4, pp. 323{363, 1989. [3] R. Lenz, B. Gudmundsson, B. Lindskog, and P. E. Danieisson, \Display of density volumes," IEEE Computer Graphics and Applications, pp. 20{29, July 1986. [4] H.-P. Meinzer, K. Meetz, D. Scheppelmann, U. Engelmann, and H. J. Baur, \The Heidelberg ray tracing model," IEEE Computer Graphics and Applications, vol. 11, pp. 34{43, November 1991. [5] H. Fuchs, M. Levoy, and S. M. Pizer, \Interactive visualization of 3D medical data," IEEE Computer, vol. 22, no. 8, pp. 46{51, August 1989. [6] R. A. Robb and D. P. Hanson, \A software system for interactive and quantitive visualization of multidimensional biomedical images," Aus-

After a model is reconstructed, a good way to present the reconstruction results is to use visual feedback. Using our system, visible light images of a reconstructed model can be rendered (see Figure 6(b)). Visible light images give enhanced perception when compared to x-ray images since lighting and shading provide good visual cues for human viewers. Dierent views of an anatomical structure can be rendered for better understanding of the 3D structure of organs. A stereo pair can also be easily generated for three dimensional viewing.

[7] R. C. Hallgren and H. M. Reynolds, \Computer display of multidimensional biomedical data," Journal of Clinical Engineering, vol. 17, no. 3, pp. 235{243, May/June 1992. [8] M. Bomans, K.-H. Hohne, U. Tiede, and M. Riemer, \3-D segmentation of MR images of the head for 3-D display," IEEE Transactions on Medical Imaging, vol. 9, no. 2, pp. 177{183, June 1990.

4.3 Generation of test images Using our geometric modeling system, test images can be generated to evaluate new algorithms for digital compression, reconstruction and automatic pattern recognition for diagnostic purposes. For example, to test the accuracy of a stereo reconstruction algorithm for vascular structures, a stereo pair of simulated angiograms can be generated. Figure 6(a) shows one view of a simulated x-ray image of some branching and overlapping vessels. Since the vessels in the image are user-de ned and of known dimensions, the accuracy of reconstruction results can be quanti ed.

tralasian Physical and Engineering Sciences in Medicine, vol. 14, no. 1, pp. 9{30, 1991.

[9] D. R. Ney and E. K. Fishman, \Editing tools for 3D medical imaging," IEEE Computer Graphics and Applications, vol. 11, pp. 63{71, November 1991. [10] T. O. Binford, \Visual perception by computer," IEEE Conference on Systems and Control, December 1971, Miami. Invited Presentation. [11] J. Hsu and D. M. Chelberg, \Visible light and x-ray ray tracing of generalized cylinders." Manuscript submitted to the CVGIP:Graphical Models and Image Processing for review. A version is also available as TR-EE-93-43, Purdue Univeristy, November 1993. [12] H. H. Baker, \Three-dimensional and fourdimensional modeling of biological material," Proceedings of SPIE Vol. 1205 : Bioimaging and Two-Dimensional Spectroscopy, January 1990,

The International Society for Optical Engineering, pp. 104{116. [13] J. S. Hersh, \A survey of modelling representations and their application to biomedical visualization and simulation," Proceedings of the First Conference on Visualization in Biomedical Computing, May 1990, The Institute of Electrical &

[14]

[15] [16] [17] [18] [19] [20]

Electronics Engineers, Inc., pp. 432{440. N. Badler and R. Bajcsy, \Three-dimensional representations for computer graphics and computer vision," Computer Graphics, vol. 12, pp. 153{160, August 1978. M. P. do Carmo, Dierential Geometry of Curves and Surfaces. Englewood Clis, New Jersey: Prentice-Hall, Inc., 1976. G. Farin, Curves and Surfaces for Computer Aided Geometric Design. San Diego: Academic Press, third ed., 1993. A. S. Glassner, An Introduction to Ray Tracing. London; San Diego: Academic Press, 1989. R. A. Goldstein and R. Nagel, \3-D visual simulation," Simulation, vol. 16, no. 1, pp. 25{31, January 1971. C. G. C. van Dijk, Ray Tracing Pro led Generalized Cylinders. PhD thesis, Delft University of Technology, The Netherlands, June 1989. B. I. Soroka, \Generalized cones from serial sections," Computer Graphics and Image Processing, vol. 15, pp. 154{166, 1981.

[21] J. O'Rourke and N. Badler, \Decomposition of three-dimensional objects into spheres," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 1, no. 3, pp. 295{305, July 1979. [22] A. Watt and M. Watt, Advanced Animation and Rendering Techniques: Theory and Practice. New

York, NY: Addison-Wesley, 1992. [23] E. E. Christensen, T. S. Curry, and J. Nunnally, An Introduction to the Physics of Diagnostic Radiology. Philadelphia: Lee & Febiger, 1972.

[24] C. E. Metz, \ROC methodology in radiologic imaging," Investigative Radiology, vol. 21, pp. 720{733, 1986. [25] J. Hsu, C. F. Babbs, D. M. Chelberg, Z. Pizlo, and E. J. Delp, \A study of the eectiveness of stereo imaging with applications in mammography," Proceedings of the SPIE Conference on Human Vision, Visual Processing, and Digital Display IV, vol. 1913, Feburary 1993, pp. 154{165.

[26] J. Hsu, C. F. Babbs, D. M. Chelberg, Z. Pizlo, and E. J. Delp, \Pre-clinical ROC studies of digital stereomammography." Manuscript submitted to the IEEE Transactions on Medical Imaging for review. A version is also available as TR-EE-9347, Purdue University, November 1993. [27] J. Hsu, Z. Pizlo, C. F. Babbs, D. M. Chelberg, and E. J. Delp, \Design of studies to test the effectiveness of stereo imaging / Truth or Dare: Is stereo viewing really better?," Proceedings of the SPIE Conference on Stereoscopic Displays and Applications V, vol. 2177A, Feburary 1994.

a(u) t(u) b(u) n(u)

T(u,v) n(u)

t(u) b(u) C b(uk ,v)

t(u) n(u) b(u)

C n (u k ,v)

b(u) n(u)

Figure 1: Notations used to describe a generalized cylinder. ~n, ~t and ~b form the Frenet frame. ~n is the unit normal, ~t is the unit tangent and ~b is the unit binormal.

Figure 3: Simulated x-ray images of a knee joint from two 90 degrees separation views: antero posterior on the left, lateral on the right.

Image Plane (of desired resolution)

Object

Source

Figure 2: Basic principles behind a ray tracer.

Figure 4: Simulated x-ray image of a human chest, created using overlapping spherical primitives. Multiple tumor nodules and a patchy lower lobe pneumonia are simulated.

Figure 5: The top is a test-pattern for studies of human perception related to mammography. More realistic simulated mammograms (as shown on the bottom) can be generated by using more spherical primitives.

Figure 6: The top is a simulated x-ray image of some vessels. The corresponding visible light image is shown on the bottom.