3D Virtual Colonoscopy
Lichan Hong , Arie Kaufman + , Yi-Chih Wei , Ajay Viswambharan+ , Mark Wax+ , Zhengrong Liang+ Department of Computer Science + Department of Radiology State University of New York at Stony Brook Stony Brook, NY 11794
Abstract
We present in this paper a method called 3D virtual colonoscopy, which is an alternative method to existing procedures of imaging the mucosal surface of the colon. Using 3D reconstruction of helical CT data and volume visualization techniques, we generate images of the inner surface of the colon as if the viewer's eyes were inside the colon. We also create interactive ythroughs and o-line automatically-produced animations through the inside of the colon. The visualization is accomplished with VolVis, which is a comprehensive system for interactive volume visualization. We are speci cally interested in visualizing colonic polyps larger than one cm since these have a high probability of containing carcinoma. We present testing results of our method as applied to two plastic pipe simulations and to the Visible Human data set.
1 Introduction
Three-dimensional visualization of human organs employing currently available medical imaging modalities, such as CT (computed tomography) and MRI (magnetic resonance imaging), has been widely used for patient diagnosis, therapy, and surgery 8]. New advances in this technology, such as visualizing the inner surface of the colon employing 3D reconstructed CT data, are still under development 9, 4]. Currently, colonoscopy and barium enema are the only means available for the detection of polyps or carcinoma of the colon. Unfortunately, each of these procedures has its drawbacks. Colonoscopy, an invasive procedure, is associated with a small risk of perforation. In addition, it also requires intravenous sedation and approximately 40 to 60 minutes of time to perform. Frequently, due to either technical diculties or patient
discomfort, the proximal colon cannot be fully evaluated. Furthermore, the cost of colonoscopy is high, approximately $1 300 to $1 900. Barium enema, although less expensive (at the cost of approximately $350), requires a good deal of patient positioning and thus physical cooperation from the patient. Results vary according to the technical quality of the procedure and the experience of the interpreter. While some investigators state that barium enema can be as sensitive as colonoscopy, others have reported that the sensitivity achieved with barium enema can be as low as 78% in detection of polyps in the range of 0:5cm to 2cm in diameter 5]. The motivation for this work was to develop an alternative technique to visualize the inner mucosal surface of the colonic wall, and therefore assist the physicians in detecting the presence and characteristics of mucosal lesions. In this study, after the colon of a patient has been cleansed and distended with air, a spiral CT scanner is employed to obtain a sequence of thin axial slices from the top of the splenic exure of the colon to the rectum based upon landmarks obtained from the scout image. Then, this set of CT images is reconstructed into a 3D volume and visualized with the VolVis volume visualization system 1]. Images of the inner structures of the colon, as well as interactive ythroughs and o-line automatically-produced animations along the colon, are generated as if the viewer's eyes were freely mobile inside the colonic lumen. This provides the capability to precisely explore the inner surface of the colon for irregularities such as polyps and tumors. In Sections 2 through 5, we describe in detail the procedure of obtaining the CT slices of the colon and the visualization techniques employed in generating
the colon images, interactive ythroughs, and animations. In Section 6, we present our testing results on two plastic pipe simulations and on the Visible Human data set. Finally, we summarize and conclude in Section 7.
2 Visualization of the Colon
Producing images of the mucosal surface of the colon involves four steps. First, the patient's colon must be cleansed, which is similar to what is required for a barium enema or colonoscopy. Second, air is inated into the colon to produce better contrast between the colon wall and the lumen. The third step is to take a helical CT scan of the abdomen. In the fourth step, visualization techniques are employed to generate the images, interactive ythroughs, and animations of the colon. In our study, starting two days prior to the 3D virtual colonoscopy, the patient is asked to drink 100cc of liquid barium with each meal, so that retained stool can be dierentiated from polyps or tumors. On the evening prior to the day of the procedure, the patient takes a standard colonic cleansing routine consisting of drinking one gallon of Golytely. After the patient arrives at the CT scanning suite, a small tube is placed into the rectum and approximately 1 000cc air is pumped into the colon to distend the colon. A spiral CT scan with an X-Ray beam of 5mm width, 1 : 1 or 1 : 2 pitch, and 40cm eld-of-view is then performed from the level of the top of the splenic exure of the colon to the rectum based upon the landmarks obtained from the digital scout image. The scanned data is next reformatted into 5mm thick slices at increments of 1mm or 2:5mm, with each slice represented as a matrix of 512 by 512 pixels. In this way, a large number of 2D slices are generated depending on the length of the scan. This varies from 107 to 280 in our experiments (see Section 6). The actual data acquisition time is between 30 and 45 seconds. Later this set of 2D slices is reconstructed into a 3D volume to be visualized using the VolVis system. VolVis is a comprehensive, diversied, and high performance volume visualization system developed at SUNY Stony Brook 1]. It supports numerous visualization algorithms and methods within a consistent and well-organized framework, ranging from fast rough approximations, to compression-domain rendering, to accurate volumetric ray tracing and radiosity, to irregular grid rendering. Input primitives accepted
by VolVis include 3D scalar volumetric data as well as 3D volume-sampled and classical geometric models. The VolVis system consists of several primary components which are intended to meet the various needs of volume visualization users. These components are File I/O, Filters, Object Control, Image Control, Rendering, Navigation, Animation, Quantitative Analysis, and Input Devices. In this study, we employ extensively the Navigation and Animation components of the VolVis system to provide interactive object manipulation and straightforward specications of complex animations 2] for the visualization of the colon.
3 Interactive Flythroughs
The Navigator of VolVis is responsible for the interactive manipulation and display of objects within the VolVis environment. There are several basic object types used in the system, including View, Volume, Light, and World. The number and types of properties vary according to the particular applications. The View species the current viewing properties. Viewing parameters such as eye position, eld of view, image size, and stereo viewing option can be specied interactively by the user. By manipulating the viewing parameters, the user can interactively ythrough the scene in a manner similar to a ight simulator, employing a variety of input devices, which range from the standard mouse, an Isotrak, a Spaceball, to the relatively complex DataGlove. To provide interactive navigation speed, fast-rendering algorithms capable of providing a rough estimate of the actual image have been developed, which involve projecting reduced resolution representations of the objects in the scene. The interactive navigation speed is especially important for our study because, during the interactive ythroughs, positioning the colon volume within the scene and selecting desired viewing parameters can be a complex and time-consuming process. After the satisfactory eye positions and viewing directions have been obtained, images of the colon are then generated with a more accurate and more time-consuming volumetric ray tracing technique 7], which incorporates the global illumination eects and performs the rayobject intersection calculation more accurately.
4 Animations
The Animator provided by VolVis allows the user to specify transformations to be applied to objects within the scene. Unlike the Navigator, which is used to apply a single transformation at one instance, the Ani-
mator provides the user with the capability of specifying a sequence of transformations to produce an animation. Preview of the animation is achieved using one of the fast rendering options within the Navigator. Finally, a high-quality animation is created oline with the volumetric ray tracing technique. Alternatively, using the Navigator, several key transformations representing a \ight path" of the view can be interactively specied, which can then be passed to the Animator and rendered to create a complex ythrough animation along the colon. Even in an extensive interactive visualization system like VolVis, however, specifying a sequence of key frames to generate a high-quality ythrough animation along the colon still involves a high level of expertise and may require a lengthy trial-and-error process. Especially for computer illiterate users, this can become an extremely challenging problem, and may prohibit the wide acceptance of our proposed method. The scenario that we want to support in this research is the following. Once the colonic CT data set is scanned during the day, the physician uses the Navigator of VolVis to get a rst impression of the colon. Then, with some input from the physician, a highquality ythrough animation along the colon can be automatically generated o-line with minimum userinteraction during the evening, and the physician can thus watch the animation the next morning for further analysis. Generally speaking, desired properties of a highquality automatically-generated ythrough animation include: the animation should not be too jerky the eye position should stay, if possible, in the center of the colon and the viewing direction should point to the end of the current visible colon segment to provide the user with the feeling of what is coming next. To achieve these requirements, we have developed a technique and incorporated it into VolVis for automatic generation of ythrough animations. This technique is described in the next section.
5 Automatically-Generated Animations
Our technique of automatically generating a ythrough animation involves the following steps: extracting the region of interest from the whole colon specifying the rst and last points of the ythrough computing the ight path along the inside of the colon determining the viewing directions for the key
frames of the animation and nally, generating the ythrough animation by interpolating the eye positions and viewing directions between the key frames.
5.1 Extracting the Region of Interest
Figure 1 shows a 2D cross-section of a volumetric colon, upon which a regular grid is superimposed. In the rst step, the user employs the \blocking" operations of VolVis to extract the region of interest, as shown in Figure 2. In other words, with these operations, the user is able to focus attention on the region enclosed by the colon surface and the blocking walls. For simplicity, the blocking walls can be specied while the user studies the colon data set slice by slice, along the X, Y, or Z direction. This scheme has been implemented in the Object Control component of VolVis.
colon surface
colon surface Figure 1: A 2D cross-section of a volumetric colon, upon which a regular grid is superimposed.
5.2 Specifying the First and Last Fly Points In addition to specifying the blocking walls, the user needs to select the rst as well as the last points of the ight path, as shown in Figure 3. Again, this is accomplished while the user views the colon slices using the Object Control of VolVis. The rest of the ight path is automatically computed with our algorithm, as described below. Note that the specications of the blocking walls and the two end points of the ight path are the only operations that the user has to perform, which not only greatly alleviates the burden on the user, but also provides helpful exibilities for generating various ythrough animations.
colon surface
first point
blocking wall colon surface
blocking wall
Figure 2: Region of interest is extracted by specifying the blocking walls.
first point last point Figure 3: The rst and last y points are selected.
5.3 Computing the Flight Path
After the user species the region of interest and the two end points of the ight path, those grid points that are reachable from either the rst or the last y point along the grid lines, without penetrating the colon surface or the blocking walls, are automatically identied. In this way, the discrete sub-volume which is enclosed by the colon surface and the blocking walls is reconstructed, as indicated in dark grey in Figure 4. Next, the \peel onion" technique 6] is employed on the reconstructed sub-volume to compute a ight path from the rst y point to the last y point (This technique has been successfully applied in 2D Optical Character Recognition (OCR) to extract the skeleton of a digital character). With this technique, at each
last point Figure 4: The discrete sub-volume enclosed by the colon surface and the blocking walls is shown in dark grey. step the outermost layer of the sub-volume is peeled o, until there is only one layer of grid points left, which essentially is the skeleton of the colon and becomes the desired ight path for the ythrough animation. More specically, each grid point on the current outermost layer of the sub-volume (except the rst and the last y points) is deleted if the deletion does not lead to the disconnection of the rst y point and the last y point. A sequence of deletions for the subvolume in Figure 4 is shown in Figures 5 through 7. A ight path from the rst y point to the last y point is thus created, and each grid point on the ight path consequently corresponds to the eye position of a key frame.
5.4 Determining the Viewing Directions
For the viewing parameters of each key frame, in addition to the eye position, there are three degrees of freedom for the viewing direction which have to be determined. From our experiments, we have noticed that simply using the vector from the current eye position to the eye position in the next key frame is undesirable. One of the problems is that, due to the discrete nature of the ight path, the viewing direction may point at the side wall of the colon surface (see Figure 7). Another problem is that, again because of the discrete nature of the ight path, the current viewing direction is always parallel to a main axis and could be 90-degree rotationally dierent from the viewing direction of the previous key frame, which may lead to a jerky animation (see Figure 7).
first point last point Figure 5: One layer of \onion" is peeled o the subvolume of Figure 4.
first point last point Figure 6: One more layer is peeled o the sub-volume of Figure 5.
In general, for an informative ythrough animation, the user's eye should always focus at the end of the currently visible colon segment, expecting new features to come next. In our implementation, instead of letting the current viewing direction focus on the eye position of the next key frame, we compute the viewing direction by using the eye positions of the next 5n to 10n key frames, where n is the number of \onion" layers in the original sub-volume, as shown in dark grey in Figure 4. If all the eye positions from the next 5n to 10n key frames are visible to the current eye position, the vector from the current eye position to the eye position of the next 5n key frame is used as the viewing direction of the current key frame. Otherwise, we backtrack one step and test the visibilities of the eye positions at the next 5n ; 1 to 10n ; 1 key frames. This procedure is recursively performed until we nd the viewing direction. The visibility of a certain point from the current eye position is determined by shooting a ray from the current eye position towards the target point. If there is an intersection between the ray and the colon surface, and the intersection point lies between the current eye position and the target point, then the target point is invisible to the current eye position. Otherwise, the target point can be seen by the current viewing eye. In this way, we can determine the viewing directions for all the key frames of the ythrough animation. It can be seen from the video segment in the video proceedings that this heuristic worked very well for our case studies.
5.5 Interpolating between Key Frames
first point last point Figure 7: Once the last layer of \onion" is peeled o the sub-volume of Figure 6, a ight path is created.
After the eye positions and viewing directions for the key frames have beed determined, the Animator of VolVis is applied to create a smooth camera-motion animation by interpolating the eye positions and the viewing directions between two consecutive key frames using cubic splines 3]. Since the animation is generated o-line without user interaction, high-quality images can be rendered with the volumetric ray tracing technique implemented in VolVis. The animation can be subsequently played back on a workstation or recorded on a video tape. Notice that the key frames of the animation can act as the basis of the interactive ythrough when further analysis of the colon is needed. For example, the user can rotate the viewing direction of a certain key frame to look around
in a search for interesting features, which is especially helpful at those positions close to a potential tumor.
6 Experiments and Results
We have conducted two experiments, one with two plastic pipe simulations generated at the University Medical Center of SUNY Stony Brook, and the other using the Visible Human data set. The objective of the rst study is to see how well we can detect the presence and characteristics of mucosal lesions measuring at least one centimeter in diameter, which are considered clinically signicant since they have a high probability of being malignant. As prototypes and for testing purposes, two dierent phantoms consisting of a plastic pipe looped inside a water tank were imaged by a GE HighSpeed CT in the helical mode, which resulted, respectively, in one 512 512 280 and another 512 512 107 volumetric data set, with each sample point represented in 2 bytes. The plastic pipe had a radius of 2cm, and its surface had corrugated and smooth segments. For the 512 512 280 data set, a rubber cylinder of length 7mm and diameter 5mm was inserted into the pipe to simulate a tumor. This data set was obtained with a slice thickness of 3mm at increments of 1mm using a eld-of-view of 34:5cm. Figure 8 shows six images in a virtual interactive ythrough of the 512 512 280 pipe data set, generated using VolVis. Figure 8a is a view of the phantom before entering the pipe Figure 8b shows the image of the pipe before the tumor is encountered Figure 8c is a distant view of the tumor within the pipe before reaching it Figure 8d is a close front view of the tumor (notice that the viewing orientation is dierent from that of Figure 8c) Figure 8e is a close side view of the tumor when passing next to it and Figure 8f is a distant view of the tumor after passing it. For the 512 512 107 data set, three rubber cylinders were inserted into the pipe. The lengths of the three cylinders were 7mm, 5mm, and 3mm, respectively, and the diameters were 5mm for all cylinders. This data set was obtained with a slice thickness of 5mm at increments of 2:5mm using a eld-of-view of 40cm. A ythrough animation of the 512 512 107 pipe data set has been generated automatically with the technique described in Section 5 and recorded on a video tape, which is included in the video proceedings. In our second study, we successfully applied the automatic animation technique of Section 5 to the Visible Human data set. In this study, we primar-
ily focused on the physical cross-sections of the data set, from slice No: 1595 to slice No: 1848. Due to memory limitations, each slice of the physical crosssections was down-sampled from the original resolution of 2048 1216 to 683 406. The red component of the 24-bit color data set was extracted to compute the eye positions and viewing directions of the ight path along the colon of the Visible Human, and later, the whole 24-bit color data set was used as a texture map when the animation frames were rendered by VolVis. Figure 9 shows six frames of this ythrough animation. Figure 9a is the start of ight through the descending colon proceeding towards the rectum Figure 9b shows the colon curving to the left Figure 9c shows three haustra (inner-folds) of the colon Figure 9d is a close view of the three haustra seen in Figure 9c Figure 9e shows the colon beyond the last haustra seen in Figure 9c and Figure 9f shows a smooth walled object projecting from the colon surface. Compared with the images in Figures 9a through 9e, the large size of the lumen in Figure 9f indicates that we are entering the rectum. The complete ythrough animation is also included in the video proceedings.
7 Concluding Remarks
The main advantages of 3D virtual colonoscopy over barium enema include patient comfort and speed. Much less patient discomfort results using the 3D virtual colonoscopy since there is no need to reposition the patient. In addition, the entire process for obtaining the helical CT images (including the preparation phase) should take only 15 to 20 minutes of patient time, as compared to 45 minutes or more for barium enema. Also, the radiation dose used in a 3D virtual colonoscopy study is approximately one half to one quarter of the average exposure in barium enema. Furthermore, an unlimited number of viewing angles can be exploited with the visualization techniques. Compared with conventional colonoscopy, this technique is expected to be less expensive (about half of the cost), involves less discomfort and risks to the patient. There is also no need for sedation, and the patient can therefore resume normal activities immediately after the CT scanning procedure. A major contribution of this paper is the incorporation of a set of volume visualization techniques for the exploration of the CT-scanned colon data sets. In particular, the idea of extending the \peel onion" technique from 2D image processing to the 3D volumetric
domain to automatically produce the ythrough animations along the inside of the colon is, to our best knowledge, an original contribution to biomedical visualization. Furthermore, unlike other approaches to 3D virtual colonoscopy 9, 4] which extract the colon surface and represent it as a list of triangles, our approach is volume-based, thus not only allowing the user to visualize the mucosal surface of the colon, but also providing the exibility of exploring the data beyond the surface. For example, a virtual surgery cut can be applied to a polyp or the outer layers of the polyp can be rendered translucently to reveal the hidden information inside the polyp. Our testing results on the two pipe data sets and the Visible Human data set are very encouraging. The images, interactive ythroughs, and automatically-generated animations that we achieved with the VolVis system clearly demonstrate the feasibility of diagnosing colonic polyps by our non-invasive, safe, and relatively comfortable procedure. We are currently working on obtaining real colon data sets from patients at the University Medical Center of SUNY Stony Brook and evaluating the application of this method on clinical cases.
Acknowledgments
This research has been supported in part by the National Science Foundation under grants CCR-9205047. The source of the Visible Human data set is the National Library of Medicine and the Visible Human Project. Special thanks to Sidney Wang for helpful discussions. We also wish to thank Patrick Tonra for his help and support in generating the video. VolVis can be obtained by sending email to
[email protected].
References
1] R. Avila, T. He, L. Hong, A. Kaufman, H. Pster, C. Silva, L. Sobierajski, and S. Wang, \VolVis: A Diversied Volume Visualization System". Proceedings Visualization '94, 31-38, Washington D. C., October 1994. IEEE Computer Society Press. 2] A. Kaufman, L. Sobierajski, R. Avila, and T. He, \Navigation and Animation in a Volume Visualization System". In N. Thalmann and D. Thalmann (eds.), Models and Techniques in Computer Animation, 64-74. Springer-Verlag, 1993.
3] D. Kochanek and R. Bartels, \Interpolating Splines with Local Tension, Continuity, and Bias Control". Computer Graphics, Vol.18, No.3, 33-41, July 1984. 4] W. Lorensen, F. Jolesz, and R. Kikinis, \The Exploration of Cross-Sectional Data with a Virtual Endoscope". In R. Satava and K. Morgan (eds.), Interactive Technology and the New Medical Paradigm for Health Care, 221-230. IOS Press, Washington D. C., 1995. 5] C. Morosi, G. Ballardini, and P. Pisani, \Diagnostic Accuracy of the Double-Contrast Enema for Colonic Polyps in Patients With or Without Diverticular Disease". Gastrointest Radiology, 16:346347, 1991. 6] T. Pavlidis, Algorithms for Graphics and Image Processing. Computer Science Press, 1982. 7] L. Sobierajski and A. Kaufman, \Volumetric Ray Tracing". 1994 Symposium on Volume Visualization, 11-18. ACM Press, October 1994. 8] J. Udupa and G. Herman (eds.), 3D Imaging in Medicine. Boca Raton, Florida. CRC Press, 1991. 9] D. Vining, D. Gelfand, and R. Bechtold, E. Scharling, E. Grishaw, and R. Shifrin, \Technical Feasibility of Colon Imaging with Helical CT and Virtual Reality". Presented at the Annual Meeting of the American Roentgen Ray Society, New Orleans, April 1994.
