Deformable Volume Rendering by 3D Texture Mapping and Octree Encoding Shiaofen Fang1 Su Huang2
Rajagopalan Srinivasan2 Raghu Raghavan2
Institute of Systems Science, National University of Singapore
ABSTRACT Rendering deformable volume data currently needs separate processes for deformation and rendering, and is expensive in terms of both computational and memory costs. Recognizing the importance of unifying these processes, we present a new approach to the direct rendering of deformable volumes without explicitly constructing the intermediate deformed volumes. The volume deformation is done by a radial basis function that is piecewise linearly approximated by an adaptive subdivision of the octree encoded target volume. The octree blocks in the target volume are then projected, reverse morphed and texture mapped, using the SGI 3D texture mapping hardware, in a back-to-front order. A template based Z-plane/block intersection method is used to expedite the block projection computation.
Keywords { Volume Rendering, 3D Texture Mapping, Morphing, Volume Deformation, Octree, Scienti c Visualization.
1 INTRODUCTION 1.1 Deformable Volume Rendering
Volume rendering is an emerging technique widely used in visualizing the internal structures of large volumetric datasets. In many scienti c applications, however, the volumetric objects are often time dependent and deformable. Using normal volume rendering methods to visualize the deformation process requires the construction of all the intermediate deformed volumes. Such volume construction is unfortunately very expensive in terms of both computational and memory costs, and is far from being real-time or interactive animation rate. For fast animated rendering of deformable volumes, new techniques are needed to handle dynamic volume data. In this paper, we present a deformable volume rendering algorithm that embeds the deformation into the visualization process for fast rendering of arbitrary 3D morphingbased deformation of various volume datasets. Hardware assisted 3D texture mapping and the octree encoding of the volumetric objects are also used in the algorithm to improve the rendering speed. 1 Department of Computer and Information Science, Indiana University Purdue University at Indianapolis, Indianapolis, IN 462025132. e-mail:
[email protected] 2 Institute of Systems Science, National University of Singapore, Heng Mui Keng Terrace, Kent Ridge, Singapore 119597.
1.2 Related Work
There are a large number of publications on the topic of volume rendering. They can be roughly classi ed into two categories: image space approaches such as raycasting based algorithms[2, 13, 25], and object space approaches[11, 21, 22]. Using hardware assisted 3D texture mapping for volume rendering has also been reported by several researchers[1, 6, 19, 23], and it usually works in the object space. Although it seems natural to use 3D texture mapping to deform the volumes, not much has been done in exploiting such a potential. 3D deformation of geometric objects has been extensively studied in computer graphics and geometric modeling including, for example, free-form deformation (FFD)[18] and physically-based modeling[20]. Recently, methods have been proposed to address the issue of 3D image and volume deformations, mostly as extensions to 2D image warping[24]. Methods based on: landmarks[3], features[12], Fourier[9], and wavelet transforms[8] have been used. These methods, however, are only for the construction of deformed volumes that can be separately rendered by static volume rendering algorithms. The construction of a deformed volume of 2563 , for example, can take hours[3, 12]. The adaptive re nement approach proposed in [12], using trilinear interpolation to approximate the morphing function to accelerate the morphing computation, still takes three minutes to construct one such deformed volume. Another relevant work is the ray de ector based raycasting by Kurzion and Yagel[10]. In their approach, the deformation is modeled by ray de ectors that deform the rays during raycasting to generate visual deformation effects. Because the rays are bent into curves, the ray/object intersection is more dicult than normal raycasting. No rendering speed information was given in their paper.
1.3 Overview
The algorithm to be presented in this paper is an object space algorithm based on octree projection. Although texture mapping is used here to take advantage of hardware features, the algorithm can be easily modi ed for software implementation of any object space rendering algorithm. In this algorithm, the original volume is subdivided into texture blocks of xed size. For each given pair of landmark point sets in the original and deformed (target) volumes, a backward morphing function from the deformed volume to the original volume is computed. An adaptive subdivision process using octree encoding is then invoked to select target octree blocks in the deformed volume so that the mor-
phing can be approximated by trilinear interpolation within each target block. Once this setup for morphing function and target blocks is done, the result can be used for the rendering of all intermediate animation frames in all viewing angles. For each viewing direction, the target blocks are rst sorted in a back-to-front order, and then projected and texture mapped to the projection plane in that order. The block projection involves computing the polygons from Z-plane/block intersections, and interpolating the morphed vertices of the polygons in the original volume and their texture coordinates. Because of the uniform sizes and orientation of the target blocks, such Z-plane/block intersection needs to be done only once for sample blocks of each size and the results are \pasted" to all other blocks. In Section 2, we rst present our method of computing the landmark-based morphing function for volume deformation. The generation of texture blocks and the adaptive subdivision of the target blocks are described in Section 3. Section 4 discusses the template-based block projection algorithm and the texture mapping process in detail. In Section 5, some implementation details and results are given, followed by a discussion on a few pratical problems in using the 3D texture mapping hardware in OpenGL. We conclude our presentation with a few further comments and future work in Section 6.
2 VOLUME DEFORMATION BY MORPHING 2.1 Hardy's Interpolation
A landmark based 3D morphing can be de ned as a globally smooth interpolation between scattered data points in 3D space. This problem has been widely studied in computer aided geometric design. Several methods have been proposed in using scattered data interpolation for volume morphing and image warping[3, 12, 17]. One particular radial basis function method, Hardy's interpolation, has been especially useful for morphing purposes because of its global smoothness and robustness[3, 4]. In our approach, Hardy's interpolation is used as the deformation function, but in the reversed direction, i: e: it is a backward morphing that transforms the points in the deformed volume back to the original volume for texture association. For given source landmarks fL0i = (x0i ; yi0 ; zi0 )g (i = 1 n) in the original volume and target landmarks fLi = (xi ; yi ; zi )g (i = 1 n) in the deformed volume, the morphing function is de ned as:
X H(P ) = h (d (P ) + r ) + L(P ) n
i=1
i
2 i
2
i
(1)
where L(P ) is a linear function to absorb any global linear transformation to the whole landmark set, di is the distance function from Li , and is the exponent parameter that should be either 0:5, ?1 or 1 for good results. ri is a set of parameters that can control the sti ness of the deformation. We set it to min di (Pj ) j6=i
to ensure that the deformation is soft in the region where landmarks are sparse and sti in the region where landmarks are dense. The coecients fhi g (hi 2 R3 ) and L are computed by solving the following linear system from the interpolation
equations to ensure that landmarks are mapped to their corresponding landmarks up to a linear transformation. H(Li ) = L0i (2)
X h x = X h y = X h z = X h = 0 (3) n
i=1
n
i
i
i=1
n
i
i
i=1
n
i
i
i=1
i
It has been shown[14] that such a linear system is very stable and the interpolant exists and is unique under normal datasets. More details about Hardy's interpolation and other scattered data interpolation methods can be found in [3, 5, 7, 15, 16, 17].
2.2 In uence Radii and Locally Bounded Hardy's Interpolation
Computing H(P ) is quite expensive because the distances from each point P to all landmarks need to be individually calculated, and there are usually a large number of points to be morphed in a volume. It was pointed out in [5, 17] and elsewhere that, when = ?1, an in uence radius Ri can be de ned for each landmark point Li , so that its contribution (the ith term in the summation in (1)) to points outside its in uence area (the sphere centered at Li with radius Ri ) is smaller than a pre-de ned tolerance , and therefore can be neglected. This leads to the following locally bounded Hardy's interpolation:
M(P ) =
X H [(d (P )+r ) n
i=1
i
2 i
2 ?1 ? (R2 + r2 )?1 ]+ + L(P ) (4) i i
i
where Ri is computed from the equation: jhi j (R2i + ri2 )?1 = (5) In function M, the ith term in the summation vanishes if the distance to Li is larger than Ri . The tolerance determines the accuracy of the approximation of (4) to (1). In our experiments, when = 0:5, the result from using (4) and (1) are almost identical. It is also commonly required that Ri > ri to ensure that all points are covered. To compute the coecients fHi g of M(P ), we need to rst solve the linear system 1 and then compute the in uence radii fRi g from 5. Finally the following interpolation equations can be solved to obtain fHi g:
M(Li ) = L0i
(6) Without noticeable loss of quality, M(P ) can be computed much faster than H(P ) because points outside the in uence areas of the landmarks need not be considered. For the examples we have tested, an average speed up of ten can be achieved. The concept of the in uence radius also plays an important role in the adaptive subdivision of the target volume to be discussed in next section.
3 TEXTURES AND TARGET BLOCKS 3.1 Texture Volume, Texture Memory and Texture Blocks We de ne a texture volume as a volume with voxels containing RGB color and opacity values. If an input volume dataset is based on intensities, it needs to be rst converted
into a texture volume by a transfer function. This texture volume will then be loaded into the hardware texture memory for texture mapping. Because the texture volume is often much larger than the texture memory, it has to be subdivided into smaller subvolumes called texture blocks that can be comfortably put into the texture memory. Each texture block is considered a separate texture map that can be accessed through the texture coordinates of points in the object space. When a texture block is rst accessed, a texture binding operation is needed to de ne the texture block as the current texture map, and subsequent accesses to the same texture block will use the same texture map until a new texture block is encountered. The binding is not expensive if the texture block is already in the texture memory. Otherwise, the content of the texture block will be loaded into the texture memory. Furthermore, if the texture memory is full, swapping of the entire texture memory is then necessary. Such texture block loading and texture memory swapping is very expensive (e: g: on an SGI VTX machines with 4MB texture memory, each swapping takes approximately 0:1 second) and is often the dominant cost in texture mapping based algorithms, as we will show later in Section 5. From our experience, using smaller texture blocks sometimes helps to reduce the texture binding time (including the time for loading and swapping) as shown in Section 5. But this is very much dataset and hardware implementation dependent, and will not be discussed further in detail in this paper. For the subdivision of the texture volume into texture blocks, an octree data structure is used. In our implementation, the subdivision is uniform, i: e: the texture blocks are the same sized octants of the texture volume. The octree data structure here serves two purposes: to eciently query the texture blocks. Such queries are used very frequently in our rendering algorithm. to have a consistent data structure with the deformed volume so that the algorithm can be uniformly applied to the texture volume directly for the rendering of undeformed volume.
3.2 Target Blocks and Their Subdivisions
To approximate the morphing function M, the deformed volume space is adaptively subdivided into target blocks so that only the block vertices need to be morphed, and all other points in the block can be trilinearly interpolated (approximations using higher degree polynomials, such as quadratic interpolations, is also possible, but may be more expensive). The subdivision process operates within a target octree. This target octree is initially set to a complete octree representing a preliminary uniform subdivision with a pre-determined depth level. The vertices of the leaf nodes of the uniform target octree are rst morphed back to the original volume space (texture space), using the morphing function M. If the bounding box of the eight morphed vertices of a leaf node falls into an empty space, i: e: it does not intersect any texture block, this leaf node is considered empty. A target block is de ned as a non-empty leaf node in the current target octree. In some blocks, the trilinear approximation of the morphing function may not be suciently accurate for the desired deformation. When this happens, these target blocks should be subdivided further into eight sub-blocks for better approximation. To identify blocks for subdivision, we adopt
Texture Space
Object Space
morph
Figure 1: Discontinuity in texture space for neighboring blocks with di erent levels of subdivisions an approach used in [12]. In this approach, several sample points in each target block are selected (often the center of the block and the centers of its six faces, since these points will be used anyway if subdivision is needed) and deformed separately by the morphing function M and the trilinear interpolation. If the distances between the morphed results and the interpolated results are all under the given tolerance, the target block is considered acceptable, otherwise subdivision is needed. This process continues recursively until all target blocks meet the tolerance requirement. For the purpose of approximating the deformation function, the tolerance used above is a measure of how close the trilinear interpolation is to the original morphing function. On the other hand, because of the existence of neighboring blocks with di erent levels of subdivision, sampling discontinuity in the texture volume can occur along the common boundary of such neighboring blocks after trilinear interpolation, as shown in Figure 1. Such discontinuity can cause a small portion (within the given tolerance) in the texture volume be either mis-sampled or double-sampled. To avoid artifacts in the rendered deformed images caused by this problem, the tolerance has to be small. In our experiments, a tolerance of the size of three voxels seems to be sucient for the examples shown in this papers. Directly using the above subdivision approach with a uniform small tolerance can generate a large number of target blocks, and consequently slow down the rendering. To reduce the number of blocks without substantially a ecting the nal rendering quality, we introduce a varying tolerance approach. The tolerance controlling the precision of the subdivision can be di erent in di erent regions of the volume, since not every place in the volume needs the same level of approximation. The in uence radii of the landmarks provide a natural and quantitative way to determine such varying tolerances. We take a small in uence area (i: e: a small in uence radius) to mean that the landmark is used for small but more delicate shape changes in a small region. In other words, in the in uence areas of the landmarks with small in uence radii, the tolerances for the approximation of the morphing function should be smaller than in larger in uence areas. For example, each target block can be assigned with a tolerance that is proportional to the radius of the smallest in uence area it falls in. Based on these tolerances, the subdivision can be coarse in regions where the deformation is more uniform, and is ner in regions where more delicate shape changes are expected. During the subdivision, the vertices of all target blocks have already been morphed, as in the uniform case. Thus, the bounding box of the morphed vertices of each target block can then be checked to see whether it falls into empty
z X
Y
Z
Z-plane
Back-to-front order 7
6 >=0
>=0
>=0
7, 6, 5, 3, 4, 2, 1, 0
=0
>=0
6, 7, 4, 2, 5, 3, 0, 1
4
5
4 >=0
=0
5, 4, 7, 1, 6, 0, 3, 2
=0
>=0
