State of the Art Report 2004 on GPU-Based

State of the Art Report 2004 on GPU-Based Segmentation Markus Hadwiger

Caroline Langer

Henning Scharsach

Katja B¨ uhler

VRVis Research Center Vienna, Austria {msh|caro|henning|katja}@VRVis.at

Figure 1: In the left three images, an interactive segmentation of a brain tumor with an active surface model given in implicit form is evolving toward the final segmentation using the level set method [Lefohn et al. 2003]. In the right image, the same method is applied to segmentation of a mouse liver. Level set computations and simultaneous visualization are performed entirely on the GPU (graphics processing unit), which performs general floating-point computations in addition to rendering.

1

Abstract

Recent advances in the computational power of graphics processing units (GPUs) have turned them into a viable platform for general purpose floating-point computations. A very promising application of these new capabilities is interactive segmentation of medical volume data, which usually involves solving a large number of partial differential equations (PDEs) for each iteration of an evolving segmentation that can be viewed and guided by user input while it is being calculated, and is thus computationally very demanding. We give an overview of segmentation algorithms with a focus on leveraging the power of GPUs in order to obtain high-quality segmentations of medical data in an interactive process, with the premise that these algorithms will lead to faster and higher-quality segmentations in clinical practice in the near future.

2

Introduction

Segmentation, i.e., the identification of individual objects of interest in images or volumes [Udupa and Herman 1999], is a fundamental task in image and volume analysis and processing. Especially in the case of medical imaging, the quality and accuracy of the obtained segmentations is crucial. However, obtaining high-quality segmentations is a tedious task that is still often done manually in clinical practice [Lefohn et al. 2003a]. Automatic segmentation algorithms are both computationally expensive and usually require many iterations with user guidance in order to achieve results of sufficient quality. A recent and very promising approach to segmentation of medical data is interactive segmentation, where the evolution of the segmentation can be observed and influenced by the user while it is being computed. However, the compu-

tational demands of interactive segmentation are very high, since many methods require the solution of a large number of partial differential equations (PDEs) for each iteration. The recent advances in graphics processing units (GPUs) have led to an increasing amount of research into using these specialized processors for general purpose computations such as solving large linear systems [Bolz et al. 2003; Kr¨ uger and Westermann 2003b], partial differential equations [Hillesland et al. 2003; Lefohn et al. 2003c], and many other tasks of numerical computing. These uses of GPUs are generally subsumed under the name general purpose computations on GPUs [Harris 2003] and conceptually build on the computational model of stream processing [Kapasi et al. 2002]. A very recent area of research is the concurrent development of real-time high-quality rendering of segmented data [Hadwiger et al. 2003a] and interactive computation of the underlying segmentation [Lefohn et al. 2003b], which allows to compute and view the entire segmentation process interactively [Lefohn et al. 2003b; Lefohn et al. 2003c]. Furthermore, different pre-filtering tasks that are often applied to a volume before actual segmentation can now also be performed interactively on GPUs, e.g., nonlinear image processing [Viola et al. 2003] as well as diffusion filtering [Sherbondy et al. 2003].

Organization This report is organized as follows. Section 2.1 briefly outlines the notion that current GPUs are almost general stream processors [Kapasi et al. 2002] performing highly parallel floating point computations. Section 2.2 gives an overview of the field of general purpose computations on GPUs, which includes all GPU-based segmentation methods. Section 3 gives an overview of segmentation methods in general, before region-based approaches are covered in more

detail in section 4, and level set methods are described in section 5. Section 6 covers different kinds of pre-processing tasks such as pre-filtering, and section 7 describes several important implementation issues relevant to GPU-based segmentation. Section 8 briefly covers rendering of segmented data.

2.1

The GPU as Stream Processor

The fundamental characteristic of the latest generation of programmable graphics hardware (GPUs) with respect to their model of computation is that they can largely be viewed as almost general high-performance stream processors [Kapasi et al. 2002]. A stream processor operates on a stream of data items of identical type by executing a stream kernel on each of them. Execution of the kernel on one data item is independent from all other data items, and thus the stream processing model allows direct parallelization by processing an arbitrary number of stream elements in parallel. Thus, GPU-based algorithms can be executed in a highly parallel manner, which is the fundamental reason for their high performance. Current GPU architectures contain highly programmable vertex and fragment processors [Rost 2004] that operate on a stream of vertices and fragments, respectively. Data items are taken from the input stream, modified by the stream kernel (in this case, either the vertex shader or the fragment shader ), and put onto the output stream. The stream processing architecture of current GPUs has been exploited for tasks as diverse as ray tracing [Purcell et al. 2002], and solving large linear systems of equations [Kr¨ uger and Westermann 2003b; Bolz et al. 2003]. In most of these algorithms, the fragment processor is the most important component, whereas the vertex processor is used much less often. Fortunately, many segmentation algorithms can naturally be cast in the model of stream processing, since they process many data elements (pixels or voxels) in an identical manner, e.g., by solving a partial differential equation for each of them. Recently, stream processing implementations of segmentation methods based on diffusion [Sherbondy et al. 2003] and level set evolution [Lefohn et al. 2003b] have been introduced, offering significantly higher performance than CPU-based implementations. Rates of ten to twenty times faster than comparable optimized CPU solutions have been reported [Lefohn et al. 2003b; Sherbondy et al. 2003].

2.2

General Purpose Computations and Segmentation on GPUs

Over the last several years, an increasing trend toward using GPUs for general computations can be observed. The breakthrough for these approaches came with the introduction of floating-point pixel processing with the ATI Radeon 9700 GPU in late 2002, and the NVIDIA GeForce FX GPU in early 2003. Earlier attempts [Rumpf and Strzodka 2001b; Rumpf and Strzodka 2001a] had been limited by precision and range restrictions of the traditional 8-bit fixed point representations for RGBA color data [Strzodka 2002; Hadwiger et al. 2003b]. The capability of current GPUs to perform general floating-point computations has finally removed the major obstacle for these approaches, and thus led to a lot of recent research into using the computational power of GPUs for solving structured and repetitive but computationally demanding algorithms.

Kr¨ uger and Westermann [Kr¨ uger and Westermann 2003b] developed general linear algebra operators including sparse as well as dense matrix computations as a basis for general computations on GPUs. Demonstrated applications included the 2D wave equation and the incompressible NavierStokes equations. An additional advantage of computing simulation results on the GPU is that they can also be rendered directly without transferring data from CPU memory to GPU memory, which is also true for GPU-based segmentation [Lefohn et al. 2003c]. In addition to sparse matrix solvers, Bolz et al. [Bolz et al. 2003] have presented a multigrid solver for regular grids. Hillesland et al. [Hillesland et al. 2003] map general nonlinear optimization to a stream processing model, and use it for image-based modeling on GPUs. In the area of solving partial differential equations applied to image and volume segmentation, Rumpf et al. were the first to go into this direction using nonlinear diffusion [Rumpf and Strzodka 2001b] as well as level set evolution [Rumpf and Strzodka 2001a]. Although these first attempts were still constrained to 8-bit fixed point computations, the results were already quite promising. In a later paper, they also presented an anisotropic level set method [Preusser and Rumpf 2002], exploring the relation to anisotropic diffusion, and an application to geometric smoothing. Level set methods have recently gained a lot of attention [Osher and Fedkiw 2003]. A very promising method for level set computations on current GPU architectures has been presented by Lefohn et al. [Lefohn et al. 2003b; Lefohn et al. 2003c; Lefohn et al. 2003a; Lefohn and Whitaker 2002]. See section 5 for more details. A simpler GPU-based approach building on isotropic diffusion of intensity data and an additional seed volume has been presented at about the same time [Sherbondy et al. 2003]. See section 4 for more details. The Hough transform, which is a well-known method for shape recognition in images, can also be performed on graphics hardware [Strzodka et al. 2003b]. The last features of GPUs also allow to perform image registration [Strzodka et al. 2003a]. Building upon the imaging subset introduced with OpenGL 1.2 [McReynolds et al. 2000] that offers support for 1D and 2D convolution operations, Hopf and Ertl [Hopf and Ertl 1999] have shown how to perform 3D convolutions for volume processing, and presented research on using graphics hardware for morphological operations [Hopf and Ertl 2000a], and wavelet transforms [Hopf and Ertl 2000b]. Yang and Welch [Yang and Welch 2003] present simple image segmentation based on morphological operators and smoothing on graphics hardware. Linear convolution filters for image processing can be evaluated very effectively on GPUs [James 2001]. The evaluation of non-linear convolution image processing filters has also become possible recently [Viola et al. 2003], which has been applied as pre-filter before actual segmentation [Viola et al. 2003]. In linear convolution filtering, arbitrary convolution filters for texture magnification can also be evaluated on graphics hardware in real-time [Hadwiger et al. 2001; Hadwiger et al. 2002]. Another interesting area of computations on GPUs is the computation of distance fields from polygonal geometry. Distance fields can be generated from polygonal models by rasterizing Voronoi cells into the slices of a 3D texture [Sigg et al. 2003; Sud et al. 2004]. Recently, generalized distance transforms and skeleton computations have also been implemented on GPUs [Strzodka and Telea 2004].

3

Overview of Segmentation Methods

This section gives an overview of important approaches to the segmentation problem, especially focusing on methods that can potentially profit from parallel implementations on GPUs. Very good surveys have been published for thresholdingbased techniques [Sezgin and Sankur 2004], nonlinear diffusion filtering [Weickert 1997], diffusion-based segmentation and the combination with level set methods [Suri and Wu 2002], deformable models [McInerney and Terzopoulos 1996], vessel extraction techniques [Kirbas and Quek 2003], vessel segmentation in CTA data sets [Felkel 2000], and different centerline approximation algorithms for blood vessels [La Cruz 2004].

3.1

Thresholding

The simplest technique for identifying individual objects in a volume assigns object membership based on a single density threshold, or a range of densities, including or excluding individual voxels exclusively based on their individual density. That is, no spatial or neighbor information is taken into account, and the same threshold is applied globally over the whole volume. Since these thresholds are given in the domain of volume density, which is often also called the transfer function domain, the same result can also be achieved by specifying a corresponding transfer function. Although a transfer function is traditionally viewed as a single RGBA function, in this context it is useful to view classification as assigning opacity only, and the assignment of colors and other optical properties as assignment of optical properties to individual classified objects. Classification identifies objects, and assignment of optical properties determines what they should look like in a volume rendered image [Engel et al. 2002]. Recent approaches also try to specify transfer functions in higher-dimensional spaces including the spatial position of individual voxels [Tzeng et al. 2003], which blurs the line between segmentation and transfer function specification. Beyond simple thresholding, a number of more powerful adaptive techniques have also been developed. Sezgin and Sankur [Sezgin and Sankur 2004] present a comprehensive survey of state of the art segmentation techniques based on thresholding, also including an extensive bibliography of the field. Due to their simple structure, thresholding techniques are well-suited to be implemented on graphics hardware.

3.2

Boundary localization

Segmentation methods based on the identification of object boundaries usually detect these boundaries with an edge detection filter and try to connect the resulting edges to closed regions afterward. Since the basic goal is identifying regions instead of boundaries, this step is crucial for and the most demanding in these methods. Edge detection filters basically try to identify edges based on gradient magnitudes, which are often not computed explicitly, but built into the filter kernel itself, e.g., a Sobel filter. Another well-known filter for finding edges is the Canny edge detector [Canny 1986]. Due to the quite irregular step of finding and connecting edges to a single object boundary, pure boundary localization approaches are not well-suited to implementation on graphics hardware architectures.

3.3

Region Growing

In contrast to approaches based on identifying boundaries, region growing approaches operate on closed regions in each iteration of the algorithm, and thus trivially achieve closed regions in the final segmentation. Starting from an initial region (which might be a single voxel), or multiple regions, these regions are grown as long as some homogeneity criteria are satisfied. Start regions are usually called seeds, e.g., in seeded region growing [Adams and Bischof 1994], and can be specified either manually by the user, or automatically based on certain heuristics, e.g., in unseeded region growing [Lin et al. 2001]. We categorize the large area of segmentation based on diffusion filtering into region growing techniques, since the diffusion process leads to growing regions instead of maintaining or detecting explicit boundaries. However, these techniques generally do not maintain homogeneity criteria for regions, but instead operate locally by solving partial differential equations. Weickert provides a very nice overview of nonlinear diffusion filtering [Weickert 1997]. The original anisotropic diffusion filtering operator has been developed by Perona and Malik [Perona and Malik 1990], building on the idea of scale-space. The original scale-space filtering concept has been presented by Witkin [Witkin 1983]. Region-based techniques based on solving partial differential equations are very well-suited to be implemented on graphics hardware. More details on region growing methods, concentrating on diffused-based techniques, are given in section 4.

Watershed The watershed transform [Beucher 1991] is a region-based segmentation method from mathematical morphology of gray-scale images. It views the gradient magnitude image corresponding to the original image as a height field in which homogeneous regions are separated by lines of gradient magnitudes above a certain threshold. These regions are constructed via a flooding process and correspondingly called catchment basins. Unfortunately, the standard watershed transform often leads to over-segmentation. The major approach to overcoming this problem is the introduction of markers, see below. Stoev and Straßer [Stoev and Strasser 2000] present a local watershed approach that applies flooding only to selected regions of interest instead of entire images. See figure 2. They also present an improved strategy for merging basins with minimal user interaction.

Figure 2: [Stoev and Straßer 2000] apply a localized watershed transformation via specification of a region of interest (ROI). (left) Flooding the relief image in the ROI, for obtaining the final segmentation (right).

Watershed from Markers As pointed out above, the basic watershed transform leads to the problem of over-segmentation. This problem can be overcome by manually placing markers (usually simply consisting of lines). A recent efficient implementation is based on computing a minimal cost forest [Felkel et al. 2001].

3.4

Shape-Based Approaches

In contrast to region growing approaches, which are entirely image-based and operate at the resolution of pixels or voxels, approaches based on the actual shape of an object to be segmented maintain an actual object surface. As in general surface representations, the two major approaches to represent, track, and deform shapes are either explicit representations, such as triangle meshes, or implicit representations, which are often given in the form of a signed distance field. When shapes are not deformed manually in order to fit a desired object better, they are evolved until they reach a final stable state, e.g., through an energy minimization process. These evolving shapes are often called active shape models, or active contour models. The original concept of active contour models (which are also called snakes) has been presented by Kass et al. [Kass et al. 1987]. Details on active shape models, especially when combined with an implicit representation as in level set methods, are given in section 5.

3.5

Top-Down Approaches

A very promising approach to segmentation of objects that traditionally are hard to segment is to build on pre-exisiting knowledge in the form of a database or atlas of pre-classified datasets that are similar to the target dataset. In practice, these approaches require databases with a large number of datasets, but are a very promising direction that has recently become a very active area of research.

3.6

Hybrid Approaches

A natural way to overcome the shortcomings of a single approach to segmentation is to combine the advantages of multiple methods. A very powerful approach is to combine atlas-based approaches with level set techniques. Leventon employs statistical models to direct the evolution of a surface with local and global priors toward the final segmentation [Leventon 2000]. Zhu [Zhu 1995] presents an image segmentation method that unifies snakes, region growing, and Bayes/MDL (Minimum Description Length) techniques. See the sections below for more examples on combining different approaches to segmentation in order to improve both efficiency and quality of the final results.

4

determine whether new voxels are merged into a region or not. Methods building on diffusion, however, do not track region properties explicitly, but perform diffusion of seeds controlled by partial differential equations. Diffusion filters are usually applied directly to an intensity image [Weickert 1997], but in diffusion-based segmentation approaches the actual diffusion equation is applied to the seed image, incorporating the intensity image as additional term into the equation [Sherbondy et al. 2003]. Suri et al. [Suri and Wu 2002] present a very good recent survey of image segmentation techniques based on diffusion processes modeled with partial differential equations (PDEs), also exploring the relation to level set methods. Pohle et al. [Pohle and T¨ onnies 2002] present an approach to evaluating different segmentation methods for segmenting medical images, which is demonstrated for segmenting the liver in CT data. Zhukov et al. [Zhukov et al. 2001] analyze and exploit the diffusion properties of diffusion weighted MRI for segmentation.

4.1

Seeded Region Growing

Seeded region growing by Adams and Bischof [Adams and Bischof 1994] maintains regions as explicit sets of pixels, and tracks their statistical properties such as the mean intensity. A sequentially sorted list keeps track of candidate pixels for merging into regions. Pixels are sorted according to some metric that measures the difference of a candidate pixel from a region, e.g., the absolute difference between pixel intensity and the mean intensity of the region. This approach ensures that the final regions are as homogeneous as possible given a certain homogeneity criterion. Fiorentini et al. [Fiorentini et al. 2003] present a very simple recent region growing method.

4.2

Nonlinear Diffusion

The seminal work by Perona and Malik [Perona and Malik 1990] introduced the use of nonlinear diffusion processes for image smoothing. The general diffusion equation is a parabolic partial differential equation in divergence

Seeded Region Growing and Diffusion Approaches

Seeded region growing and approaches based on diffusion filtering both belong to the general class of region growing approaches outlined in section 3.3. However, although their basic idea is very similar, in practice these two approaches are very different. The major difference is that seeded region growing approaches explicitly maintain the set of voxels belonging to a given region and track statistical properties of these regions. These properties

Figure 3: [Diewald et al. 2001]. Top row images show noisy echocardiographical data of the left ventricle of the heart and three steps of anisotropic level set evolution. Bottom row images show isotropic Perona Malik diffusion (left), anisotropic Perona Malik diffusion (center), and mean curvature motion (right) as presented in their paper.

form [Weickert 1997]: ∂u/∂t = div(D · ∇u)

(1)

The change of the image values u over time t depends on the diffusion tensor D, and the gradient of the image ∇u. Common classifications of this kind of diffusion include linear diffusion (where D does not depend on the image itself, i.e., is constant over time t), non-linear diffusion (where D depends on the evolution of the image, e.g., its gradient magnitude at time t), isotropic diffusion (where D is a scalar and the diffusion thus in direction of ∇u), anisotropic diffusion (where D is a general matrix, i.e., a general linear transformation of ∇u determines the diffusion direction), homogeneous diffusion (where D is constant over the entire image u), and inhomogeneous diffusion (where D varies with spatial location). However, the actual meaning of these classifications often depends on the community in which they are used [Weickert 1997]. One of the problems of the original nonlinear diffusion is that it is not able to handle large amounts of noise [Suri and Wu 2002]. Multi-channel anisotropic diffusion introduced by Gerig et al. [Gerig et al. 1992], alleviates many of these shortcomings. Weickert [Weickert 1997] introduced tensor non-linear anisotropic diffusion. Other well-known works on anisotropic diffusion include Catte et al [Catte et al. 1992a; Catte et al. 1992b], and Sapiro et al. [Sapiro et al. 1994; Sapiro and Caselles 1997], as well as Alvarez et al. [Alvarez et al. 1992]. Diewald et al. [Diewald et al. 2001] compare several anisotropic diffusion models, and present an early implementation on graphics hardware. They illustrate diverse applications, such as surface smoothing and vector field visualization, including the visualization of principal curvature directions on surfaces. See figure 3. Droske et al. [Droske et al. 2000] present a multilevel segmentation method using anisotropic diffusion as a postprocessing step. Preusser and Rumpf [Preusser and Rumpf 1999] use anisotropic diffusion processes in flow visualization.

[Sherbondy et al. 2003] The method of Sherbondy et al. [Sherbondy et al. 2003] is a GPU implementation of a simple but effective isotropic diffusion model based on the Perona and Malik model [Perona and Malik 1990]. Diffusion is used for image smoothing in a pre-process, as well as computing seed evolution during an interactive segmentation process. The diffusion equation is discretized with an explicit forward Euler approach, which only requires access to six voxel neighbors in the fragment shader. Active voxels are tracked by using morphological dilation on the seed mask, and invoking the fragment shader only for voxels contained in the dilated mask. This approach is called computational masking in the paper, which in reality is implemented using the early z-test of current graphics architectures that allows to excludes pixels from processing depending on the outcome of an early (i.e., before the fragment shader is invoked) comparison between a fragment’s interpolated depth value and the corresponding value stored in the depth buffer. User interaction is enabled via a simple seed painting interface on 2D slices. Rendering is done in a very simple way, using the unfiltered segmentation mask and optionally applying a transfer function, which for all examples demonstrated in the paper is fully opaque.

The seed mask is updated by directly rendering into the slices of a single 3D texture. Voxel values are 32-bit, 16-bit of which contain the voxel density, and the other 16-bit the current seed count of the voxel.

4.3

Medical Applications

Zhukov et al. [Zhukov et al. 2003] segment regions of the brain using anisotropic diffusion properties of brain tissue, as an alternative to segmentation with active contour models. They use a level set approach to remove noise and generate a smooth model as result. Krissian et al. [Krissian et al. 1997] present an anisotropic diffusion process reducing noise and preserving small structures such as vessels. They also include local curvature measures. Pohle and T¨ onnies [Pohle and T¨ onnies 2001a; Pohle and T¨ onnies 2001b] use an adaptive region growing approach for semi-automatic and fully automatic segmentation of CT and MRI data, and determine the homogeneity criterion automatically from characteristics of the region to be segmented. They demonstrate segmenting the liver, the kidneys, and the brain. Hinz et al. [Hinz et al. 2002] use an extension of this adaptive region growing method for segmentation and also generate transfer functions for volume rendering automatically, for which they are using a hybrid approach including isosurfaces. They demonstrate their approach on the liver, liver tumors, the kidneys, and blood vessels.

5

Level Sets and Deformable Surfaces

In contrast to the approaches discussed in the previous section that basically operate at the granularity of individual voxels without any notion of the surface of the segmented object, maintaining and modifying, i.e., deforming, actual object surfaces has many advantages. A comprehensive survey of deformable models in medical image analysis has been presented by McInerney and Terzopoulos [McInerney and Terzopoulos 1996], although they do not consider implicit representations of the model, i.e., level set methods. A very good introduction to level set methods, including their application to image segmentation, can be found in the book by Osher and Fedkiw [Osher and Fedkiw 2003]. Leventon employs statistical models to direct the evolution of a surface with local and global priors toward the final segmentation [Leventon 2000], a method that represents a hybrid of using a priori knowledge obtained from a training set and level set methods.

5.1

Active Contours (Snakes)

Active contours, or snakes [Kass et al. 1987], start with an initial estimate of a curve surrounding the two-dimensional object of interest in the form of a contour model, which is then successively deformed to match the actual target object. The snake model tries to minimize an energy function over this object contour, simultaneously imposing a smoothness constraint via internal forces, and constricting the contour toward the actual object contour via external forces. The idea of active contour models can also be implemented using an implicit representation of the contour curve and updating it via the level set method, see below. Liersch et al. [Liersch et al. 2003] represent shapes by normal displacements from an average contour, and thus reduce

the number of parameters of the corresponding shape representation. Although originally developed for the two-dimensional case, i.e., image segmentation, active contours have also been extended to three dimensions, where they are usually called active surfaces.

5.2

Level Set Methods

Where active contour models maintain an explicit representation of object contours, curves and surfaces can naturally also be represented in implicit form, turning them into deformable isosurfaces. Level set methods represent an implicit surface via a grid of scalar function samples, which most commonly are the signed distances to the surface [Osher and Fedkiw 2003]. In this scalar field, the surface itself is the zero level set of the embedding φ [Osher and Fedkiw 2003]: S = {x|φ(x) = 0}, where φ : R3 7→ R

(2)

Level set methods [Sethian 1982; Osher and Fedkiw 2003; Whitaker et al. 2001a] have established themselves as a very powerful framework for maintaining and updating interfaces, i.e., hyper-surfaces, by solving partial differential equations (PDEs). Surface motion is the result of updating the distance values to the surface (φ(x)) according to a speed function v that moves the surface into the direction of the implicit surface normal, i.e., the gradient of the scalar field: ∂φ/∂t = −∇φ · v

(3)

Common speed functions v include the local curvature of the model [Malladi and Sethian 1999]. In segmentation with level set methods, a central part of the speed function is a measure for intensity differences between adjacent voxels in the original volume [Osher and Fedkiw 2003]. For rendering, the actual surface must be extracted or rendered directly from the sampled implicit representation, e.g., generating a mesh via marching cubes or one of its variants, or rendering a view of the surface without generating any geometry via ray-casting. A very important difference of level set methods and other deformable surface models, including snakes, is that surfaces represented implicitly can easily change their topology, e.g., break apart or merge, and develop sharp features. In each step, only the scalar sample values are updated, possibly changing the surface’s topology implicitly. A major drawback of level set methods is that the quality of the resulting segmentation often depends significantly on parameter tuning [Lefohn et al. 2003c], a problem which can be tackled by computing and visualizing the level set solution interactively. Preusser and Rumpf [Preusser and Rumpf 2002] present a multiscale method for nonlinear diffusion of 3D geometry. Weeratunga and Kamath [Weeratunga and Kamath 2004] evaluate the use of implicit active contours for image segmentation and compare them with traditional edge detectors. Whitaker et al. [Whitaker et al. 2001b] present the application of level set methods to the segmentation of biological data sets. Leventon et al. [Leventon et al. 2000] combine intensity and curvature information obtained from a training set with level set evolution in order to reach the final segmentation. Level set topology can be controlled with sparse information added to the volume in order to make use of surface genus information known a priori [Bischoff and Kobbelt

Figure 4: [Goldenberg et al. 2002] detect the inner (center) and the outer (right) boundary of the gray matter of the brain (left), and initialize two coupled level sets from these boundaries. The coupled surfaces are then evolved simultaneously.

2003a; Bischoff and Kobbelt 2004; Bischoff and Kobbelt 2002]. Westermann et al. [Westermann et al. 2000] were using a levelset approach for flow visualization.

Narrow-Band Approaches The major approach to improving the performance of level set methods considerably is the concept of a narrow band of computation surrounding the current position of the zero level set [Osher and Fedkiw 2003]. Since voxels outside this band cannot become relevant for the zero level set in the next iteration, the level set PDEs are only solved in this band of voxels. As soon as the zero level set leaves the band, the set of voxels belonging to it is updated. Lefohn et al. [Lefohn et al. 2003c] present a streaming narrow-band algorithm, i.e., a narrow-band level set solver on the stream architecture of a GPU.

[Lefohn et al. 2003] The level set solver by Lefohn et al. [Lefohn et al. 2003b; Lefohn et al. 2003c] uses a streaming approach on a GPU, and attains 15 times faster evaluation of level set evolution than possible in previous optimized narrow-band solvers on the CPU. See figure 1 for example images produced with their system. Crucial to the performance of their solver is an efficient memory management scheme, which packs the volume into subtiles of 2D textures, and performs all computations in this packed representation. This tile-based representation is used directly for rendering. A related paper [Lefohn et al. 2003a] presents the results of a user study, where the relative performance of manual segmentation and interactive segmentation using their level set solver have been compared. Their tool offers three parameters controlling the level set evolution, and they state that in a typical five minute interactive segmentation session, the model parameters are modified between 10 and 30 times. An earlier technical report contains details on their GPU implementation of curvature flow [Lefohn and Whitaker 2002].

5.3

Medical Applications

Goldenberg et al. [Goldenberg et al. 2002] propagate two coupled bounding surfaces in an active surface model for segmentation of cortical gray matter. See figure 4.

Zeng et al. [Zeng et al. 1999] segment and measure the cortex from MRI data using two coupled surfaces tracked with the level set method. See figure 5. Zhu and Tian [Zhu and Tian 2003] combine non-linear diffusion, the watershed transformation, and subsequent level set evolution using a modified fast marching method for segmentation of the white matter of the brain, the knee joint, and aortic aneurysms. Ho et al. [Ho et al. 2002] present an automatic approach for brain tumor segmentation in MRI scans using a level set method, and present comparisons with manual expert segmentation. See figure 6. Pichon et al. [Pichon et al. 2003] combine a statistical model with surface evolution for semi-automatic segmentation of the brain and the left ventricle of the heart. Montagnat et al. [Montagnat et al. 2003] use a combination of anisotropic diffusion and a model-based method for segmenting 4D (time-dependent) echocardiographic data. Neubauer et al. have used and adapted a snake model for effective segmentation of the myocardium of the left ventricle of the heart [Neubauer and Wegenkittl 2003b; Neubauer and Wegenkittl 2003c; Neubauer and Wegenkittl 2003a]. Wang et al. [Wang et al. 2004] track the deformation of aorta walls caused by drug stimulation over a time series using a time sequence snake model. Kovacevic et al. [Kovacevic et al. 1999] used an active contour model with a level set evolution for segmenting the

abdominal aorta in CT scans. A similar approach has been used by Loncaric et al. [Loncaric et al. 2000]. Magee et al. [Magee et al. 2001a] combine mesh-based deformable models with a level set approach for segmentation of abdominal aortic aneurysms (AAA) [Magee et al. 2001b]. They use this combined approach in order to segment renal and coliac arteries that are too small for a proper segmentation with the mesh-based model alone. Hinz et al. [Hinz et al. 2001] segment vessels in DSA (digital subtraction angiography) using an active double contour model and exploit a vessel template. Pohle et al. [Pohle et al. 1996] use active contour models to segment and detect skin tumours in ultrasound images. Pohle et al. [Pohle et al. 2003] present a two-step approach to segmentation of the liver and kidneys. They initialize an active surface model with a coarse region-based segmentation from an Image Foresting Transformation [Falcao et al. 2000], and obtain the final segmentation by evolution of this active surface. Pekar et al. [Pekar et al. 2001] deform meshes for medical segmentation using a shape model-based approach. Droske et al. [Droske et al. 2001] present an adaptive multigrid level set approach for medical image processing in a neurological setting. Watanabe et al. [Watanabe et al. 2003] combine simulation of fluid flow and segmentation in a level set framework to correct flow data obtained from magnetic resonance phase contrast (MR-PC) angiography. The couple an incompressible Navier-Stokes solver and a level set segmentation method, and observe significant improvements in the computed blood velocity field.

6

Volume Pre-Processing and Filtering

This section summarizes approaches for pre-processing volumes, and especially filtering, in order to improve the quality of the final segmentation. That is, before the actual segmentation algorithm is executed for a volume, it is often pre-filtered in order to reduce artifacts due to noise in the original volume data, e.g., a CT or MRI scan.

Figure 5: [Zeng et al. 1999] start two simultaneous level set evolutions be manually placing two sets of spheres inside the brain. The two level sets are evolved concurrently to approach the shape of the other and inner cortical surfaces, respectively.

Figure 6: [Ho et al. 2002] compare the result of their level set segmentation with ground truth obtained manually. Manual segmentaion is shown in red, automatic in white. The bottom row images illustrate the surface distance of the two kinds of segmentations.

6.1

Volume Pre-Filtering

The most common approach to filtering images or volumes is to perform convolution of the original data with a filter kernel, e.g., a Gaussian filter for smoothing. Since convolution is a very regular operation, it can easily be performed on graphics hardware [Hopf and Ertl 1999]. In contrast to convolution, which is a linear operation, non-linear, e.g., edge-preserving, filters can also be applied. Work on using non-linear filters on GPUs has been presented recently [Viola et al. 2003].

In GPU-based segmentation, this approach can be used to limit the pixels for which a PDE is actually evaluated [Sherbondy et al. 2003]. In GPU-based ray casting [Kr¨ uger and Westermann 2003a], computational masking (the early z-test) is used to turn off rays that have reached a given opacity threshold or hit and isosurface. In the context of solving large but sparse linear systems [Kr¨ uger and Westermann 2003a], computational masking is used to mask out empty elements of matrices.

Morphological Operators

7.2

Filtering operators from mathematical morphology, e.g., erosion and dilation, are a standard ingredient of many segmentation algorithms. H¨ ohne and Hanson [H¨ ohne and Hanson 1992] perform interactive segmentation using morphological operations, e.g., of the brain and the left ventricle of the heart. Morphological operations can also be performed entirely on GPUs [Hopf and Ertl 2000a].

6.2

Other Pre-Processing Operations

Apart from filtering, other pre-processing tasks such as distance and wavelet transforms can also be performed on GPUs. Museth et al. [Museth et al. 2002] generate a high-quality level set volume from multiple input volumes.

Distance Transforms Transforming a polygonal mesh into a volumetric distance field can be done very efficiently on GPUs [Sigg et al. 2003; Sud et al. 2004], since it is essentially a rasterization operation. In the context of segmentation, these approaches are relevant in order to transform an initial coarse segmentation given as a polygonal mesh into an implicit representation for subsequent evolution of the surface toward the final high-quality segmentation result. Distance fields can also be approximated instead of stored for an entire volume, e.g., the piecewise linear approximation presented by Wu and Kobbelt [Wu and Kobbelt 2003].

Wavelet Transforms Hopf and Ertl have shown how to perform wavelet transforms in the Haar basis on graphics hardware [Hopf and Ertl 2000b].

7

Implementation Issues

This section reviews several important implementation issues that are shared by all GPU-based approaches.

7.1

Computational Masking

The term computational masking describes avoiding to execute a stream kernel [Kapasi et al. 2002] for stream elements that are not an active part of the current computation. In GPU-based approaches this usually means avoiding to invoke the fragment shader, which implements the stream kernel, for fragments that are not active. A simple example is disabling rays in GPU-based ray tracing [Purcell et al. 2002] that have already been traced to a sufficient extent, while still continuing to trace other rays.

Memory Management

An important issue in all GPU-based approaches dealing with general computations, especially when the computational domain consumes a lot of memory, is memory management [Lefohn et al. 2003b; Lefohn et al. 2003c]. There are two major issues in memory management. First, in order to avoid computations for inactive parts of the volume, a tile-based approach can be used to effectively skip large inactive parts [Lefohn et al. 2003c]. Second, a natural issue is dealing with large data, by blocking or bricking the volume and swapping in active areas on demand.

8

Rendering

In interactive segmentation, the current as well as the final result of the segmentation can be viewed immediately in a volume rendering view. Tiede et al. [Tiede et al. 1998] render attributed, i.e., segmented, volume data with high quality using ray casting. High-quality rendering of segmented data has recently become possible on GPUs [Hadwiger et al. 2003a]. Such a 3D view can show the isosurface of the segmented object itself [Sherbondy et al. 2003], the segmentation mask overlayed on top of a direct volume rendering [Lefohn et al. 2003b], or render segmented objects with different optical properties such as transfer functions and rendering modes [Hadwiger et al. 2003a]. High-quality direct rendering of isosurfaces [Sigg et al. 2004] is a natural rendering front-end for level set solvers. Especially in dealing with large data, compressing the volume and the segmentation mask is a promising approach [Schneider and Westermann 2003]. Moreover, for high-quality display it is also possible to use cubic or other higher-order filter kernels for reconstruction purposes [Hadwiger et al. 2001; Hadwiger et al. 2002].

Isosurface extraction Isosurface extraction can be done with controlled topology [Gerstner and Pajarola 2000]. Bischoff and coworkers perform topology control with sparse information added to the volume in order to make use of surface genus information known a priori [Bischoff and Kobbelt 2003a; Bischoff and Kobbelt 2004; Bischoff and Kobbelt 2002]. They also achieve sub-voxel topology control [Bischoff and Kobbelt 2003b]. Bischoff and Kobbelt [Bischoff and Kobbelt 2004] extract topologically correct isosurfaces from a brain segmentation using the level set method.

9

Conclusions

Medical image and volume segmentation is a fundamental part of medical image analysis, and although many automatic and semi-automatic segmentation algorithms have been developed, in clinical practice segmentation is still often done manually by skilled operators. Unfortunately, this is a very time-consuming and tedious process. A very promising approach to tackling this problem and establishing semiautomatic segmentation is interactive segmentation, which has only become possible very recently due to its high computational complexity. In these approaches, the segmentation can be observed and guided while it is being computed, which can reduce overall segmentation time significantly. On the other hand, the recent introduction of full floating point support in programmable consumer graphics architectures (GPUs) has created an entire new field of general purpose computations on these architectures, i.e., algorithms that do not perform rendering per se. Recently, interactive segmentation tools on GPUs have been developed that demonstrate very promising first results. Especially the efforts of Lefohn et al. [Lefohn et al. 2003b; Lefohn et al. 2003c] with a level set approach, and the work of Sherbondy et al. [Sherbondy et al. 2003] with a nonlinear diffusion approach, have shown that these approaches are a very promising alternative to previous methods. Both of these approaches build on the evaluation of partial differential equations (PDEs) in a stream processing model.

Future directions In the future, hybrid approaches seem to be an especially interesting alternative. A very promising approach appears to be the computation of an initial estimate of the segmentation from a priori knowledge, i.e., a data set of previous segmentations of similar data sets. This coarse segmentation can then be refined using the level set method, for instance. Leventon et al. [Leventon et al. 2000] have successfully used such an approach by obtaining intensity and curvature priors from a level set, and evolving it further using a level set approach. Another possibility is obtaining a coarse segmentation from region growing, and refining it using an active surface model [Pohle et al. 2003].

Acknowledgments

Bischoff, S., and Kobbelt, L. 2002. Isosurface reconstruction with topology control. In Proceedings of Pacific Graphics 2002, 246–255. Bischoff, S., and Kobbelt, L. 2003. Snakes with topology control. The Visual Computer . Bischoff, S., and Kobbelt, L. 2003. Sub-voxel topology control for level set surfaces. In Proceedings of EUROGRAPHICS 2003, 273–280. Bischoff, S., and Kobbelt, L. 2004. Topologically correct extraction of the cortical surface of a brain using level-set methods. In Proceedings of BVM 2004. ¨ der, P. Bolz, J., Farmer, I., Grinspun, E., and Schro 2003. Sparse matrix solvers on the gpu: Conjugate gradients and multigrid. In Proceedings of SIGGRAPH 2003, 917–924. Canny, J. 1986. A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 8, 6. Catte, F., Lions, P.-L., Morel, J., and Coll, T. 1992. Image selective smoothing and edge detection by nonlinear diffusion - i. SIAM Journal of Numerical Analysis 29, 1, 182–193. Catte, F., Lions, P.-L., Morel, J., and Coll, T. 1992. Image selective smoothing and edge detection by nonlinear diffusion - ii. SIAM Journal of Numerical Analysis 29, 3, 845–866. Diewald, U., Preusser, T., Rumpf, M., and Strzodka, R. 2001. Diffusion models and their accelerated solution in image and surface processing. Acta Mathematica Univeritatis Comenianae 70, 1. Droske, M., Preußer, T., and Rumpf, M. 2000. A multilevel segmentation method. In Proceedings of Vision, Modeling, and Visualization 2000, 327–336. Droske, M., Meyer, B., Rumpf, M., and Schaller, K. 2001. An adaptive level set method for medical image segmentation. Tech. rep., Universit¨ at Bonn.

This work has been done in the Effective Medical Visualization Group at the VRVis Research Center, which is funded in part by the Austrian Kplus project.

Engel, K., Hadwiger, M., Kniss, J., and Rezk-Salama, C. 2002. High-Quality Volume Graphics on Consumer PC Hardware. Course Notes for Course #42 at SIGGRAPH 2002, ACM SIGGRAPH.

References

Falcao, A., Lotufo, R., and Araujo, G. 2000. The image foresting transformation. Tech. Rep. Relatorio Tecnico IC-00-12.

Adams, R., and Bischof, L. 1994. Seeded region growing. IEEE Transactions on Pattern Analysis and Machine Intelligence 16, 6, 641–647. Alvarez, L., Lions, P.-L., and Morel, J.-M. 1992. Image selective smoothing and edge detection by nonlinear diffusion. SIAM Journal of Numerical Analysis 29, 3, 845– 866. Beucher, S. 1991. The watershed transformation applied to image segmentation. In 10th Pfefferkorn Conf. on Signal and Image Processing in Microscopy and Microanalysis, 16–19.

Felkel, P. 2000. Segmentation of vessels in peripheral cta datasets. Tech. Rep. TR-VRVis-2000-008, VRVis Research Center. Felkel, P., Wegenkittl, R., and Bruckschwaiger, M. 2001. Implementation and complexity of the watershedfrom-markers algorithm computed as a minimal cost forest. In Proceedings of Eurographics 2001, 26–35. Fiorentini, S., Larrabide, I., and Venere, M. 2003. A simple 3d image segmentation technique over medical data. In Proceedings of SIS 2003.

¨bler, O., Kikinis, R., and Jolesz, F. Gerig, G., Ku 1992. Nonlinear anisotropic filtering of mri data. IEEE Transactions on Medical Imaging 11, 2, 221–232. Gerstner, T., and Pajarola, R. 2000. Topology preserving and controlled topology simplifying multiresolution isosurface extraction. In Proceedings of IEEE Visualization 2000, 259–266. Goldenberg, R., Kimmel, R., Rivlin, E., and Rudzsky, M. 2002. Cortex segmentation: A fast variational geometric approach. IEEE Transactions on Medical Imaging 21, 2, 1544–1551. ¨ ller, Hadwiger, M., Theußl, T., Hauser, H., and Gro E. 2001. Hardware-accelerated high-quality filtering on PC hardware. In Proc. of Vision, Modeling, and Visualization 2001, 105–112. Hadwiger, M., Viola, I., Theußl, T., and Hauser, H. 2002. Fast and flexible high-quality texture filtering with tiled high-resolution filters. In Proceedings of Vision, Modeling, and Visualization 2002, VMV’02, 155–162. Hadwiger, M., Berger, C., and Hauser, H. 2003. Highquality two-level volume rendering of segmented data sets on consumer graphics hardware. In Proceedings of IEEE Visualization 2003, 301–308. ¨ ller, T. 2003. QualHadwiger, M., Hauser, H., and Mo ity issues of hardware-accelerated high-quality filtering on pc graphics hardware. In Proceedings of WSCG 2003, 213–220. Harris, M. 2003. General Purpose Computations on GPUs (GPGPU) web page. http://www.gpgpu.org/. Hillesland, K., Molinov, S., and Grzeszczuk, R. 2003. Nonlinear optimization framework for image-based modeling on programmable graphics hardware. In Proceedings of SIGGRAPH 2003, 925–934. ¨ nnies, Grohmann, M., and Pohle, R. Hinz, M., K., To 2001. An active double-contour for segmentation of vessels in digital subtraction angiography. In Proceedings of SPIE Medical Imaging 2001, 1554–1562. ¨ nnies, K. 2002. Hinz, M., Pohle, R., Shin, H., and To Region-based interactive 3d image analysis of structures in medical data by hybrid rendering. In Proceedings of SPIE (Visualization, Image-Guided Procedures, and Display) 2002, 388–395. ¨ hne, K. H., and Hanson, W. 1992. Interactive 3d Ho segmentation of mri and ct volumes using morphological operations. Journal of Computer Assisted Tomography 16, 2, 285–294. Hopf, M., and Ertl, T. 1999. Accelerating 3D convolution using graphics hardware. In Proc. of IEEE Visualization ’99, 471–474. Hopf, M., and Ertl, T. 2000. Accelerating morphological analysis with graphics hardware. In Workshop on Vision, Modelling, and Visualization VMV 2000. Hopf, M., and Ertl, T. 2000. Hardware accelerated wavelet transformations. In Proc. of Eurographics/IEEE TCVG Symposium on Visualization 2000.

Ho, S., Bullitt, E., and Gerig, G. 2002. Level set evolution with region competition: Automatic 3-d segmentation of brain tumors. In Proceedings of 16th International Conference on Pattern Recognition (ICPR) 2002, 532–535. James, G. 2001. Operations for hardware-accelerated procedural texture animation. In Game Programming Gems 2, Charles River Media, 497–509. Kapasi, U. J., Dally, W. J., Rixner, S., Owens, J. D., and Khailany, B. 2002. The imagine stream processor. In Proceedings of IEEE International Conference on Computer Design 2002, 282–288. Kass, M., Witkin, A., and Terzopoulos, D. 1987. Snakes: Active contour models. International Journal of Computer Vision 1, 4. Kirbas, C., and Quek, F. 2003. A review of vessel extraction techniques and algorithms. In Proceedings of IEEE Conference Bio-Informatics and Bio-Engineering 2003. Kovacevic, D., Loncaric, S., and Sorantin, E. 1999. Deformable contour based method for medical image segmentation. In Proceedings of 21st International Conference on Information Technology Interfaces (ITI) ’99. Krissian, K., Malandain, G., and Ayache, N. 1997. Directional anisotropic diffusion applied to segmentation of vessels in 3d images. In Proceedings of Scale-Space Theory in Computer Vision ’97, 345–348. ¨ger, J., and Westermann, R. 2003. Acceleration Kru techniques for gpu-based volume rendering. In Proceedings of IEEE Visualization 2003, 287–292. ¨ger, J., and Westermann, R. 2003. Linear algeKru bra operators for gpu implementation of numerical algorithms. In Proceedings of SIGGRAPH 2003, 908–916. Lefohn, A., and Whitaker, R. 2002. A gpu-based, threedimensional level set solver with curvature flow. Tech. Rep. UUCS-02-017, 2002, University of Utah School of Computing. Lefohn, A., Cates, J., and Whitaker, R. 2003. Interactive, gpu-based level sets for 3d brain tumor segmentation. In Proceedings of Medical Image Computing and Computer Assisted Intervention (MICCAI) 2003. Lefohn, A., Kniss, J., Hansen, C., and Whitaker, R. 2003. Interactive deformation and visualization of level set surfaces using graphics hardware. In Proceedings of IEEE Visualization 2003, 75–82. Lefohn, A., Kniss, J., Hansen, C., and Whitaker, R. 2003. A streaming narrow-band algorithm: interactive computation and visualization of level sets. IEEE Transactions on Visualization and Computer Graphics. Leventon, M. 2000. Statistical Models for Medical Image Analysis. Ph.D. thesis, MIT. Leventon, M., Faugeras, O., Grimson, W., and Wells, W. 2000. Level set based segmentation with intensity and curvature priors. In Proceedings of Workshop on Mathematical Methods in Biomedical Image Analysis 2000, 4–11.

Liersch, D., Sovakar, A., and Kobbelt, L. 2003. Parameter reduction and automatic generation of active shape models. In Workshop Bildverarbeitung f¨ ur die Medizin.

Perona, P., and Malik, J. 1990. Scale-space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence 12, 7.

Lin, Z., Jin, J., and Talbot, H. 2001. Unseeded region growing for 3d image segmentation. In Pan-Sydney Workshop on Visual Information Processing.

Pichon, E., Tannenbaum, A., and Kikinis, R. 2003. A statistically based surface evolution method for medical image segmentation: Presentation and validation. In Proceedings of MICCAI 2003, 711–720.

Loncaric, S., Subasic, M., and Sorantin, E. 2000. 3-d deformable model for aortic aneurysm segmentation from ct images. In World Congress on Medical Physics and Biomedical Engineering.

¨ nnies, K. 2001. A new approach for Pohle, R., and To model-based adaptive region growing in medical image analysis. In Proceedings of 9th Int. Conference on Computer Analysis and Patterns, 238–246.

Magee, D., Bulpitt, A., and Berry, E. 2001. Combining 3d deformable models and level set methods for the segmentation of abdominal aortic aneurysms. In British Machine Vision Conference.

¨ nnies, K. 2001. Segmentation of mediPohle, R., and To cal images using adaptive region growing. In Proceedings of SPIE Medical Imaging 2001, 1337–1346.

Magee, D., Bulpitt, A., and Berry, E. 2001. Level set methods for the 3d segmentation of ct images of abdominal aortic aneurysms. In Medical Image Understanding and Analysis. Malladi, R., and Sethian, J. 1999. Image processing: Flows under min/max curvature and mean curvature. La Cruz, A. 2004. Accuracy evaluation of different centerline approximations of blood vessels. In Proc. of Eurographics/IEEE TCVG Symposium on Visualization 2004. McInerney, T., and Terzopoulos, D. 1996. Deformable models in medical image analysis: A survey. Medical Image Analysis 1, 2. McReynolds, T., Blythe, D., Grantham, B., and Nelson, S. 2000. Advanced graphics programming techniques using OpenGL. In SIGGRAPH 2000 course notes. Montagnat, J., Sermesant, M., Delingette, H., Malandain, G., and Ayache, N. 2003. Anisotropic filtering for model-based segmentation of 4d cylindrical echocardiographic images. Pattern Recognition Letters 24 , 815– 828. Museth, K., Breen, D., Zhukov, L., and Whitaker, R. 2002. Level set segmentation from multiple non-uniform volume datasets. In Proceedings of IEEE Visualization 2002, 179–186. Neubauer, A., and Wegenkittl, R. 2003. Analysis of four-dimensional cardiac data sets using skeleton-based segmentation. In Proceedings of WSCG 2003. Neubauer, A., and Wegenkittl, R. 2003. A skeletonbased inflation model for myocardium segmentation. In Proceedings of Vision Interface 2003. Neubauer, A., and Wegenkittl, R. 2003. Skeleton-based myocardium segmentation. In Proceedings of Visualization and Data Analysis 2003. Osher, S., and Fedkiw, R. 2003. Level Set Methods and Dynamic Implicit Surfaces. Springer Verlag New York. Pekar, V., Kaus, M., Lorenz, C., Lobregt, S., Truyen, R., and Weese, J. 2001. Shape model based adaptation of 3-d deformable meshes for segmentation of medical images. In Proceedings of SPIE Medical Imaging 2001, 281–289.

¨ nnies, K. 2002. A three-level evaluation Pohle, R., and To process for segmentation methods in medical imaging. In Proceedings of SPIE Medical Imaging 2002, 287–298. ¨ ber, T. 1996. Detection Pohle, R., Pereit, S., and Bo of skin tumors in high frequency ultrasound images using active contours. In Proceedings of 3D Image Analysis and Synthesis 1996, 207–212. ¨ nnies, K. 2003. SegmenPohle, R., Behlau, T., and To tation of 3-d medical image data sets with a combination of region based initial segmentation and active surfaces. In Progress in Biomedical Optics and Imaging, Proceedings of the SPIE International Symposium on Medical Imaging: Image Processing, 1225–1231. Preusser, T., and Rumpf, M. 1999. Anisotropic nonlinear diffusion in flow visualization. In Proceedings of IEEE Visualization ’99, 325–332. Preusser, T., and Rumpf, M. 2002. A level set method for anisotropic geometric diffusion in 3d image processing. SIAM Journal of Applied Mathematics 62, 5, 1772–1793. Purcell, T., Buck, I., Mark, W., and Hanrahan, P. 2002. Ray tracing on programmable graphics hardware. In Proceedings of SIGGRAPH 2002, 703–712. Rost, R. 2004. OpenGL Shading Language. Addison Wesley. Rumpf, M., and Strzodka, R. 2001. Level set segmentation in graphics hardware. In Proceedings of International Conference on Image Processing (ICIP) 2001, 1103–1106. Rumpf, M., and Strzodka, R. 2001. Nonlinear diffusion in graphics hardware. In Proc. of Eurographics/IEEE TCVG Symposium on Visualization 2001. Sapiro, G., and Caselles, V. 1997. Contrast enhancement via image evolution flows, graphical models and image processing. Graphical Models and Image Processing 59, 6, 407–416. Sapiro, G., Tannenbaum, A., You, Y., and Kaveh, M. 1994. Experiments on geometric image enhancement. In Proceedings of First IEEE International Conference on Image Processing. Schneider, J., and Westermann, R. 2003. Compression domain volume rendering. In Proceedings of IEEE Visualization 2003, 293–300.

Sethian, J. 1982. An analysis of flame propagation. Ph.D. Thesis University of California at Berkeley. Sezgin, M., and Sankur, B. 2004. Survey over image thresholding techniques and quantitative performance evaluation. Journal of Electronic Imaging 13, 1, 146–165. Sherbondy, A., Houston, M., and Napel, S. 2003. Fast volume segmentation with simultaneous visualization using programmable graphics hardware. In Proceedings of IEEE Visualization 2003, 171–176. Sigg, C., Peikert, R., and Gross, M. 2003. Signed distance transform using graphics hardware. In Proceedings of IEEE Visualization 2003, 83–90. ¨hler, K. Sigg, C., Hadwiger, M., Gross, M., and Bu 2004. Real-time high-quality rendering of isosurfaces. Tech. Rep. TR-VRVis-2004-015, VRVis Research Center. Stoev, S., and Strasser, W. 2000. Extracting regions of interest applying a local watershed transformation. In Proceedings of IEEE Visualization 2000, 21–28.

Wang, J., Ji, L., Lin, X.-S., and Ma, H. 2004. Tracking deforming aortas in two-photon autofluorescence images and its application on quantitative evaluation of aortarelated drugs. Computerized Medical Imaging and Graphics 28 , 51–59. Watanabe, M., Kikinis, R., and Westin, C.-F. 2003. Level set-based integration of segmentation and computational fluid dynamics for flow correction in phase contrast angiography. Academic Radiology 10, 12, 1416–1423. Weeratunga, S., and Kamath, C. 2004. An investigation of implicit active contours for scientific image segmentation. In Proceedings of Visual Communications and Image Processing Conference 2004. Weickert, J. 1997. A review of nonlinear diffusion filtering. Scale Space Theories in Computer Vision, 3–28. Westermann, R., Johnson, C., and Ertl, T. 2000. A level-set method for flow visualization. In Proceedings of IEEE Visualization 2000, 147–154.

Strzodka, R. 2002. Virtual 16 bit precise operations on rgba8 textures. In Proceedings of Vision, Modeling, and Visualization 2002, 171–178.

Whitaker, R., Breen, D., Museth, K., and Soni, N. 2001. A framework for level set segmentation of volume datasets. In Proceedings of the International Workshop on Volume Graphics 2001, 159–168.

Strzodka, R., and Telea, A. 2004. Generalized distance transforms and skeletons in graphics hardware. In Proc. of Eurographics/IEEE TCVG Symposium on Visualization 2004.

Whitaker, R., Breen, D., Museth, K., and Soni, N. 2001. Segmentation of biological volume datasets using a level set framework. In Proceedings of the International Workshop on Volume Graphics 2001, 249–263.

Strzodka, R., Droske, M., and Rumpf, M. 2003. Fast image registration in dx9 graphics hardware. Medical Informatics and Technologies 6, 43-49.

Witkin, A. 1983. Scale space filtering. In Proceedings of International Joint Conference on Artificial Intelligence ’83.

Strzodka, R., Ihrke, I., and Magnor, M. 2003. A graphics hardware implementation of the generalized hough transform for fast object recognition, scale, and 3d pose detection. In Proceedings of International Conference on Image Analysis and Processing 2003, 188–193.

Wu, J., and Kobbelt, L. 2003. Piecewise linear approximation of signed distance fields. In Proceedings of Vision, Modeling, and Visualization 2003, 513–520.

Sud, A., Otaduy, M., and Manocha, D. 2004. Difi: Fast 3d distance field computation using graphics hardware. In Proceedings of EUROGRAPHICS 2004. Suri, J., and Wu, D. 2002. A comparison of stateof-the-art diffusion imaging techniques for smoothing medical/non-medical image data. In Proceedings of 16th International Conference on Pattern Recognition (ICPR) 2002. ¨ hne, K. H. 1998. High Tiede, U., Schiemann, T., and Ho quality rendering of attributed volume data. In Proceedings of IEEE Visualization ’98, 255–262. Tzeng, F.-Y., Lum, E., and Ma, K.-L. 2003. A novel interface for higher-dimensional classification of volume data. In Proceedings of IEEE Visualization 2003, 505– 512. Udupa, K., and Herman, G. Medicine. CRC Press.

1999.

3D Imaging in

¨ ller, M. 2003. Viola, I., Kanitsar, A., and Gro Hardware-based nonlinear filtering and segmentation using high-level shading languages. In Proceedings of IEEE Visualization 2003, 309–316.

Yang, R., and Welch, G. 2003. Fast image segmentation and smoothing using commodity graphics hardware. Journal of Graphics Tools 7, 4, 91–100. Zeng, X., Staib, L., Schultz, R., and Duncan, J. 1999. Segmentation and measurement of the cortex from 3d mr images using coupled surfaces propagation. IEEE Transactions on Medical Imaging 18, 10. Zhu, S. 1995. Region competition: unifying snakes, region growing, and bayes/mdl for multi-band image segmentation. Tech. Rep. 94-10, Harvard Robotics Laboratory. Zhukov, L., Museth, K., Breen, D., and Whitaker, R. 2001. 3d modeling and segmentation of diffusion weighted mri data. In Proceedings of SPIE Medical Imaging 2001, 401–412. Zhukov, L., Museth, K., Breen, D., Whitaker, R., and Barr, A. 2003. Level set modeling and segmentation of dt-mri brain data. Journal of Electronic Imaging 12, 1, 125–133. Zhu, F., and Tian, J. 2003. Medical image segmentation using level set and watershed transform. In Proceedings of SPIE Advanced Biomedical and Clinical Diagnostic Systems 2003, 294–302.