Transfer Function Specification for the Visualization of Medical Data Silvia Castroz , Andreas Ko¨ nigy , Helwig L¨offelmanny, and Eduard Gr¨ollery y Institute of Computer Graphics
z Dpto.
de Cs. de la Computacion Universidad Nacional del Sur Avda. Alem 1253 8000-Bahia Blanca, Argentina
Vienna University of Technology Karlsplatz 13/186/2, A-1040 Wien, Austria
http://www.uns.edu.ar/WebDCC/
[email protected]
f koenig j helwig j groeller g @cg.tuwien.ac.at
Abstract The application of transfer functions to map data values to visual properties as, e.g., color and opacity, is a crucial step in direct volume rendering. Due to the complex relationship between the transfer function and the resulting image it is usually extremely difficult to model an appropriate mapping. In this paper we present an advanced transfer function specification scheme for the visualization of medical data. The concept of metadata is used to make the modeling of transfer functions more intuitive. A small number of parameters is sufficient to completely describe a transfer function, thus this specification scheme is suitable for (semi-)automated search techniques. Keywords: visualization of medical data, visualization systems, multi-modal and multi-dimensional data visualization, volume rendering.
1 Introduction The visualization of volumetric data has aided many scientific disciplines ranging from material technology to the biomedical sciences. Advanced scanning procedures, such as magnetic resonance imaging (MRI) and computed tomography (CT) are widely used for applications of non-destructive object inspection. In these applications the user typically investigate the 3D structure by sequentially examining cross sections. He has to do a 3D reconstruction of the data mentally. This is a difficult task and it is obvious that in this case it is desirable to create three dimensional views of the object from the 2D slices. Besides surface-based techniques, direct volume rendering has emerged as an important approach in this case. The need to visualize the acquired data has given rise to a wide variety of volume visualization techniques [8, 9, 10, 15, 17, 18] and the development of a number of specialized visualization systems [1, 2, 4, 6]. In spite of the progress and the advantages of direct volume visualization, its use is not widespread and it has not been integrated in many of the commercial systems. In the context of volume visualization, transfer functions that assign optical properties, such as color and opacity, to data values, are of particular importance to the outcome of direct volume rendering. Transfer functions have been applied in volume visualization [7, 14, 16] but there are no suggestions on how to determine transfer functions in an intuitive and interactive manner. The reasons are:
There is a complex interrelationship between the transfer function and the resulting image. The accumulation process in the rendering phase is difficult to imagine. Thus, the transfer function usually cannot be deduced from an idea of a suitable result image. This dependency on the dataset and the rendering algorithm makes the transfer-function specification challenging.
http://www.cg.tuwien.ac.at/home/
The set of the transfer functions is potentially infinitek dimensional, since they are mappings [0; 1] [0; 1] . Therefore, automated techniques based on exhaustive search do not work in this case.
!
There is a significant lack of interactivity with direct volume rendering. Thus, it is usually extremely difficult to prototype a transfer function with many iterations.
For this reasons, the specification of an appropriate transfer function is a really hard problem. Certain types of volume data (like CT data) are standardized in the range of data values. Special subranges do always correspond to the same type of tissue. In this cases it would be sufficient to find one useful transfer function and use it for all data sets of this type. When other scanning procedures are used (like MRI), the data ranges differ. Depending on the patient, environmental circumstances, and certain machine parameters the scanning procedure produces data sets with different distributions. In order to create useful images from these data sets, a different transfer function has to be found for every single data set.
2 Related Work Various approaches to these problems can be found in the literature. One class of solutions proposes search algorithms in huge search spaces. Others techniques are based on the reduction of the search space itself. One approach is to treat this task as a parameter optimization problem [5]. Methods developed for global optimization like genetic algorithms or simulated annealing are used. Initial solutions are optimized by local improvements until an optimal solution is found. The initial population of transfer functions can be generated randomly, provided by the user, or taken from a library of some typical functions. Starting from this initial population of transfer functions, the evolution is controlled by the user or automatically by the system. With these approaches, the user or the algorithm respectively interacts only with the final images. The authors propose, that it may be beneficial to have an intelligent selection of the initial transfer functions in order to obtain a better convergence speed. “Design Galleries” [11] are another method for computer assisted parameter setting. The user is presented a broad selection of automatically generated images. These are depicted by perceptually different icons, that can be produced by varying a given inputparameter vector. The basic strategy is to extract a subset with optimal coverage from the set of all possible images. The construction phase of the gallery is typically computationally expensive and has to be done off-line. After the gallery is built, the user is able to interactively browse through the space of obtained images. Other approaches [3, 12] use rule-based systems to determine appropriate transfer functions. Potentially useful transfer functions
Figure 2: Three component functions CF i , three weights the transfer function TF as weighted sum. Figure 1: Identifying characteristic features on the data range of the volume data set and building a appropriate transfer function with the use of component functions
are chosen from a database, which contains a variety of predefined transfer functions, represented as color maps. The selection of the transfer function is based on a set of perceptual rules, derived from the structure of the data or the goals of the visualization.
3 Specifying transfer functions using metadata information In order to solve some of the presented problems, we focus on the specification of a transfer function first. We adhere to the proposal [12], that suggests that some tasks in visualization are simplified when the user is allowed to work on a higher level of abstraction of the data. We use the term metadata to characterize higher-level attributes of the data, as was established in the context of visualization [13]. This concept of metadata defines a higher-level characterization of the data which provides a higher level interface to the user. Representing the data in terms of these higher-level abstractions provides a more intuitive interface to the visualization process. In medical applications, for example, the metadata information would be the characterization of different tissues like bone, skin and muscle. For a natural and intuitive way of specification the user may select specific features in the data, which represent different tissue types. These component functions can be defined over different ranges of data values. They will define the transfer function TF (see Fig. 1) when combined in an appropriate way. The next section discusses details on the definition of the transfer function by means of component functions. Details on the definition of the component function will be given in Section 3.2. Due to the small number of parameters needed for the specification of transfer functions by component functions, two useful possibilities can be exploited:
A very simple and intuitive user interface for the definition of component functions can be built (discussed in Sect. 3.4). As the dimensionality of the parameter space is reduced, semi-automated search strategies can be used (discussed in Sect. 3.6).
Finally the results obtained by a software prototype implementation of the presented concepts will be given in Section 4.
wi , and
3.1 Definition of the transfer function A transfer function TF is specified by a sum of n component functions CF i (see Fig. 2).
TF (d) =
n X i=1
wi (d)CF i (d)
A detailed description of the component functions is given in the next section. The support of a component function CF i is specified by values Sa i Sb i Sc i Sd i :
CF i (d) 6= 0 , Sa i < d < Sd i
The support is divided into two regions: inner support and extended support. The inner support of a component function is defined by [Sb i ; Sc i ], the extended support by [Sa i ; Sb i ) (Sc i ; Sd i ]. By definition, the inner support regions of the component functions must not overlap:
[
; Sc ] \ [Sb ; Sc ] = ; , i 6= j 8i; j 2 [1; n] The n weighting functions wi are defined in a way that their sum [Sbi
i
j
j
evaluates to 1 on the range of the transfer function:
n X i=1
wi (d) = 1; 8d 2 [0; 1]
In order to have the possibility to model the transfer function locally, the weighting function must not change the impact of the component function at the range of the inner support. Therefore, it is defined to evaluate to 1 at this range. More details on the weighting functions will be give in Section 3.3. Evaluating the transfer function for a certain data value d in the defined range [0; 1] gives the corresponding opacity value. The color value is determined in a similar fashion. Each component function is assigned a specific color value. Interpolation is done in RGB-space using the weighting functions wi :
Col (d) =
n X i=1
wi (d)Col i (d)
The user defines just a small number of parameters for the component functions. The weighting functions can be derived implicitly from these parameters, as it will be shown in Section 3.3. Therefore, this reduced number of parameters does not only provide a simple interface for the user, but also allows the usage of semiautomated exhaustive search strategies (see Sect. 3.6).
Figure 3: Parameters of a component function
3.2 Defining the component functions A component function CF i (d) can be characterized by a small number of parameters (see Fig. 3). As stated before, the inner support is bounded by [Sb i ; Sc i ]. The outer support is defined by [Sa i ; Sb i ) (Sc i ; Sd i ]. The height of the component function is defined by amplitude Ai . For each component a color Col i is defined. The geometric continuity of CF i (d) is defined at Sa i ; Sb i ; Sc i ; Sd i by Ca i ; Cb i ; Cc i ; Cd i , respectively. Cx i may take the discrete values of 0 or 1, corresponding to local C 0 or C 1 properties of CF i . A component function CF i for assigning of opacity values to data values d is defined as follows:
[
8> 0 >< Ai c i (d) CF i (d) = Ai >> Ai cfr i(d) : 0
0 d < Sa i Sa i d < Sb i Sb i d Sc i Sc i < d Sd i Sd i < d 1
Figure 4: Definition of the continuity of the component functions can be used for fine tuning
transitions can be seen in Figure 4. These special properties can be used to fine tune the appearance of images rendered with the transfer function. With our definition of the parameters of a component function, a variety of shapes can be modeled: When the inner support is equal to the extended support, the component function has the shape of a box function.
If the inner support collapses, the component function has the shape of a tent.
The functions c i and cfr i are defined as polynomials of minimal degree, which fulfill the following constraints:
c i (Sa i ) = 0 cfr i (Sc i ) = 1 c i (Sb i ) = 1 cfr i (Sd i ) = 0 c 0i (Sa i ) = 0 if Ca i = 1 cfr 0i (Sc i ) = 0 if Cc i = 1 c 0i (Sb i ) = 0 if Cb i = 1 cfr 0i (Sd i ) = 0 if Cd i = 1
If Cx i is set, the slope of the function has to be 0 at point xi 2 fSa i ; Sb i ; Sc i ; Sd i g. If it is not set, every other slope (defined by the linear connection to the neighboring points) is allowed. This yields the geometric continuity of the component function to be C 0 or C 1 . Therefore c i and cfr i are characterized by polynomials of degree 1, 2, or 3. More sophisticated methods for the specification of the detailed shape of the component function could be integrated, but using this scheme the user interaction remains simple while still a large variety of functions can be modeled. With this approach, the user only has to decide, whether he wants a smooth or a sharp transition at Sa i ,Sb i ,Sc i ,Sd i . The importance of the characteristic of these
When the inner support is zero and one side of the extended support coincides, the component function has a ramp shape. If the inner support is smaller than the extended support (and the transitions at the support region borders are defined to be C 0 ), the component function becomes a trapezoid. When defining the transitions Cx i to be smooth, a variety of Gauss-like functions and non-linear ramp functions can be modeled.
3.3 Defining the weighting functions The weighting functions wi are used to combine the component functions CF i . They have to fulfill certain restrictions:
The sum of all weighting functions has to evaluate to 1 for every data value:
n X i=1
wi (d) = 1
8d 2 [0; 1]
A weighting function wi must not alter the definition of the inner support of component function CF i .
TF (d) = Ai , wi (d) = 1
d 2 [Sb i ; Sc i ]
The continuity of transfer function TF has to be according to the user definition:
TF (Sb i ) : : : C Cbi TF (Sc i?1 ) : : : C Cci?1
1
>< p (d) i wi (d) = > 1 >: 1 ? pi+1(d) 0
0 d < Sc i?1 Sc i?1 d < Sb i Sb i d Sc i Sc i < d Sb i+1 Sb i+1 < d 1
1
