Integrating the Insight Toolkit itk into a Medical Software ... - CiteSeerX

CARS 2002 – H.U. Lemke, M.W. Vannier; K. Inamura, A.G. Farman, K. Doi & J.H.C. Reiber (Editors)  CARS/Springer. All rights reserved.

1

Integrating the Insight Toolkit itk into a Medical Software Framework Nils Hanssen, Bartosz von Rymon-Lipinski, Thomas Jansen, Marc Liévin, Erwin Keeve Surgical Systems Laboratory research center c a e s a r Friedensplatz 16, 53111 Bonn, Germany

Abstract In this paper, we present the integration of an image processing toolkit into our platformindependent medical software framework Julius. The Insight Toolkit (itk) consists of state-of-the-art segmentation and registration1 methods, focused on medical applications. Each processing filter of the segmentation toolkit is represented as an individual element in the user interface. The central part of every segmentation routine is the anatomical list, which is fully customizable. By clicking on an anatomical entity in this list, the user is guided during the segmentation of the corresponding structure. This guidance is directly dependent on the underlying modality as well as the selected anatomical structure itself. Keywords: software framework, segmentation, insight toolkit

1. Introduction Segmentation of anatomical structures from medical images of different modalities is a central part of the processing pipeline in image-guided surgery. By identifying structures and defining boundaries, parts of the images or volumes can be assigned to anatomical structures, which is important for successional processing or simulation steps. Nonautomatic segmentation methods have to be parameterized with a-priori knowledge by the user. This knowledge includes typical voxel values of tissue types in the considered modalities (thresholds) as well as spatial and geometrical properties (seed-points or initial contours). In this paper, we present the basic concept of anatomy-based segmentation. A customizable knowledge base contains a-priori information about anatomical structures and tissue types for different modalities. By means of this knowledge base, sequences of processing operations from the Insight Toolkit (itk) [1] and their parameters are automatically driven by the medical software framework [2]. The sequences and derived parameter values are displayed in the user-interface. Missing parameters that are mandatory for the operation and cannot be automatically determined have to be supplied by the user.

1

In this paper, the focus lies on the integration of the segmentation methods.


2

2. Methods Our medical software framework serves as a scaffold for any medical image processing application. It is prepared for various use cases that include data-I/O, segmentation, registration, navigation and visualization procedures. All elements of the user-interface are based on the Qt library [3], which is platform independent. The internal representation as well as the visualization of volumetric and polygonal data is based on the Visual Toolkit (vtk) [4]. The open-source toolkit itk provides a collection of C++ classes for medical data representation and processing. It follows a data-flow approach that is based on data objects (representation) that are manipulated by filter objects (processing). Supported generic data structures are n-dimensional images as well as unstructured meshes that are templated over the voxel or vertex type and the dimension. The images are used for holding the acquired medical image data, whereas meshes are mainly utilized as base or helper classes for active contour models or surface-based segmentation algorithms. The toolkit makes intense use of C++ templates, also referred to as generic programming [5]. In the following sections, we describe the concept of anatomy-based segmentation as well as the integration of itk in the framework. The presented concepts can be directly transferred to the integration of other third-party C++ processing libraries.

2.1 Anatomy-based Segmentation A central concept within Julius is the anatomical list, which is the starting point for each segmentation procedure (see figure 1, left). It incorporates a comprehensive list of anatomical structures indexed by their Latin and English term. Individual anatomical entities can be grouped by various predefined categories including topographic or functional relationship as well as by tissue type like bone or fat. Custom groups suitable for the current case can be defined. For example, all significant structures for the planning of a knee-intervention can be assembled and handled in a convenient way. One significant advantage of the anatomical list is, that a-priori information can be automatically included in the segmentation, since typical properties of individual structures or tissue types can be stored in an internal knowledge base. This knowledge base, which can be customized via a XML [6] file, contains for example information about typical value ranges of tissue types in the respective modality. Arbitrary custom properties of anatomical entities can be added to the XML file and can be evaluated by the processing filters in the system. Based on this a-priori information in the knowledge base, the framework can recommend a suitable sequence of segmentation methods and parameters. The typical workflow for a segmentation procedure is as follows: 1) An anatomical entity is selected from the list. 2) The framework suggests a suitable sequence of processing methods and parameters. 3) The recommended sequence of processing methods and derived parameters is shown in the user-interface. Next, the actual execution of every single segmentation step can be customized and supervised by the user. Missing parameters that cannot be derived from the knowledge-


3 base have to be specified by hand. Intermediate processing results can be inspected in the slice- and volume-views that are part of the framework (see figure 2, right).

2.2 Stepcards for Processing Sequences The execution of the recommended processing sequences is controlled by user interface elements called stepcards. A stepcard represents one single step of execution that is performed on the data and can be dynamically inserted during program execution. Automatically derived parameters for the processing step are displayed in the stepcard. It is displayed as a Qt-Widget, where the user is able to edit the parameters for the processing step. If interaction with the data, like setting of seed-points or drawing of initial contours is required, the user can click and draw on three orthogonal slice-views. By providing a sequence of stepcards (see figure 2, center), the recommended processing from the pipeline can be monitored. When executing, each stepcard passes its parameters to the corresponding processing filter. The execution of each stepcard can be repeated several times with different parameters until the desired result is achieved. After that, the result is frozen and the next stepcard is displayed.

Figure 1. Julius GUI-elements. Left: Anatomical list organized by tissue types. Center: Stepcard sequence for watershed segmentation. Each step can be applied multiple times until the desired result is achieved. Right: Volume view of mandible extracted from CT images with a connectivity filter visualized via hardware-accelerated 3D texture mapping. The seed-points have been specified by the user.


4 2.3 Integration of ITK in the Data Pool The data pool is a collection of data modules, which implement the same consistent interface and encapsulate different data structures with different internal formats. In contrast to a linear processing pipeline, this pool has no prescribed inherent data-flow structure but is first of all a place where data can be stored for processing. The actual dataflow structure from one data module to the other is defined by the methods that operate on the data. The two generic types of data that can reside in the data pool and can be directly visualized are volumetric images and polygonal meshes in 3D. By means of the registry, a globally unique object, the pool can be accessed and data modules can be distinguished by various properties like the type of data, a description string or a modification timestamp. All actions on the pool that result in manipulation of data are encapsulated by controller modules. The input and output types for a controller are specified in the implementation of the module. The controller can automatically take the latest modified data module with the adequate type as input. Depending on the implementation of the controller, the input data is either kept in the pool or is replaced by the output module. While the in-place mode saves memory since it does not keep provisional results, undo-mechanisms are of course only possible, if the data is duplicated. All modules in the framework can reside in an own runtime-library. Thereby, no recompilation of the framework is necessary if new modules are added. In the current version of Julius, 3D volume objects of itk can be accessed via a data module. With the integrated DICOM-interface itk volume data is added to the data pool and can be displayed with the supported 3D visualization techniques.

2.4 Data Exchange with ITK At any time, data modules, which encapsulate different internal representations of data, can reside in the pool. In order to perform filter operations from all libraries that are encapsulated in controller modules, the data has to be converted to the appropriate internal format on demand. If the internal representations (in terms of memory usage) of the source and destination format matches, an adaptor for format conversion can be used. This is often true for volume datasets. Thereby, data in the pool is not replicated, but merely gets a new interface that is consistent with the corresponding filter of the library. If the memory format of the source and destination formats do not match, a format conversion filter has to be executed. Thereby, the data has to be replicated and reorganized in memory in order to get the appropriate data type (see figure 2). By means of a data format adaptor, itk volume datasets are connected to the internal volume format of Julius. First image processing and segmentation filters like anisotropic diffusion, image gradient and watershed segmentation of itk have been integrated. The actual execution is performed by the itk filter object that is encapsulated in the controller module. After the execution, the result of the filter operation is itself encapsulated in a data module and placed in the data pool.


5

memory 1100101 Reference

Reference

Data module

Data module

Data format A

Data format B

Interface A

Adaptor

memory 1100101

Interface B

memory 1011000 conversion

Reference

Reference

Data module

Data module

Data format A

Data format B Filter

Interface A

Interface B

Figure 2. Data format adaptor (top) and data format filter (bottom). Depending on the memory structure, the data from different internal formats can be converted by an adaptor without duplication of memory.

3. Discussion We presented an anatomy-based segmentation concept in our software-framework Julius, which uses basic a-priori knowledge of structures and tissue types to guide the segmentation. For a selected anatomical structure, image processing and segmentation steps from the Insight Toolkit are recommended and initial parameterizations are suggested. By means of stepcards, the user is able to tune the recommended values and can specify additional parameters that cannot be automatically derived by the system. Data adaptors for volumes have been implemented, which handle the data-exchange between itk volumes and the internal volumetric data format of the framework. Due to the data encapsulation paradigm, further image processing packages can be integrated with minimum effort. By using the standardized XML syntax, custom a-priori information can be added to the knowledge base and can be evaluated by the processing algorithms. We see this association of anatomical structures and segmentation methods with respect to the underlying modality as a first step towards an expert system for segmentation. For the future, we plan to include region of interests as well as seed-points in the knowledge base.


6

Acknowledgements Thanks to Lutz Ritter for providing the anatomical list and its concepts. This work is funded by the German Federal Ministry of Education and Research (BMBF) research grant 01IRA08E – ARSyS-Tricorder. For further information on this research project please refer to http://www.arsys-tricorder.de. The content of this manuscript does not necessarily reflect the position of the BMBF.

References 1. NLM Insight Segmentation and Registration Toolkit, http://www.itk.org, 2001. 2. von Rymon-Lipinski B., Jansen T., “An Extendable Application Framework for Medical Visualization and Surgical Planning,” Proceedings Computer Assisted Radiology and Surgery CARS’01, pp. 181-186, Berlin, Germany. 3. Qt library, Trolltech; http://www.trolltech.com/products/index.html. 4. Schröder W., Martin K., Lorensen B., “The Visualization Toolkit – 2nd Edition”, Prentice Hall PTR, New Jersey, 1998. 5. Austern M.H., “Generic Programming and the STL: Using and Extending the C++ Standard Template Library”, Addison Wesley Professional Computing Series, 1999. 6. Extensible Markup Language (XML), http://www.xml.org.