GIMIAS: An Open Source Framework for Efficient Development of ...

GIMIAS: An Open Source Framework for Efficient Development of Research Tools and Clinical Prototypes Ignacio Larrabide1,2, , Pedro Omedas2,1 , Yves Martelli2,1 , Xavier Planes2,1 , Maarten Nieber2,1 , Juan A. Moya2,1, Constantine Butakoff2,1 , Rafael Sebastián2,1 , Oscar Camara2,1, Mathieu De Craene1,2 , Bart H. Bijnens3,2,1 , and Alejandro F. Frangi2,1,3 Center for Computational Imaging and Simulation Technologies in Biomedicine (CISTIB) 1 Networking Biomedical Research Center on Bioengineering, Biomaterials and Nanomedicine (CIBER-BBN), Barcelona, Spain 2 Universitat Pompeu Fabra, Barcelona, Spain 3 Instituci´ o Catalana de Recerca i Estudis Avan¸cats (ICREA), Barcelona, Spain {ignacio.larrabide,pedro.omedas,alejandro.frangi}@upf.edu

Abstract. GIMIAS is a workflow-oriented environment for addressing advanced biomedical image computing and build personalized computational models, which is extensible through the development of applicationspecific plug-ins. In addition, GIMIAS provides an open source framework for efficient development of research and clinical software prototypes integrating contributions from the Virtual Physiological Human community while allowing business-friendly technology transfer and commercial product development. This framework has been fully developed in ANSI-C++ on top of well known open source libraries like VTK, ITK and wxWidgets among others. Based on GIMIAS, in this paper is presented a workflow for medical image analysis and simulation of the heart. Keywords: Biomedical imaging, cardiovascular modeling, Virtual Physiological Human, personalized simulations, open source software.

1

Introduction

The integrative multi-scale and multi-disciplinar community that tries to understand the human patho-physiology is internationally known as “IUPS Physiome Project” [1,2] or, in the European context as “Virtual Physiological Human” (VPH) [3]. Besides the strategic character that it has regarding the priorities of the 7th Framework Programme of the EU, the VPH is of great relevance in the current context given its integrative vision of the physiological and computational knowledge. In this perspective, a key challenge in the development of integrative and predictive models for the human physiology is the creation of

Corresponding author.

N. Ayache, H. Delingette, and M. Sermesant (Eds.): FIMH 2009, LNCS 5528, pp. 417–426, 2009. c Springer-Verlag Berlin Heidelberg 2009

418

I. Larrabide et al.

computational tools that enable the collaborative work between scientists, industry and health care to bridge the gap between scientific discovery and clinical treatment. An essential condition for the success of the VPH is the development of software tools that ensure methodology integration. As examples we can site all multi-scale integration, access and exploitation of large repositories and databases, processing of data with state-of-the-art algorithms, creation of personalized models, data fusion and visualization. In this paper we present GIMIAS (Graphical Interface for Medical Image Analysis and Simulation), a workflow-oriented environment that has been specially tailored for fast development of biomedical imaging and modeling software prototypes. GIMIAS provides an open source framework for the development of research tools and clinical prototypes, allowing the integration of contributions from the Physiome community while allowing business-friendly technology transfer.

2

Related Work

Current tools (see Table 1) have their own strengths and weaknesses, mainly linked to the core skills of the research group behind it. However, there is not currently, a single application that fulfills all the requirements of all possible VPH applications. For this reason, the VPH community is steadily progressing towards establishing data, modeling and communication standards that ensure interoperability between different tools rather than creating a single application that accounts for all possible needs. Furthermore, there is also a need for a Free Open Source Software platform that allows efficient development of prototypes, that requires minimal training time or prior knowledge to get started. What GIMIAS brings to the community is a common framework to help on data manipulation and visualization in which processing algorithms can be easily integrated. In this environment new algorithms and models (developed by the user or provided by an existing library) can be easily integrated in more complex processing workflows. Furthermore, GIMIAS and all the libraries on which it relies are Free Open Source Software. In this way, different teams using the platform, either from academia or industry, can interact and share modules easily. The main goal of this platform is to transfer novel technologies into clinical environment by the fast development of prototypes.

3

Specifications

Taking into account all the requirements for the platform, GIMIAS architecture has been designed to accomplish the following specifications. The following nonfunctional requirements, i.e., criteria usable to judge the system other than its functionalities, should be addressed by GIMIAS: – Modularity: its functionalities should be modular allowing flexibility in the way that they are combined to facilitate exchangeability and reuse.

Graphical Interface for Medical Image Analysis and Simulation

419

Table 1. Summarized list of current software development projects focused on medical image processing and end-user software for model personalization. The brief feature list is based on the description provided by their distributors. The following list contains the corresponding websites for the tools described above where further details can be found: www.brainvisa.info, www.cmiss.org/cmgui, wwwsop.inria.fr/asclepios/software/MedINRIA, www.mevislab.de/, www.openmaf.org/, www.paraview.org, www.slicer.org. Name BrainVISA CMGUI

Distribution license Open source software. CeCILL License. Open source software. Mozilla Public License.

MedINRIA Free for non-commercial use.

MeVisLab

Basic version free for noncommercial use. SDK version requires a commercial license.

OpenMAF

Open source software. BSD-style License.

ParaView


Slicer 3D


Brief list of features - scientific visualization - signal and image processing - scientific visualization - scripting capabilities - model personalization - signal and image processing - scientific visualization - model personalization - signal and image processing - scientific visualization - scripting capabilities - plug-in architecture - image processing - generic application framework - scientific visualization - model personalization - image processing - scientific visualization - scripting capabilities - plug-in architecture - scientific visualization - plug-in architecture - image processing

– Simplicity: to allow technologists and scientists to easily integrate their algorithms in the platform in with small amount of time and effort. – Flexibility: to support the development of research and clinical prototypes with different degrees of complexity. Also, binary (compiled) and/or interpreted (script-based) plug-in development should be possible requiring an even smaller amount of prior knowledge to integrate new features. – Interoperability: to cope with the increasing amount of image processing and modeling tools available and to ensure maximum reuse, it is of crucial importance that this software is capable of inter-operating with others. In order to illustrate how GIMIAS can be used for the implementation of a particular workflow, let us use as an example an application for cardiac modeling. On this regard, the treatment of biomedical data often involves (sometimes large) work-flows that require different data processing tools. This is particularly important in the case of cardiac modeling, where the use of computational tools

420

I. Larrabide et al.

for image and signal processing, surface and volume mesh editing, creation of numerical models and data visualization is required. GIMIAS framework users GIMIAS, as a fast prototyping open source framework, has been developed thinking in two different user profiles: 1. Researchers who want to test methodologies and easily create prototypes for validating the practical feasibility of research concepts. Using the proposed framework, they can create new plug-ins that incorporate their methods and algorithms, allowing potential end-users (e.g., medical doctors, clinical scientists, among others) to assess them. 2. Scientific developers (from academic or industrial environment) that need to create fast and cheap prototypes for research tools and/or clinical products. Moreover, by having a BSD licence, GIMIAS is appropriate for promoting interaction between industrial and academic software development teams. GIMIAS provides a simple Application Programming Interface (API) that permits, with minimal effort, to trial methodologies and algorithms in the form of end-user applications. Also, prototypes developed within/for GIMIAS can be tested by end users in real scenarios and with real data at early development stages, providing very useful feedback as well as reducing the time required to get new concepts from research to the clinical environment.

4

The GIMIAS Architecture

GIMIAS has been designed to be extended via the addition of plug-ins. Plugins are extension units (e.g., dynamic link libraries or shared objects depending on the platform) for an application whose architecture allows functionalities to

Fig. 1. Left: Schematic overview of the plug-in based architecture of GIMIAS. General infrastructure services are used by the different plug-ins according to the specific application. Plug-ins can be changed to create new workflows. Right: Classes to be developed, and their associated complexity and interaction, for the creation of a new plug-in. For the classes Widget and DataView the developer can choose either implementing them or using the standard implementation provided by the framework.


421

be introduced by users or programmers at well-defined places after it has been installed. In GIMIAS, plug-ins are software entities that can be created with minimal experience in programming for extending the application by providing specific features in a modular way. Plug-in based software architecture GIMIAS provides an API for the plug-ins to interact with the application core and with other plug-ins. The classes that can be extended for the creation of a new plug-in are: Widget, SceneView, DataEntityList, FrontEndPlugin and Processor (see Figure 1). Depending on the complexity of the desired plug-in, not all these classes need to be developed. The minimal set of classes needed to create a plug-in with new behavior are WorkingData (containing the data used by the plug-in), FrontEndPlugin (which describes the main functionality of the plug-in) and Processor (executes an algorithm given one or more input data and generates one or more output data). Very briefly, the classes that need to be used and/or extended for the creation of plug-ins are (see Fig. 1): – DataEntity: This class stores and encapsulates a single data object (e.g., medical image, transformation, surface mesh, model simulation result, etc.). – DataHolder: This class allows objects to register as observers of data, for instance, to be notified when the data is changed. – WorkingData: This class holds a reference to the DataHolders associated to the data required and is currently used by the plug-in (selected input, output, etc.). – DataEntityList: This class contains a list populated with the data entities currently available for processing in the application. The input data that will be processed by the plug-in and also their outputs are stored in this list. – FrontEndPlugin: This is the base class for all the plug-ins, every new plugin must extend this class. This base class provides access to the rendering classes, but delegates the creation of the rendering scene to the extending class’ implementation. – Processor: This class executes one algorithm and given on or more input data and generates one or more output data (usually coming from the DataEntityList). A plug-in may contain more than one Processor class. No data is stored inside the Processor. Regarding data processing, each plug-in uses its own (one or more) Processor classes that encapsulate the application logic indicating which algorithm needs to be executed on the data. External algorithms (user defined or furnished by a library) are called by a Processor after the required input data have been specified. After the execution of an algorithm, the Processor notifies (through the DataHolder) the corresponding observers that new data is available. These data can be rendered or processed depending on the application. It is the responsibility of the plug-in to determine how the processing pipeline continues. In this way, the plug-in developer is responsible for defining how the end-user operates over the data and visualizes it. DataEntity encloses and represents 4D

422

I. Larrabide et al.

working data in a single entity that can be modified, transformed and rendered by any plugin. This class allows plugins to share data, encapsulating its format conversion thus facilitating the exchange and flow data. For its simple architecture, the sole specialization/implementation of three classes is enough for creating new plug-ins. In this way, new features can be integrated very fast. The architecture is flexible to support plug-ins with different complexity, going from simple (e.g., data is processed with a specific algorithm and the results are visualized with standard viewers) to complex (e.g., the entire user interface can be refurbished to create a new data interaction and visualization application). In addition to its plug-in API, the framework provides basic services which can be used by the plug-ins: – scientific visualization through standard screen layout (i.e., 3D plus orthogonal slicing), – multimodal (MRI, SPECT, MSCT, US, PET) image processing, – heterogeneous data management (images, polylines, surface geometries, volumetric meshes), – standard I/O formats (others can be added by the users). All these services are accessed trough the FrontEndPlugin class. Further details on other features and plug-ins provided by the framework can be found at www.gimias.org. Regarding interoperability with external tools, the framework allows to easily integrate different format readers and writers. A closer integration (e.g., in runtime) with other tools is possible by developing a specific plug-in for this matter. Third-party libraries GIMIAS has been entirely developed in ANSI-C++ and is distributed under BSD-licence. The reason to make the software open source is to facilitate and foster the cooperation and interaction between different research groups and with industry for technology transfer. This licence model together with its plugin architecture make it ideal for this kind of interaction. A proper selection of third party libraries is crucial for the long term continuity of the software. For this reason only BSD-licensed libraries with consolidated and active user communities where selected. For the graphical user interface, wxWidgets1 was considered. The mesh manipulation is based on The Visualization Toolkit (VTK)2 , an open source, freely available software system for 3D computer graphics, image processing, and visualization with a large and established user community around the world. For user interaction and visualization of data, the Medical Imaging Interaction Toolkit (MITK) [4] is used. For handling data in DICOM standard, DCMTK library is used3 . For image processing operations The Insight Toolkit (ITK)4 was chosen. 1 2 3 4

www.wxwidgets.org www.vtk.org dicom.offis.de/dcmtk www.itk.org


5

423

Example Workflow in GIMIAS

In this section we analyze in detail a case study based on image analysis and modeling of the heart. The objective of this case study is to show the capability of GIMIAS to integrate different algorithms in a single workflow. To this end, patient images in DICOM format are loaded. Next the images are segmented using an anatomical atlas of the heart. The last step involves generating a personalized computational model of the heart containing myocardial fiber orientation, the Purkinje system and different mesh labels (endo- and epicardium) for the subsequent electrophysiological simulation. This is addressed in a single user application that goes from the image to the personalized simulation model. DICOM reading plug-in The objective of the DICOM plug-in is to let the user browse medical images stored in DICOM format, allowing to load 2D, 3D and 4D data (Fig. 2). Once the data is loaded, it is organized using a tree viewer to facilitate visualization of patient, study, series and time point data. Using this plug-in, the user can inspect different studies and navigate through the slices and corresponding time points. The DICOM plug-in creates a volumetric or 3D+t image from the selected DICOM data including the slices and time points selected by the user. After that, the image can be easily rendered or located in the DataEntityList where it becomes visible to the rest of the application for further processing. Cardiac image analysis plug-in In order to obtain a personalized geometry of the heart from a medical image, image segmentation based on an anatomical heart atlas is used [5]. This atlas is first initialized using anatomical landmarks such as the left ventricle apex, the aorta and mitral valve ring (Fig. 2). Once the heart atlas has been positioned within the image, the plug-in allows the user to adjust the shape of the atlas using the image as reference and thus, semi-automatically segment the heart. Using this feature, the user can deform the heart model (by simply dragging the surface interactively by using the mouse) and adjust the shape to the image. Automatic segmentation using active shape and image appearance models is currently under development and will be integrated in the prototype in the future. In the case of 4D images, it is possible to segment the whole sequence of images starting from any given time frame. The plug-in allows the user to obtain a number of quantitative measurements of the heart including the ejection fraction, wall motion, wall thickness, wall thickening, among others, during the manual segmentation process to give immediate feedback to the user. After the segmentation, the result can be analyzed using Bull’s eye plots (AHA 17-segment model) for the different variables such as wall thickening, radial wall motion, among others. As will be shown below, the resulting geometrical models are used for the generation of personalized simulations of the cardiac electrophysiology.

424

I. Larrabide et al.

DICOM plugin

Cardiac segmentation and quantification plugin

Cardiac simulation plugin

Fig. 2. Cardiac modeling workflow as implemented in GIMIAS. From top to bottom are presented the DICOM, the Cardiac segmentation and quantification, and the Cardiac simulation plug-ins. After desired images are selected from the time sequence, an atlas is used to segment the images and adjust the segmentation result where required. Then, from the resulting heart geometry, a model for simulation of the heart electrophysiology is created. Finally, the propagation front in a selected region of the cardiac muscle is presented.


425

Cardiac modeling plug-in Before being able to use the anatomical model to perform electrophysiological or electromechanical simulation it is necessary to generate a proper FEM model. It is necessary to mesh the surface obtained from the segmentation step, and produce a volumetric fine mesh which will be made up of tetrahedral elements. Volumetric mesh quality is ensured by controlling both the maximum distance between neighboring nodes and the radius-to-edge ratio to assure regularly-shaped elements. In keeping with recommendations from literature high-resolution meshes with inter-node distance ≤ 500 µm to avoid physiologically inaccurate numerical effects such as wavefront warping and other artifacts. Excitation and recovery sequences are influenced by the direction of myocardial fibers, which determine the longitudinal and transversal orientation of the anisotropic conductivity of intra- and extra-cellular media. As current imaging techniques are unable to non invasively extract myocardial fiber orientation from the beating heart, mathematical models are used to replicate the rotation of fibers across the ventricular wall. In GIMIAS, the fiber orientation is calculated for every element of the mesh, using a mathematical formulation based on the work of Streeter [6]. In this way, the familiar helical structure of fibers was reproduced, with orientations rotating from 50◦ at the endocardium to −60◦ at the epicardium. The Purkinje System (PS) is also an important part of the system that helps the fast and synchronous electrical propagation of the impulse coming from the sinus node. Since the reconstruction of the PS from specific patients is not feasible, a fiber network has been defined according to the descriptions by Durrer. The PS is represented as a 1D cable model that is placed upon the endocardial surface, containing the bundle branches and 100 Purkinje cables that expanded from the bundle terminals towards the ventricular apex continuing onto the left and right free walls. The procedure to include the PS consists in a manual delineation of the fibers in the heart atlas, and subsequently a propagation of the network to each mesh. The propagation is done during the segmentation of the ventricles, where the atlas is deformed to match the specific geometry of the patient heart, fitting at the same time the PS embedded in the atlas.

6

Conclusions

GIMIAS is an environment specially tailored for the creation of work-flow applications in the field of biomedical image computing and personalized simulation. The architecture of the underlying framework is based on plug-ins through which its core functionalities can be extended. Other services such as PACS connection, parallel computation through grid middleware, and scripting are in the development roadmap of the framework. In this work we presented a cardiac image analysis and simulation application developed within this environment. In the presented workflow, GIMIAS provides the capability of segmenting images of the heart and allows improving the outcome manually when the segmentation is not perfect.

426

I. Larrabide et al.

GIMIAS provides an open source framework for efficient development of research and clinical software prototypes within the Physiome community while allowing business-friendly technology transfer and commercial product development.

Acknowledgements A complete list of contributors to the analysis, design and implementation of this platform can be found at www.gimias.org. This research has been partly funded by CIBER-BBN (VPHTk project), the industrial and Technological Development Center (CDTI) under the CENIT programme (CDTEAM project) and the European Community’s Seventh Framework Programme (FP7/2007-2013) under the grants agreement n.224495 (euHeart Project) and n.223920 (VPH NoE). Dr. O. Camara and R. Sebastian acknowledge grant support from the Spanish Ministry of Research and Innovation, under a Ramon y Cajal and Juan de la Cierva Research Fellowship, respectively.

References 1. Hunter, P., Robbins, P., Noble, D.: The IUPS Human Physiome Project. Eur. J. Physiol. 445(1), 1–9 (2002) 2. Hunter, P., Borg, T.: Integration from Proteins to Organs: The Physiome Project. Nature 4(3), 237–243 (2003) 3. Fenner, J., Brook, B., Clapworthy, G., Coveney, P., Feipel, V., Gregersen, H., Hose, D., Kohl, P., Lawford, P., McCormack, K., Pinney, D., Thomas, S., Jan, S.V.S., Waters, S., Viceconti, M.: The EuroPhysiome, STEP and a roadmap for the virtual physiological human 366, 2979–2999 (2008) 4. Wolf, I., Vetter, M., Wegner, I., B¨ ottger, T., Nolden, M., Sch¨ obinger, M., Hastenteufel, M., Meinzer, H.P.: The Medical Imaging Tookit 9, 594–604 (2005) 5. Ordas, S., Oubel, E., Leta, R., Carrera, F., Frangi, A.: A statistical shape model of the heart and its application to model-based segmentation. In: Proc. SPIE Medical Imaging, San Diego, CA, USA, vol. 6511 (2007) 6. Streeter, D.: Handbook of Physiology, Section 2: The Cardiovascular System. In: Gross morphology and fiber geometry of the heart. The Heart, ch. 4, vol. I, pp. 61–112. American Physiological Society (1979)