The International Journal of Virtual Reality, 2009, 8(1):37-41
37
A Framework for GPU-accelerated Virtual Cardiac Intervention Rongdong Yu1, Patricia Chiang2, Wenyu Chen2, Jianmin Zheng3, Yiyu Cai2, Xiuzi Ye1,4, Sanyuan Zhang4, Yin Zhang4 and Koon-Hou Mak5 1
College of Life Science, Zhejiang University, China School of Mechanical & Aerospace Engineering, Nanyang Technological University, Singapore 3 School of Computer Engineering, Nanyang Technological University, Singapore 4 College of Computer Science, Zhejiang University, China 5 Mak Heart Clinic, Singapore1 2
Abstract—Cardiovascular diseases is a major cause of deaths in many developed countries. To treat cardiovascular diseases, minimally invasive cardiac intervention is widely used as an effective yet complicated solution. Training of cardiac intervention thus plays a crucial role. This paper addresses the use of Virtual Reality (VR) technology for the simulated training of cardiac intervention. In particular, the paper describes the architecture of a simulation system for virtual cardiac intervention. The framework proposed consists of hardware component, software component and methodology component. As an enabling technology, Graphics Processing Unit (GPU) is applied in this work to enhance cardiac modeling, cardiac visualization, and cardiac interaction with the VR simulation environment. Index Terms—VR Environment, GPU, Architecture, Simulation, Cardiac Intervention.
I.
INTRODUCTION
Heart disease is one of the leading causes of death worldwide [Ch'en et al, 1998] and the prevalence rises with age. Cardiac diseases like ischemic heart disease and myocardial infarction are the main causes of hospitalization among the diseases of the circulatory system. Ischemia or myocardial infarction can eventually lead to heart failure in which the heart cannot pump sufficient amount of blood into the peripheral circulation. Another key factor is the failed or diseased heart does not have the ability to repair itself. Positron Emission Tomography (PET) and Single Photon Emission Computed Tomography (SPECT) using various radionuclide isotopes have been used to assess the viability of myocardium after myocardial infarction or patients with severe coronary artery disease. Current therapeutic approaches focus on the regeneration of myocardium using cellular and gene therapies. However, the procedures of these novel cardiac intervention techniques are complex and require intensive training. Therefore, hand-eye coordination [Cai et al, 2006] is critical in this form of cardiac intervention. Previously, we have developed an interactive device for the treatment of coronary artery lesions with balloon catheters and stents. This paper focuses on the use of Virtual Reality (VR) technology for the simulation of the novel technique of cardiac Manuscript received on 20 February 2008. Rongdong Yu and Patricia Chiang are joint first authors. E-Maill:
[email protected]
intervention. In particular, the paper describes the framework of a VR simulation system for cardiac intervention focusing on VR hardware, software and methodology. With Graphics Processing Unit (GPU) as an enabling technology [Jens et al, 2005], this paper discusses GPU-enhanced cardiac modeling, cardiac visualization, and cardiac interaction with the VR simulation system. II. A FRAMEWORK OF VIRTUAL CARDIAC INTERVENTION Much efforts have been made to simulate the interventional surgery [Cai et al, 2003] for clinical and research purposes. However, simulation of cardiac intervention is more challenging due to the complex and dynamic natures of the heart, especially given limited computational power of Central Processing Unit (CPU). In this paper, we propose to apply the GPU technology for real-time and realistic simulation. Fig. 1 illustrates the architecture of the VR simulation system for cardiac intervention.
Fig. 1. System Architecture
38
The International Journal of Virtual Reality, 2009, 8(1):37-41
2.1 Software Component The simulation system is based on a Windows XP SP3 Operating System. Microsoft Visual Studio 2005 is used as the main programming platform. All software development is implemented in C++. Both DirectX and OpenGL are used as Graphic Libraries for graphic programming. Microsoft Foundation Class and Windows API are used for application development for VR cardiac interventional simulation. The software design of the VR cardiac simulation is designed on top of the graphics libraries and APIs. The central parts of the software component are the 3D modeling, graphics visualization and interaction. GPU is applied here to accelerate the process of modeling, visualization and interaction for real-time and realistic performance. 2.2 Hardware Component Cardiac simulation is often a data crunching task. Instead of transferring the data (e.g., coordinates) into system memory for CPU to handle, GPU can significantly improve the performance of cardiac simulation by direct mapping through the CUDA driver and later drawing using the graphic pipeline. The CUDA Toolkit 2.0 [nVIDIA, 2007; John et al, 2008] is used in this work with a workstation having 1.866GHz Intel E6300 processor, 2GB system memory and a nVIDIA GeForce GTX 280 graphic card with 1GB graphics memory. The graphic card driver nVIDIA ForceWare 180.24 edition is used to support the CUDA 2.0 architecture. Taking the advantage of GPU’s parallel processing, the simulation performance is dependent to the number of stream processors used. In CUDA hardware architecture, the minimal operating unit which threads run on in the GPU pipelines is stream processor. Every 8 stream processors are grouped into 1 multiprocessor. In our case, the graphic card has 30 multiprocessors and thus 8 stream processors to enable up to 8 threads per multiprocessor, giving a total of 240 threads running simultaneously. The hardware component is heavily relying on the GPU in this work. But CPU is still playing an active role in relation to the interactive devices. In particular, VR devices are integrated into the cardiac simulation system via different channels. These devices can be tactile or even haptic based.
simulated [Nash, 1998]. The ventricular mechanics model divides the entire cardiac cycle into four phases: the diastolic filling, isovolumic contraction, ejection and isovolumic relaxation. The system architecture with GPU implementation has the advantage to design simulation procedures, algorithms, and optimization. For example, the dynamic simulation of the heart model is very computational expensive. To simplify, a digitized heart model is constructed for the dynamic simulation. To process thousands or millions of data points with this discrete model, GPU offers parallel computing capability to speed up the process [John et al, 2008]. With the CUDA hardware, we can use the location computing operation for one data point with one fixed thread. Thus the total number of threads required is determined by the number of data point ( N ). These threads are distributed to the multiprocessors, and each stream processor is assigned a subset of N /P M data points ( P is the number of processors per multiprocessor, M is the number of multiprocessors on a graphic card and P M is the total number of stream processors on the graphic card). We can group every 8 threads in a set which is the number of stream processors on the graphic card. VR Environment
Physical Users
Virtual 3D Heart
Modeling
Interaction
Visualization
Interaction
Fig. 2. VR Environment
2.3 Methodology Component This component is composed of mainly the simulated interventional procedures, the computational algorithms, and optimization techniques. Several procedures including catheterization and electrophysiological mapping are involved in cardiac intervention. The simulated procedures are built with an emphasis on the hand-eye coordination for diagnosis and treatment purposes. Behind the simulated procedures are a number of algorithms each designed to address specific issues including electrical and mechanical modeling, rendering and collision detection. These algorithms must be optimized to enhance accuracy in simulated training and to fit real-time dynamic animation and interactive cardiac intervention. The methodology component is also GPU enabled. Following the myocardium constitutive laws, an active element for contraction controlled by trans-membrane potential, and a passive element representing the mechanical elasticity can be
III. GPU-ENHANCED CARDIAC VR ENVIRONMENT For cardiac simulation, the proposed VR environment (Fig. 2) consists of physical and virtual sides. VR interaction comes in as an interface between the two sides. In this section, we discuss the cardiac modeling, visualization and interaction in the cardiac VR environment with the aid of GPU technology. 3.1 Cardiac Modeling The heart consists of four chambers, two atria (left and right) and two ventricles (left and right). The atria are located in the upper part where ventricles are located in lower part. There is an opening called mitral orifice with mitral valve situated between the left atrium and left ventricle and tricuspid valve between the right atrium and right ventricle. Aortic orifice is an opening with aortic valve which communicates between the left ventricle and aorta. There is another opening called pulmonary orifice
The International Journal of Virtual Reality, 2009, 8(1):37-41 between the right ventricle and pulmonary artery. The heart pumps blood through the blood vessels by repeated, rhythmic contractions to the rest of the body. The deoxygenated blood from the body collects in the right atrium then goes into the right ventricle and is pumped into the lungs via the right ventricle. The left atrium collects oxygenated blood from the lungs and moves the blood to left ventricle and from which pumps out to the body. In the GPU-enabled VR environment, the heart is digitally modeled. Depending on applications, the heart can be discretized in different level of details. Fig. 3 shows the meshed heart model consisting of 148,516 points. Different parts of the heart are marked by different colors. With the digitized heart, digital geometry processing techniques can be developed for cardiac modeling supported by the powerful GPU technology. In building a realistic virtual 3D heart, the dynamic beating of the heart should be incorporated where GPU acceleration can be best exploited. Another aspect of cardiac modeling is the electrical excitation propagation in the heart via specialized conduction pathways and myocardial conduction. Starting from the sino-atrial (SA) node, action potentials are propagated to atrioventricular (AV) node, giving rise to atrial contractions, then crossing atria to ventricles via the Bundle of His and descending down left and right bundles in interventricular septum towards the apex, finally conducting upwards via Pukinje fibres, giving rise to ventricular contractions. GPU can help to simulate various propagation patterns with different input parameters.
39
From the 3D cardiac surface models, we have worked separately on a multi-sectional visualization scheme to display the cross-sectional relationships of the heart structure. Fig. 4(a) shows three sectional views of the cardiac model; Fig. 4(b) shows the combinatory relations of the cross-sections of the cardiac model; and Fig. 4(c) shows the overall cross-sectional relations of the cardiac model.
(a)
XY
XY
YZ
ZX
ZX
ZX
(b)
3.2 Cardiac Visualization Cardiac visualization plays an important role in cardiac intervention. In order to achieve high quality dynamic and real-time simulation, high performance graphics processing is of great importance. In our system, we can visualize isolated components of the heart in point, mesh or surface configurations in tandem with interventional device and we have navigational tools that allow user to manipulate views and transverse through the heart. Our goal is to enable immersive visualization for the VR environment.
Fig. 3. Cardiac modeling with mesh digitalization. (See Color Plate 7)
(c) Fig. 4. Cardiac cross-sectional visualization
3.3 Cardiac Interaction Surgeons or interventionists use interventional devices such as catheters navigated through the blood vessels to reach the diseased lesion. During the catheterization procedure, the catheter device needs to negotiate frequently with the heart wall to get equilibrium positions inside the lumen of the 3D heart. The operators maneuver the catheter via forwarding, backwarding, twisting and bending actions. The interaction between the catheter and heart wall needs to be computed in real-time. Fig. 5 illustrates the bending of the catheter when interacting with the heart wall. The interventional device, in this case, a catheter is modeled as one dimensional chain having 6 degrees of freedom that supports a range of main axis directions from orifice into chamber (for example from aortic valve into left ventricle) with translation along axis for forwarding/backwarding and rotation about axis for twisting and additional dimension for bi-directional bending along a plane incorporating the axis. The catheter can be controlled with VR device while forces governing deformation upon interaction can be fed back through haptic device.
40
The International Journal of Virtual Reality, 2009, 8(1):37-41 [6] John D. Owens, Mike Houston, David Luebke, Simon Green, John E. Stone, and James C. Phillips.GPU computing, Proceedings of the IEEE, no.96, pp. 879-899 ,2008. [7] M.Nash. Mechanics and Material Properties of the Heart using an Anatomically Accurate Mathematical Model. PhD thesis, University of Auckland, 1998.
Rongdong Yu is a Ph D student with Zhejiang University, China. Sponsored by the China Scholar Council, he is doing a collaborative research with Nanyang Technological University under an attachment project. His research interests include Bio-medical Science and Computer Sciences.
Fig. 5. Catheter and heart wall interaction. The catheter is represented in white and blue nodes and the part outside the heart are bent back inside the heart (red nodes). (See Color Plate 8)
Patricia Chiang receives her BEng degree from National University of Singapore and her MSc degree from Nanyang Technological University. Her research interests are in Signal Processing for Biomedical Engineering applications.
IV. CONCLUSIONS This paper describes the architecture of a VR simulation system for a novel cardiac intervention technique. Software component, hardware component and methodology component are discussed The proposed solution uses GPU technology for high performance computing. The cardiac modeling, cardiac visualization, and cardiac interaction are also briefly discussed for the purpose to develop a VR environment for simulation of cardiac intervention. The presented work takes the advantage of GPU technology for complicated cardiac simulation. It allows parallel processing of the cardiac model in order to achieve real-time and realistic simulation. In the current study with this ongoing project, blood flow is not included. ACKNOWLEDGEMENT Yu Rong Dong is sponsored by the China Scholar Council for his research attachment with the Nanyang Technological University, Singapore. This work is also partly supported by the National 973 Program(No.2009CB320804) and the National 863 Program of China (2007AA01Z311, 2007AA0 4Z1A5). REFERENCES [1] Ch’en, F. F-T., Vaughan-Jones, R. D.K., Clarke and D. Noble. Modelling myocardial ischaemia and reperfusion, Progress in Biophysics and Molecular Biology, vol.69, no.2-3, pp. 515-538, 1998. [2] YY Cai et al. Tactile VR for Hand-eye Coordination in Simulated PTCA, Computers in Biology and Medicine, Pergamon, vol.36, no.2, pp 167-180, 2006. [3] Jens Krüger and Rüdiger Westermann .GPU Simulation and Rendering of Volumetric Effects for Computer Games and Virtual Environments, Computer Graphics Forum, vol.24, no.3, pp. 685-693,2005. [4] YY .Cai et al. VR simulated training for less invasive vascular intervention, Computers & Graphics, vol.27, no.2, pp. 215-221, 2003. [5] nVIDIA. nVIDIA CUDA Compute Unified Device Architecture- Programming Guide, 2007. http://developer.download.nvidia.com/compute/cuda
Wenyu Chen is a Ph D student with the School of Mechanical & Aerospace Engineering, Nanyang Technological University, Singapore. His research interests include Geometric Modeling, Surface Modeling.
Dr Jianmin Zheng is with the School of Computer Engineering, Nanyang Technological University. His research interest includes computer aided geometric design, CAD/CAM, computer graphics, animation, digital imaging and visualization.
Dr Yiyu Cai is associate professor with the School of Mechanical & Aerospace Engineering, Nanyang Technological University, Singapore. His research interests include VR, Computational Biomedical Sciences, and Interactive & Digital Media.
Xiuzi Ye, Professor, he is with the College of Computer Science, Zhejiang University. His research interest includes Bio-informatics, Medical Visualization and CAD/CAM
The International Journal of Virtual Reality, 2009, 8(1):37-41 Dr Sanyuan Zhang is Professor with the College of Computer Science, Zhejiang University. His research interests are mainly in computer aided geometric design, and image processing.
Yin Zhang, Associate Professor, she is with the College of Computer Science, Zhejiang University. Her research interests are mainly in artificial intelligence, geometric modeling and image processing.
Dr Koon-Hou MAK is an interventional cardiologist and a visiting associate professor with the School of Mechanial and Aerospace Engineering, Nanyang Technological University, Singapore. His research interest is the development of medical devices, interventional cardiology and acute coronary syndromes.
41
