gpu accelerated alignment of 3-d cta with 2-d x-ray data ... - CiteSeerX

GPU ACCELERATED ALIGNMENT OF 3-D CTA WITH 2-D X-RAY DATA FOR IMPROVED GUIDANCE IN CORONARY INTERVENTIONS C.T. Metz1 , M. Schaap1, S. Klein1 , A.C. Weustink2, N.R. Mollet2 , C. Schultz3, R.J. van Geuns3 , P.W. Serruys3 , T. van Walsum1, W.J. Niessen1,4 1 Departments of Medical Informatics and Radiology, Erasmus MC, Rotterdam, The Netherlands 2 Departments of Radiology and Cardiology, Erasmus MC, Rotterdam, The Netherlands 3 Department of Cardiology, Erasmus MC, Rotterdam, The Netherlands 4 Faculty of Applied Sciences, Delft University of Technology, The Netherlands. ABSTRACT This paper presents a 2-D/3-D registration method for the alignment of cardiac X-ray images to ECG gated CTA data of the coronary arteries. The purpose of our work is to provide visualization of instruments in relation to pre-operative CTA data during interventional cardiology for improved image guidance, especially in complex procedures. The method utilizes the graphical processing unit (GPU) for rendering of digitally reconstructed radiographs (DRRs) from a 3-D CTA-derived coronary model. Using a multi-scale gradient ascent framework, the normalized cross correlation between the simulated and real X-ray image is optimized. Quantitative evaluation is performed by computing the projection error of the vessel centerlines, which was on average 1.34 mm. Index Terms— CTA, X-ray, coronary arteries, 2-D/3-D registration, image guided interventions 1. INTRODUCTION According to the World Health Organization, coronary artery disease (CAD) is currently one of the main causes of death in the world [1]. Atherosclerosis causes plaque build-up in the vessel wall, leading to stenoses and occlusions, and eventually heart failure. Stenotic vessels can be reopened by inflation of a balloon positioned at the location of the pathological region, often followed by stent placement to prevent re-occlusion of the treated vessel segment [2]. During these procedures X-ray imaging is used to visualize the arteries and guide wires. This modality unfortunately lacks depth information due to its projective nature, and only visualizes the vessel lumen, which makes the visualization of total occlusions impossible. Coronary CTA is a modality that allows visualization of coronary pathology and total occlusions. Integration of this information into the interventional scene may thus improve the treatment of these disorders. Especially the treatment of chronic total occlusions, where occlusions need to be opened by e.g. an ablating catheter, is in need of better visualization of the pathological region, so as to prevent complications such as perforation of the vessel wall. The purpose of our work is to integrate pre-operative CTA data into the interventional scene. In this paper, we present first results on the 2-D/3-D registration of ECG gated CTA data of the coronary arteries to cardiac X-ray images. This work was supported by the Dutch Foundation for Scientific Research, NWO, STW.

978-1-4244-3932-4/09/$25.00 ©2009 IEEE

959

Most existing work on 2-D/3-D registration has focused on the registration of static structures, such as the spine and neural vasculature. A commonly followed approach to 2-D/3-D registration of CT(A) images to X-ray images is to generate digitally reconstructed radiographs (DRRs) [3, 4], and subsequently optimize a similarity measure between the DRR and the X-ray image. Others have applied similar approaches to MR(A) registrations, e.g. for cerebral vasculature [5] and the spine [6]. DRR generation is computationally expensive and various methods have been reported to improve the performance of these approaches, such as pre-computing projections [7, 8], applying fast rendering techniques [9] and utilization of graphical processing units (GPUs) [10]. In other methods, similarity between image gradients in the 2-D and 3-D images is optimized [11–13]. In [14], a 3-D registration is performed using a 3-D reconstruction from multiple 2-D projection images. Application of 2-D/3-D registration to vasculature has mainly been done for the cerebral anatomy [5, 15–18], which has a static nature. For the coronary arteries, to the best of our knowledge only one paper was published. Turgeon et al. [19] presented results on both monoand biplane registration of coronary angiograms, and evaluated their method using simulated X-ray images. The focus of this paper is on the registration of CTA and monoplane X-ray data of the same patient at end diastole of the heartcycle. There are two main contributions: first, the method is based on the extraction of a coronary model from the CTA data, which makes fast rendering of the DRRs on the GPU possible, and second, the method is quantitatively evaluated in 2-D using real clinical data, contrary to an evaluation on simulated data as was previously presented [19]. 2. METHOD The proposed intensity-based rigid registration framework is outlined in Fig. 1. Optimization is performed for six degrees of freedom (three translations and three rotations), corresponding to a rigid movement of the vasculature. Mono-plane X-ray imaging is used and registration is performed at end-diastole. The framework consists of several steps which are described in more detail in the following paragraphs. 2.1. Preprocessing of imaging data As we are using an intensity based approach for the alignment of CTA and X-ray data, DRRs need to be computed from the CTA data, which are subsequently compared to the X-ray images. In the CTA

ISBI 2009

Pre-processing

2-D/3-D registration

Transform: Translation (t) Rotation (θ)

CTA

Extract coronaries

za 3-D model

zc

zd

p

DRR image

Subtract background

Metric X-ray w. bg

zb

Fig. 2. An example of the DRR rendering approach. The DRR value for pixel p is determined by computing p = −za + zb − zc + zd .

Optimizer

X-ray w/o. bg

Fig. 1. The registration framework. X-ray w. bg stands for the original X-ray image in which background structures are still present and X-ray w/o. bg stands for the resulting X-ray image after background subtraction. acquisition process, contrast material is applied to make the coronary artery lumen visible in CT, causing the heart chambers to be filled with contrast too. In the DRR rendering process, these high intensity structures over-project on the coronary arteries, causing the coronary arteries to be hardly visible in the resulting DRRs. This makes the DRR images less suitable to be used in the intensity based registration procedure and we therefore render DRRs from a 3-D CTA-derived coronary model and propose a GPU based approach for fast rendering as described in section 2.2. For our experiments, the coronary tree of interest (left or right) was extracted by manually annotating the centerlines of the vessels followed by automatic segmentation of the coronary lumen, using the method described in [20]. On the X-ray series background subtraction is performed by subtracting from each pixel in the selected frame the average pixel intensity of this pixel in all frames of the X-ray sequence. This subtraction technique removes static structures such as the ribs from the image, resulting in a better similarity between DRRs and X-ray images. 2.2. Digitally reconstructed radiographs The intensity (I) of an X-ray traversing a combination of different structures can be described by I = I 0 e−



i

μi di

(1)

with I0 the input intensity, μi the linear attenuation coefficient of structure i, and di the thickness of structure i [21]. To simulate an X-ray image one thus needs to compute for every pixel the length of the ray from the X-ray source to that pixel passing through the different structures. These lengths can be computed efficiently on the GPU by assuming a constant attenuation of the contrast filling of the coronary arteries and using a polygon mesh representing the borders of the coronary lumen. By changing the sign of the distance from the source to where an X-ray intersects the polygon mesh to negative when the normal of the polygon is facing the X-ray source and positive otherwise, the resulting sum of these signed depth values corresponds to the length we are looking for (see for an example Fig. 2). This notion can speed up the DRR computation considerably as we do not have to compute all intersections of the rays with the 3-D voxels or polygons anymore, but we can simply draw all polygons using OpenGL

960

and compute the signed depth values by means of a fragment shader program. In this program the normals of the polygons are known and the distances can be computed from the values in the depth buffer of the graphics card. Accumulation of the distance values is performed using OpenGL blending. The exponential of equation (1) is not implemented, as X-ray images are approximately linear in the accumulated attenuation, because of processing in the X-ray detection system. By also employing the assumption of constant attenuation valuesfor the contrast material, equation (1) can be rewritten as: I = I0 i μi di . 2.3. Similarity measure Normalized cross correlation (NCC) is used to determine the similarity between the X-ray image and the DRR rendered for the current pose of the 3-D coronary model. This pose is described by parameter vector p, which has a translational (t) and rotational component (θ): (2) p = (t, θ)T , with translations described in millimeters and rotations (Euler angles) in degrees. NCC is subsequently described by: NCC(p) =

1  (f (x) − f )(g(p, x) − g(p)) , n−1 x σf σg(p)

(3)

with f (x) and g(p, x) the X-ray and DRR image respectively, n the number of pixels in the images and i and σi the average intensity value and standard deviation of intensity values of image i. 2.4. Initialization Registration of the 3-D model with the 2-D X-ray image is performed in a three step approach: automatic pre-alignment, in-plane manual initialization and automatic alignment. First, the 3-D model of the coronary arteries is automatically positioned in such a way, that it is aligned with the standard position of the patient on the table, having its center of mass on the origin of the world coordinate system, i.e. the 3-D position where the projection lines of the X-ray system intersect each other. After this automatic alignment step, a manual initialization is performed by clicking a point at the location of the ostium in the X-ray image. The ostium of the 3-D model is subsequently moved to this location by a 2-D translation in the plane orthogonal to the projection line. 2.5. Optimization After the initialization, a multi-resolution gradient ascent optimization procedure is applied [22]: pk+1 = pk + γ∇NCC(pk ) .

(4)

At every resolution level the input images are smoothed using a Gaussian kernel and the size of this kernel is reduced by a factor of two in the next level. To compute the gradient for the normalized cross correlation cost function, a central difference operator is applied with stepsize h. Convergence of the optimization is assumed when the magnitude of ∇NCC(p) is smaller than a positive constant c or when the number of iterations is larger than nmax . 3. EXPERIMENTS AND DATA Six CTA datasets and X-ray sequences of patients who underwent coronary angioplasty at Erasmus MC, Rotterdam, The Netherlands, were selected. CTA scans were acquired using a dual-source CT scanner (Somatom Definition, Siemens Medical Solutions, Forchheim, Germany). The slice thickness of the images is 0.75 mm and the images were reconstructed at end-diastole using retrospective ECG gating and a B30f reconstruction kernel. Dimensions and voxel sizes of the reconstructed volumes are approximately 512x512x350 and 0.35x0.35x0.40 mm3 respectively. The X-ray data was acquired with an AXIOM-Artis C-arm system (Siemens Medical Solutions, Forchheim, Germany). The pixel size of these images is 0.22x0.22 mm2 . The frame at end-diastole from the X-ray sequence was chosen based on the ECG signal, making sure the selected frame was part of a heart beat that was as regular as possible. Subsequently, registration was performed using the framework described in section 2. The projection matrix was calculated from the primary and secondary rotation angles of the X-ray system, and the source-to-detector and source-to-patient distance obtained from the DICOM header of the X-ray files. After the registration procedure, the resulting transformation and projection matrix were used to project the centerlines of the 3-D coronary model onto the 2-D X-ray images. Evaluation was performed by computing the average projection distance [6] between the projected centerlines and the centerlines manually annotated by an observer, for the two largest vessels. Based on pilot experiments on one dataset, the following parameter settings were established, which were γ = 1, h = 0.3, c = 0.02 and nmax = 200. For the multi-resolution registration approach, six resolution steps were performed with a standard deviation for the Gaussian kernel of respectively 16, 8, 4, 2, 1 and 0.5 pixels.

of the method was evaluated using CTA and X-ray images of six patients by computing the average projection distance between the projected centerlines of the 3-D model and manually annotated centerlines. The resulting average projection distance over all patients was 1.35 mm. Different sources for this error can be appointed. In the first place the errors can be caused by inaccurate reporting of the projection geometry of the X-ray system in the DICOM headers, causing the projection matrix to be incorrect. Second, the errors may be caused by a time-mismatch between the selected X-ray image and the reconstruction window of the CTA data. Moreover the CTA reconstruction window is larger than the window in which the Xray image is made, causing the 3-D coronary model to be an average spanning multiple X-ray images from the sequence. A third cause for errors may be irregular heart motion during the intervention caused by stress or (severe) pathology. Finally, deformation of the heart due to breathing motion is not taken into account as the CTA image is acquired during breath-hold. An in-depth analysis of these sources of error and their contribution to the final error is subject of future work. The 2-D error measure used can be relevant for the application of overlaying 3-D vessel structures on the X-ray images, but for navigation the real 3-D position of the guide-wire with respect to the CTA data must be determined. Future work will therefore focus on this 3-D evaluation, for which biplane X-ray imaging is required in order to determine the 3-D position of the guide-wire. With respect to speed, it may be noticed that the optimizer is not converging very fast. Convergence may be improved by implementing a line-search algorithm or by using a nonlinear conjugate gradient or quasi-Newton method [22]. Different metrics may be taken into account as well, which may improve the result of the alignment and eliminate the need for the presented background subtraction technique. Despite these issues, the current results are already very promising. The proposed method improved the alignment of the CTA and X-ray images in all six datasets. 6. REFERENCES [1] “The top ten causes of death - fact sheet n310,” World Health Organization, 2007. [2] B. Meier, D. Bachmann, and T.F. L¨uscher, “25 years of coronary angioplasty: almost a fairy tale,” The Lancet, vol. 361, no. 9356, pp. 527 – 527, 2003.

4. RESULTS An overview of the average projection distances for all vessels after manual initialization and after automatic registration, the running time for the algorithm, the number of optimization iterations and the number of DRRs rendered can be found in Table 1. Illustrations of registration results can be found in Fig. 3. The projected borders of the vessels after initialization (left image) and after automatic registration (right image) are overlayed on the X-ray images. As can be seen in Table 1 for ten out of the eleven vessels, the projection error improved noticeable after automatic registration. Fig. 3 gives an indication of the residual misalignment. 5. DISCUSSION AND CONCLUSIONS We presented a technique for 2-D/3-D registration of ECG gated CTA data of the coronary arteries to cardiac X-ray images. The method relies on a 3-D mesh representation of the coronary arteries, and a fast GPU based rendering method was proposed. Performance

961

[3] G. P. Penney, P. G. Batchelor, D. L. Hill, D. J. Hawkes, and J. Weese, “Validation of a two- to three-dimensional registration algorithm for aligning preoperative CT images and intraoperative fluoroscopy images.,” Med. Phys., vol. 28, no. 6, pp. 1024–1032, 2001. [4] L. Lemieux, R. Jagoe, D. R. Fish, N. D. Kitchen, and D. G. Thomas, “A patient-to-computed-tomography image registration method based on digitally reconstructed radiographs.,” Med. Phys., vol. 21, no. 11, pp. 1749–1760, 1994. [5] J.H Hipwell, G.P. Penney, R.A. McLaughlin, K. Rhode, P. Summers, T.C. Cox, J.V. Byrne, J.A. Noble, and D.J. Hawkes, “Intensity-based 2-D-3-D registration of cerebral angiograms.,” IEEE Trans. Med. Imaging, vol. 22, no. 11, pp. 1417–1426, 2003. [6] E.B. van de Kraats, G.P. Penney, D. Tomazevic, T. van Walsum, and W.J. Niessen, “Standardized evaluation methodology

Dataset

1

2

3

4

5

6

Mean

St.dev.

Running time (sec.)

23.7

8.1

25.5

10.5

11.3

1.95

13.51

9.21

Number of iterations

731

253

762

146

439

24

393

306

319

5110

3983

Number of DRRs rendered Vessel

9510

3296

9913

1905

5714

1

2

1

2

1

2

1

2

1

2

1

Distance before (mm.)

3.91

2.16

4.32

2.84

2.28

8.88

3.77

7.77

1.98

2.03

2.35

3.84

2.37

Distance after (mm.)

0.84

2.24

1.32

1.50

0.82

2.34

1.71

1.59

0.45

1.39

0.63

1.35

0.62

Table 1. Running time (seconds), number of optimization iterations and number of DRRs rendered for all datasets and average projection distance (in millimeters) before and after automatic registration for one or two large vessels in all datasets.

Fig. 3. Example of registration results for two datasets. Left image: after manual initialization. Right image: after automatic registration. for 2-D-3-D registration,” IEEE Trans. Med. Imaging, vol. 24, no. 9, pp. 1177–1189, 2005. [7] D.B. Russakoff, T. Rohlfing, D. Ruekert, R. Shahidi, D. Kim, and C.R. Mauere Jr., “Fast calculation of digitally reconstructed radiographs using light fields,” in Proc. SPIE (Med. Img.): Image Processing), 2003. [8] T. Rohlfing, D.B. Russakoff, J. Denzler, and C.R. Maurer Jr., “Progressive attenuation fields: Fast 2D/3D image registration without precomputation,” in Med. Img. Comp. and Comp.Assisted Intervention – MICCAI, Christian Barillot, David R. Haynor, and Pierre Hellier, Eds., 2004, vol. 3216 / 2004, pp. 631–638. [9] J. Weese, R. G¨ocke, G. P. Penney, P. Desmedt, T. Buzug, and H. Schumann, “A fast voxel-based 2D/3D registration algorithm using a volume rendering method based on the shear warp factorization,” in Proc. SPIE (Med. Img.: Img. Processing). 1999, l. [10] T.S.Y. Tang, N.J. MacIntyre, H.S. Gill, R.A. Fellows, N.A. Hill, D.R. Wilson, and R.E. Ellis, “Harware-assisted 2D/3D intensity-based registration for assessing patellar tracking,” in Med. Img. Comp. and Comp.-Assisted Intervention – MICCAI, 2004, vol. 3217, pp. 1095–1096. [11] D. Tomazevic, B. Likar, T. Slivnik, and F. Pernus, “3-D/2-D registration of CT and MR to X-ray images,” IEEE Trans. Med. Imaging, vol. 22, no. 11, pp. 1407–1416, 2003. [12] Harel Livyatan, Ziv Yaniv, and Leo Joskowicz, “Gradientbased 2-D/3-D rigid registration of fluoroscopic X-ray to CT.,” IEEE Trans. Med. Imaging, vol. 22, no. 11, pp. 1395–1406, 2003. [13] W. Wein, B. Roeper, and N. Navab, “2D/3D registration based on volume gradients,” in Proc. SPIE (Med. Img.: Img. Processing), J. M. Fitzpatrick and J. M. Reinhardt, Eds., 2005, vol. 5747, pp. 144–150.

962

[14] D. Tomazevic, B. Likar, and F. Pernus, “3-D/2-D registration by integrating 2-D information in 3-D,” IEEE Trans. Med. Imaging, vol. 25, no. 1, pp. 17–27, 2006. [15] J.H. Hipwell, G.P. Penney, T.C.S. Cox, J.V. Byrne, and D.J. Hawkes, “2D-3D Intensity Based Registration of DSA and MRA - A Comparison of Similarity Measures,” in Med. Img. Comp. and Comp.-Assisted Intervention – MICCAI, London, UK, 2002, pp. 501–508, Springer-Verlag. [16] E. Bullitt, A. Liu, S. R. Aylward, C. Coffey, J. Stone, S. K. Mukherji, K. E. Muller, and S. M. Pizer, “Registration of 3d cerebral vessels with 2d digital angiograms: clinical evaluation.,” Acad. Radiol., vol. 6, no. 9, pp. 539–546, 1999. [17] H. Sundar, A. Khamene, C. Xu, F. Sauer, and C. Davatzikos, “A novel 2D-3D registration algorithm for aligning fluoro images with 3D pre-op CT/MR images,” in Proc. SPIE (Med. Img.: Img. Processing). SPIE, 2006, vol. 6141, p. 61412K. [18] J. V. Byrne, C. Colominas, J. Hipwell, T. Cox, J. A. Noble, G. P. Penney, and D. J. Hawkes, “Assessment of a technique for 2d-3d registration of cerebral intra-arterial angiography.,” Br. J. Radiol., vol. 77, no. 914, pp. 123–128, 2004. [19] G.-A. Turgeon, G. Lehmann, G. Guiraudon, M. Drangova, D. Holdsworth, and T. Peters, “2D-3D registration of coronary angiograms for cardiac procedure planning and guidance.,” Med. Phys., vol. 32, no. 12, pp. 3737–3749, 2005. [20] M. Schaap, L. Neefjes, and C. Metz et al., “Coronary lumen segmentation using graph cuts and robust kernel regression,” in Information Processing in Medical Imaging, 2009. [21] M. Muniyandi and S. Cotin, “Real-time pc based x-ray simulation for interventional radiology training,” 2003. [22] J. Nocedal and S.J. Wright, Numerical optimization, SpringerVerlag, New York, 1999.