Advanced 2D-3D Registration for Endovascular Aortic Interventions: Addressing Dissimilarity in Images Stefanie Demircia , Oliver Kuttera,b , Frode Manstad-Hulaasc,d (MD), Robert Bauernschmittb (MD) and Nassir Navaba a Chair for Computer Aided Medical Procedures (CAMP), Technische Universit¨ at M¨ unchen, Boltzmannstr. 3, 85748 Garching, Germany; b Cardiovascular Surgery Department, TUM - Deutsches Herzzentrum M¨ unchen, Lazarettstr. 36 , 80636 M¨ unchen, Germany; c Department of Surgery, NTNU - St. Olavs Hospital, Olav Kyrres gate 17, 7006 Trondheim, Norway; d Department of Circulation and Medical Imaging, Norwegian University of Science and Technology 7489 Trondheim, Norway;
ABSTRACT In the current clinical workflow of minimally invasive aortic procedures navigation tasks are performed under 2D or 3D angiographic imaging. Many solutions for navigation enhancement suggest an integration of the preoperatively acquired computed tomography angiography (CTA) in order to provide the physician with more image information and reduce contrast injection and radiation exposure. This requires exact registration algorithms that align the CTA volume to the intraoperative 2D or 3D images. Additional to the real-time constraint, the registration accuracy should be independent of image dissimilarities due to varying presence of medical instruments and contrast agent. In this paper, we propose efficient solutions for image-based 2D-3D and 3D-3D registration that reduce the dissimilarities by image preprocessing, e.g. implicit detection and segmentation, and adaptive weights introduced into the registration procedure. Experiments and evaluations are conducted on real patient data. Keywords: Registration, Image-Guided Therapy, Intraoperative imaging, Procedures.
1. DESCRIPTION OF PURPOSE The implantation of an endovascular stent graft inside the aorta is a minimally-invasive procedure for the treatment of aortic aneurysms and aortic dissections. After the insertion of a pigtail catheter and guide wires, a shaft catheter including a folded stent graft is placed inside the aneurysm or dissection. Before unfolding the stent graft, the physician must ensure that branching vessels are not occluded. In this stage, misplacements of the stent graft can certainly lead to partial or total cut-offs of blood supply of vitally important organs and will lead to a life-threatening emergency surgery. The entire interventional catheter navigation is done under 2D angiography imaging where the physician is missing the important 3D information. As the catheter and stent position is only visualized in plane, more image acquisitions are needed during fine positioning of the stent graft before unfolding. This means an increase in radiation dose and used contrast agent at the same time as branching vessels need to be made visible in the Further author information: (Send correspondence to S.D.) S.D.: E-mail:
[email protected] O.K.: E-mail:
[email protected] F.M.: E-mail:
[email protected] R.B.: E-mail:
[email protected] N.N.: E-mail:
[email protected]
Medical Imaging 2008: Visualization, Image-guided Procedures, and Modeling, edited by Michael I. Miga, Kevin Robert Cleary, Proc. of SPIE Vol. 6918, 69182S, (2008) 1605-7422/08/$18 · doi: 10.1117/12.769835 Proc. of SPIE Vol. 6918 69182S-1 2008 SPIE Digital Library -- Subscriber Archive Copy
images. With the implementation of systems providing three dimensional angiography reconstructions during endovascular aortic repairs,1 the image information available for the interventionalist has been improved. However, such acquisitions produce an intense radiation exposure and are too time-consuming for constant application throughout the entire procedure. Physicians therefore choose to make three dimensional image acquisitions for difficult navigation and positioning tasks only once or twice during one intervention. However, normal size detectors of today’s interventional angiographic imaging systems generally cover a field of 30 x 40 cm; a region that doesn’t include enough information for navigation tasks during abdominal aortic interventions. Thus, there is a need for enhancing the intraoperative visualization with three dimensional image information. As the diagnosis and the planning of the procedure is done by means of a CTA (Computed tomography angiography), by integrating this volume into the intervention room, one can provide physicians with high quality 3D information during intervention. An alignment of the interventionally taken 2D or 3D image to the preoperative CTA volume provides a larger overview of the patient’s body and the accuracy of advanced technical guidance such as electro-magnetic tracking can thereby be optimized. In our previous work,2 we developed a fiducial-based 2D-3D registration algorithm for the alignment of X-Ray and CT images. Although fiducial markers are still necessary to evaluate registration results, they are highly disadvantageous for practical use. Here, we present an image-based solution, which in combination with the initial C-arm position information currently available within interventional angiography suites makes no use of fiducials. Existing image-based methods for 2D-3D3 and 3D-3D4 registration of interventional X-Ray and CT images give efficient and accurate results provided that dissimilarities in the images are limited. Interventional fluoroscopy images however also display medical instruments such as catheters and stent shafts that cover large parts of the image information due to high focus settings of the X-Ray machine. These interventional instruments are not present within the CTA data and can thereby cause registration algorithms to obtain wrong results. Further dissimilarities can be produced by varying presence of contrasted blood vessels. As the CTA image is by specification contrasted, existing rigid registration algorithms might not be able to match an uncontrasted intraoperative fluoroscopy image to it. Saturation effects of the contrasted aorta can cover large parts of bony structures that are crucial for an accurate registration (see Figure 2). It is the main objective of this paper to provide efficient, robust registration algorithms for aligning the preoperative volume to the intraoperatively acquired images. Here, the robustness constraint means independence of severe dissimilarities in the images. As a result of the registration, a roadmap for the catheter navigation can be displayed and the physician has access to all spatial information necessary for the exact graft placement. Eventually, we aim at an integrated navigation system for aortic interventions that enhances the accuracy of the surgeon’s actions and reduces the amount of contrast agent as well as radiation exposure.
2. METHODS 2.1 2D-3D Registration: We employ 2D-3D intensity-based registration to align the preoperative CTA images and intraporative X-Ray images. The algorithm iteratively optimizes the pose of a virtual X-Ray source by repeatedly generating DRR∗ images from the preoperative CTA while minimizing the image dissimilarity of the DRR and intraoperative X-Ray image. ∗
DRR - Digitally Reconstructed Radiograph
Proc. of SPIE Vol. 6918 69182S-2
Image Plane
Eye DigitaIly Recorristructed Radio graph (DRR)
Figure 1. Generation of weighted DRR.
DRR Generation Digitally reconstructed radiographs (DRRs) are produced by casting rays through a 3D CT volume. For its generation, we use GPU† accelerated raycasting, implemented in GLSL‡ , OpenGL’s § native high level shading language, similar to the work of Krger et al. 5 Our DRR renderer uses their scheme to determine the ray entry and exit points into the volume, however we employ a single render pass to compute the X-Ray attenuation Axy along each ray lxy : Axy = (−
p∈lxy
µ(p, E)dlxy )
where µ is a function for the conversion from CT houndsfield units to X-Ray attenuation values as described in Khamene et al. 6 Effectively, µ is a linear 1D transfer function which takes care of scaling, truncation, and saturation effects during conversion, that is implemented via dependent texture reads directly on the GPU. The DRR images are rendered into a 32bit floating point offscreen render target, via the OpenGL Framebuffer Object extension, and afterward transferred to the CPU for evaluation of the similarity metric. Our registration problem is then defined by the ten parameters of a rigid-body perspective projection as described in Penny et al. :3 six rigid-body parameters for translation and rotation in three dimensions and four paramteres concerning the perspective projection, focal length fx , fy and principal point px , py . Addressing Image Dissimilarities A problem for image-based registration for aortic interventions is the presence of contrast in the preoperative CTA dataset and the presence of surgical instruments in the intraoperative X-Ray images. DRR images generated from the CTA dataset contain a contrasted aorta, however intraoperative X-Ray images are not necessarily contrasted. This poses a problem for the image-based registrations as similarity metrics can hardly deal with such large dissimilarities. To address this problem we implemented a method to remove the contrast inside the aorta during DRR generation. The idea, simple yet effective, is to do a lookup in a second volume, containing only the segmented aortic tree, before updating the attenuation along the ray. If the lookup yields a value greater than zero we scale the value interpolated from the original preoperative CTA dataset with a user set value. Thereby we can interactively scale the value of the voxels of the contrasted aorta. †
GPU - Graphics Processing Unit GLSL - OpenGL Shading Language § OpenGL - Open Graphics Library, http://www.opengl.org ‡
Proc. of SPIE Vol. 6918 69182S-3
The intraoperative X-Ray images contain all medical instruments that are inside the detector’s field of view at acquisition time. In case of endovascular aortic interventions the image dissimilarities induced by these obstacle cover large parts of the image and therefore, affect existing registration algorithms. We therefore segment the instrument by a connected threshold filter (ITK¶ ) and then, during registration, apply a modified optimization scheme that only considers the background pixels for the similarity comparison. Thereby, we make sure that registration process only takes the anatomic data into account. Pose Optimization The above described methods are embedded into a framework for optimizing the pose of the virtual X-Ray source by various image similarity metrics and optimization algorithms. We have found Nearest Neighbour and Powell-Brents Direction search method to be the most robust optimization algorithms for our setup. During experiments that are described below, we compared several similarity measures including normalized cross correlation, normalized gradient correlation, sum of squared differences and sum of absolute differences. A full specification of these metrics can be found in Penney et al. 3 Similarly, we have found normalized cross correlation the most accurate similarity metric for our experiment datatsets. The optimal pose calculation was further optimized by a good initial guess that can be automatically extracted out of any mounted interventional angiography system. -a
I.
(a)
(b)
Figure 2. 2(a) shows a DRR image of a contrasted abdominal aortic aneurysm. Saturation effects cover the upper and lower parts of the spine. 2(b) was generated with the same settings and enabled removal of the contrasted aorta. Here, the entire bony structure of the spine can be used for registration.
2.2 3D-3D Registration: Due to the more and more common application of interventional 3D imaging, robust 3D-3D registration of these images to the preoperative CTA is of high interest. The medical instruments that are displayed inside the interventional 3D images prove to be disadvantageous for existing registration algorithms, as the image dissimilarities are too severe for general similarity measures. We propose a new solution for handling dissimilarities in 3D images for registration purposes. Instrument Removal First, the instruments are segmented in the interventional image by a connected threshold image filter provided by ITK. Let I be the image volume of width w, height h and slices d consisting of voxels (x, y, z) and let S denote the set of segmented voxels. Each segmented voxel V = (xv , yv , zv ) is then interpolated ¶
ITK - Insight Toolkit,7 Kitware
Proc. of SPIE Vol. 6918 69182S-4
in all directions x, y and z by setting the surrounding background voxels to knot points of B-Splines8 Cx , Cy , Cz : Cx (V ) =
w
I(i, yv , zv )Ni,3,τ (V )
i=0,(i,yv ,zv )∈S
where τ = (τ1 , ..., τw ) is the vector of knot points and Ni,3,τ are the basis functions that build up a certain B-Spline of degree 3. Cy and Cz accordingly calculate the B-Spline interpolation in direction y and z. The interpolated voxel value of V is then calculated by 1 (Cx (V ) + Cy (V ) + Cz (V )) 3 By applying the interpolation scheme to all segmented voxels all visible instruments are removed from the interventional volume (see figure 2) and the following 3D-3D registration procedure can be applied to align it to the CTA volume. Registration For evaluation purposes, our registration algorithm compares several similarity measures including normalized cross correlation, normalized gradient correlation and mutual information. For the registration process, we have found normalized gradient correlation the most accurate similarity measure for our experiment datasets. The optimization of the cost function is done by a best neighbour approach (also hill climbing) which we preferred to the Powell-Brent method as it proved to be more independent of initial pose settings and therefore more robust.
(a)
(b)
Figure 3. The medical instruments displayed inside the intraoperative 3D volume in 3(a) has been removed in 3(b).
3. RESULTS All experiments were performed on an Intel Core 2 PC containing 2.66 GHz CPU and 4096 MB of main memory. For the DRR computation we used GeForce 8800 GTX (NVIDIA Corp.) GPU with 768 MB dedicated GDDR3 memory. The evaluations of the proposed registration algorithms requires a selection of 2D and 3D datasets with varying presence of contrast agent and medical instruments. Each of them is then registered to the preoperative CTA volume by applying the appropriate registration algorithm. CTA images were acquired by either Siemens Somatom Sensation 64 or Siemens Somatom Definition, interventional 2D and 3D X-Ray images were taken by Siemens AXIOM Artis dTA angiography suite with dynaCT software. Thanks to the fully automatic handling of both the C-arm and the table, we were able to access all
Proc. of SPIE Vol. 6918 69182S-5
Dataset A B C D E F G H
Image resolution 256 x 256 x 219 256 x 256 x 219 256 x 256 x 219 256 x 256 x 219 1024 x 1024 1024 x 1024 1024 x 1024 1024 x 1024
Volume resolution (CTA) 512 x 512 x 280 512 x 512 x 398 512 x 512 x 651 512 x 512 x 398 512 x 512 x 398 512 x 512 x 398 512 x 512 x 398 512 x 512 x 399
Contrast no yes no yes no no yes yes
Instruments yes yes yes yes no no no yes
Table 1. Experimental datasets A-D are used for 3D-3D registration evaluation. Datasets E-H are used for 2D-3D registration evaluation. All preoperative data is contrasted and no medical instrument is visible whereas varying presence of contrast and medical instruments is given in the intraoperative images.
necessary extrinsic parameters and the two angulation parameters of the C-arm directly via DICOM header. Thereby, our registration problem was reduced from 10-DOF to 4-DOF. We strongly wish our registration methods to be integrated into the clinical workflow in the near future and therefore, performed experiments and validations on real patient data. As all images were acquired without fiducial markers sticked to the patient’s body, we calculated a ground truth transformation Tg for each 2D-3D and 3D-3D registration manually upon agreement with a medical expert. The validation of our 2D-3D registration algorithm is twofold. First, a performance evaluation of the DRR computation was conducted. Thereby, the influence of the additional code path for removing the contrasted aorta was evaluated for varying DRR image resolution and fixed maximum number of samples along a ray. All results are shown in Table 2. In the following accuracy evaluation of our 2D-3D registration method, six anatomical points were picked in the preoperative CTA volume and projected onto the intraoperative 2D image plane using a Gold-Standard-Transformation Tg which had been chosen manually by expert knowledge. Another image transformation Tr eg were computed by applying the appropriate registration method with an initial settings of each rigid body parameter to zero. The accuracy of our proposed algorithm was evaluated by mean target registration error (mTRE9 ). In Table 4, the resolution for each of the preoperative CTA and the intraoperative projection is given. The first line of parameter values shows the manually extracted gold-standard configuration DRR Resolution 256 x 256 512 x 512 1024 x 1024 256 x 256 512 x 512 1024 x 1024 256 x 256 512 x 512 1024 x 1024 256 x 256 512 x 512 1024 x 1024 256 x 256 512 x 512 1024 x 1024
Volume resolution 256 x 256 x 348 256 x 256 x 348 256 x 256 x 348 256 x 256 x 348 256 x 256 x 348 256 x 256 x 348 256 x 256 x 398 256 x 256 x 398 256 x 256 x 398 256 x 256 x 398 256 x 256 x 398 256 x 256 x 398 512 x 512 x 512 512 x 512 x 512 512 x 512 x 512
Contrast Removal no no no yes yes yes no no no yes yes yes no no no
Avg. Time [ms] 5.26 11.76 24.39 7.69 14.29 38.46 5.56 12.5 25.64 9.09 15.38 41.67 8.33 16.67 43.47
Table 2. Performance evaluation of DRR renderer for three volume datasets with different sizes.
Proc. of SPIE Vol. 6918 69182S-6
Data
Volume Resolution CTA 2D X-Ray
E
512x512x398
1024x1024
F
512x512x398
1024x1024
G
512x512x398
1024x1024
H
512x512x399
1024x1024
Tx (mm) -10.3 -10.1 2.4 2.0 -0.3 -0.1 0.2 0.0
Translation Ty (mm) Tz (mm) 3.9 -2.3 3.4 -2.3 -8.3 -3.2 -8.1 -3.2 -7.7 2.0 -7.0 2.0 1.3 3.0 1.6 3.0
θx (deg) 0.4 0.4 0.3 0.3 28.0 28.0 0.4 0.4
Rotation θy (deg) θz (deg) 0.5 0.2 0.5 0.2 -31.3 0.04 -31.3 0.04 -0.5 0.0 -0.5 0.0 0.5 0.25 0.5 0.24
mTRE 0.53 0.45 0.73 0.36
Table 3. Accuracy evaluation of 2D-3D registration.
Tg , the second line gives all values calculated by Tr eg. For a dataset D of ten random points, the mean target registration error in the last column is calculated by 10
mT RE(D, Treg , Tg ) =
Data
Volume Resolution CTA 3D X-Ray
A
512x512x280
256x256x219
B
512x512x398
256x256x219
C
512x512x651
256x256x219
D
512x512x398
256x256x219
Tx (mm) -23.7 -22.2 -16.0 -13.9 -3.8 -3.4 -16.6 -16.6
1 ||Treg di − Tg di || 10 i=1
Translation Ty (mm) Tz (mm) -12.0 -73.8 -10.0 -76.1 -8.0 22.9 -7.7 23.1 -3.3 25.0 -3.4 23.6 -28.7 -11.8 -28.7 -12.3
θx (deg) 0.04 0.05 0.03 0.03 0.0 0.0 -0.0 0.0
Rotation θy (deg) θz (deg) 0.05 0.05 0.04 0.05 -0.03 0.04 -0.02 0.03 0.0 0.1 0.0 0.1 0.0 0.05 0.0 0.05
mTRE 1.96 2.91 1.93 0.38
Table 4. Accuracy evaluation of 3D-3D registration.
A similar experiment was conducted for the accuracy evaluation of our 3D-3D registration method, shown in Table 4. Again, a gold-standard transformation Tg was defined by a medical expert. On the basis of ten randomly chosen points, the error of our registration method and the resulting transformation Treg was assessed by a mean target registration error (mTRE). In this experiment, however, all six rigid-body parameters had to be minimized.
4. CONCLUSION In this paper, we have presented a new solution for alignment of interventional 2D or 3D images to the preoperatively acquired CTA volume. The main advantage of our method is its robustness in terms of image dissimilarities which is achieved by image preprocessing and appropriate registration algorithms. A registered overlay of the interventionally taken 2D or 3D image to the preoperative CTA volume provides a larger overview of the patient’s body and therefore improves the insight view of the physician during difficult navigation tasks. In future work, we will focus on further optimizing our registration pipelines by moving as much of its stages from the CPU to the GPU. For a registration algorithm to work well intraoperatively on real patient data, it is crucial to take organ deformation into account. During endovascular aortic repairs, deformation of the aorta mostly occurs due to movement of the patient and medical instruments such as stiff guide wires and stent
Proc. of SPIE Vol. 6918 69182S-7
catheters. Furthermore, we will focus on integrating electro-magnetic tracking of the stent mounting system with the presented methods in the medical workflow enabling a correct and robust guidance of catheters and guide wires intraoperatively. This will bring us closer to our main goal of reducing contrast agent and radiation exposure during endovascular interventions.
Acknowledgements The research described in this paper was supported by Siemens AX (Angiography and X-ray). We would like to thank Asbjorn Odegard (MD), Department of Radiology, St Olavs Hospital, NO-7006 Trondheim, for making all experimental data available.
REFERENCES 1. K. R. Eide, A. Odegard, H. O. Myhre, and O. Haraldseth, “Initial observations of endovascular aneurysm repair using dyna-ct,” Journal of Endovascular Therapy 14, pp. 65 – 68, 2007. 2. O. Kutter, S. Kettner, E. U. Braun, N. Navab, R. Lange, and R. Bauernschmitt, “Towards an integrated planning and navigation system for aortic stent-graft placement,” Int’l J. of Computer Aided Radiology and Surgery , 2007. 3. G. P. Penney, J. Weese, J. A. Little, P. Desmedt, D. L. G. Hill, and D. J. Hawkes, “A comparison of similarity measures for use in 2-d3-d medical image registration,” IEEE Trans. Med. Imag. 17(4), pp. 586 – 595, 1998. 4. D. L. Hill, P. G. Batchelor, M. Holden, and D. J. Hawkes, “Medical image registration,” Phys. Med. Biol. 46(3), pp. 1–45, 2001. 5. J. Kr¨ uger and R. Westermann, “Acceleration Techniques for GPU-based Volume Rendering,” in Proceedings IEEE Visualization 2003, 2003. 6. A. Khamene, P. Bloch, W. Wein, M. Svatos, and F. Sauer, “Automatic registration of portal images and volumetric CT for patient positioning in radiation therapy,” Medical Image Analysis , pp. 96–112, Feb. 2006. 7. L. Ibanez, W. Schroeder, L. Ng, and J. Cates, “The itk software guide, second edition,” 2005. 8. P. Dierckx, Curve and Surface Fitting with Splines, Oxford University Press, 1995. 9. E. van de Kraats, G. Penney, D. Tomaˇzeviˇc, T. van Walsum, and W. Niessen, “Standardized evaluation methodology for 2-d?3-d registration,” IEEE Trans. Med. Imag. 24(9), pp. 1177–1189, 2005.
Proc. of SPIE Vol. 6918 69182S-8
