Validation of 3D Ultrasound - CT Registration of Prostate images Evelyn A. Firle, Stefan Wesarg, Grigorios Karangelis and Christian Dold Fraunhofer IGD, Fraunhoferstr. 5, 64283 Darmstadt, Germany ABSTRACT All over the world 20% of men are expected to develop prostate cancer sometime in his life. In addition to surgery - being the traditional treatment for cancer - the radiation treatment is getting more popular. The most interesting radiation treatment regarding prostate cancer is Brachytherapy radiation procedure. For the safe delivery of that therapy imaging is critically important. In several cases where a CT device is available a combination of the information provided by CT and 3D Ultrasound (U/S) images offers advantages in recognizing the borders of the lesion and delineating the region of treatment. For these applications the CT and U/S scans should be registered and fused in a multi-modal dataset. Purpose of the present development is a registration tool (registration, fusion and validation) for available CT volumes with 3D U/S images of the same anatomical region, i.e. the prostate. The combination of these two imaging modalities interlinks the advantages of the high-resolution CT imaging and low cost real-time U/S imaging and offers a multi-modality imaging environment for further target and anatomy delineation. This tool has been integrated into the visualization software ”InViVo” which has been developed over several years in Fraunhofer IGD in Darmstadt. Keywords: Registration, CT, 3D U/S, Brachytherapy, Prostate
1. INTRODUCTION The traditional treatment for cancer is surgery. A modern and very rapidly growing alternative is radiation treatment. In the U.S. the radiotherapy market is forecast to expand at a 7.8% annual rate up to about $ 800 million by the year 2003. A significant amount is due to prostate cancer which is a growing concern all over the world. Brachytherapy radiation treatment is the most promising therapy for prostate cancer at the present, as it offers similar efficacy and considerable lower morbidity when compared to radical prostatectomy and external beam radiation, and because of the lower risk of incontinency and impotency. In addition the costs of Brachytherapy employing seeds ranging from $ 10,000 to $ 15,000 are lower than those for radical prostatectomy where they range from $ 20,000 to $ 30,000. Brachytherapy is a technique where the radioactive sources are inserted in the patient’s body, is conducted by two principal methods: Either by implanting radioactive seeds of low activity (LDR) and leaving them in the tumour forever (continuously decreasing activity of the seeds because of the decay), or by a short-duration treatment using a computer driven single high activity radioactive source (HDR) and catheters or needles inserted temporary in the tissue. Present treatment conditions are as follows: Prior to insertion of seeds or catheters, usually a CT scan (occasionally a 2D-U/S scan is used for prostate) is conducted as basis for planning the number and position of radioactive sources necessary. After insertion of the seeds or catheters - sometimes under 2D U/S guidance -, an additional scan is required to reconstruct the final positions of the needles. The planning systems are then calculating the radiation pattern as the result of this geometry. This procedure has a number of shortcomings. Registering available CT Scans with 3D U/S volumes could improve the whole therapy. In several cases where a CT device is available a combination of the information Further author information: (Send correspondence to E.A.F.) E.A.F.: E-mail:
[email protected], Adress: Fraunhofer IGD, Cognitive Computing & Medical Imaging, Fraunhoferstr. 5, 64283 Darmstadt, Germany S.W.: E-mail:
[email protected] G.K.: E-mail:
[email protected] C.D.: E-mail:
[email protected]
provided by CT and 3D Ultrasound (U/S) images offers advantages in recognizing the borders of the lesion and delineating the region of treatment. For these applications the CT and U/S scans should be registered and fused in a multi-modal dataset. This fused data can provide more information to the clinicians for a better diagnosis and treatment planning than one single imaging modality. Each of the modalities has its advantages and disadvantages. High resolution CT is often used for segmenting the planning target volume (PTV) - here the prostate -, the organs-at-risk (OAR) - e.g. urethra and bladder - and for calculating the dose to be delivered. But for navigating the catheters U/S has advantages over CT because of its real-time capability. The static CT picture is a poor basis for conducting the radiation itself for two reasons: First, the position of the patient can be different between CT acquisition and treatment, especially when soft tissue is involved. Second, CT scan acquired prior to the seed or needle insertion provides no real-time indication of true position after insertion in the body relative to their planned position. A preceding step for the fusion of volume data is the registration of the data sets to be fused. Currently there is a growing interest in algorithms for the registration of volumes from different or the same modalities. For this calculation the positions of reference points, which can be found in both data sets have to be known if the registration cannot be performed on the information delivered by the image itself (e.g. the grey value). These points can be anatomical landmarks, contours in the data set or externally attached fiducially markers. Here, we used the geometry urethra which was visible and segmented in both modalities. Purpose of the present development is a registration tool (registration, fusion and validation) for available CT volumes with 3D U/S images of the same anatomical region, i.e. the prostate. This tool has been integrated into the visualization software ”InViVo”. This software is a system for visualisation of medical volume data, such as CT, MRI or 3D ultrasound data, that has been developed over several years in the Fraunhofer Institute for Computer Graphics in Darmstadt (see Ref. 1, 2 and 3). For this application the CT data can be read in by the software directly from the CT device via the hospital network in DICOM 3 format. The data can be displayed as 2D CT scans, oblique oriented slices in the volume data or as 3D visualisation. The available modes for the 3D volume rendering are for example: maximum/minimum intensity projection, X ray-simulation and surface reconstruction as semitransparent cloud or as gradient shaded surfaces (see Ref. 4, 5, 6 and 7). The data acquisition has been done in cooperation with the clinical partners at the Clinical Complex Offenbach/Strahlenklinik.
2. MATERIALS AND METHODS 2.1. Phantom Study The phantom study was performed using a CIRS Model 53 Ultrasound Prostate Training Phantom (see Figure 1). This disposable phantom was originally developed for practicing procedures which involve scanning the prostate with a rectal probe. The prostate along with structures simulating the rectal wall, seminal vesicles and urethra is contained within an 11.5cm x 7.0cm x 9.5cm clear acrylic container. A 3mm simulated perineal membrane enables various probes and surgical tools to be inserted into the prostate. More information about the prostate phantom can be found on the Internet at http://www.cirsinc.com/products/model053a.html. Several CT and U/S scans of the phantom were performed. Two of the images were acquired of the phantom without any implants, another two with implanted plastic catheters (one with catheters and one with 7 catheters) and a last one with implanted seeds.
2.2. Clinical Study The clinical study was performed in three patients with prostate cancer. These patients received radiation treatment using Brachytherapy (HDR). CT and 3D U/S scans were acquired pre- and postplanning, i.e. before and after the insertion of the hollow needles. Plastic catheters were used for the irradiation.
2.3. Catheters For the radiation treatment Nucletron ProGuide 6F plastic catheters with a length of 200 mm and a diameter of 2 mm were used.
Figure 1. Prostate phantom with inserted plastic catheters
2.4. Data acquisition 2.4.1. CT scanning procedure The CT scans were performed on a Siemens Somatom 4Plus (Siemens, Erlangen, Germany). For the clinical study the general protocol for CT scanning of the Department of Radiation Oncology was used, with a dimension of 512x512 pixels (0.51 mm) and a slice thickness of 3.00 mm. For both CT scans the patients were positioned on the scanner bed with the tumor in the center of the FOV (FOV ≈ 65530 mm2 ). 2.4.2. 3D U/S scanning procedure We acquired ultrasound prostate image sets of the phantom (three different cases) and three different patients. for the phantom study we used 7.5 MHz ESAOTE TRT-12 bi-planar U/S probe. For the clinical study the B&K falcon device with a Multifrequency 8658 transrectal U/S probe was taken. The usage of the lateral scan plane minimizes the deformation of the organ. Images were acquired in a pre- and post-planned clinical condition (i.e. before and after implanting the catheters). The U/S devices were directly connected to a computer and images stored digitally. In the phantom study the images have voxelsizes of 0.42 mm in each direction and a resolution of 162 x 148 x 172. In the clinical study, the images are 5 mm apart, and have a pixel size of 0.31 mm, and a matrix size of 248 x 217 (FOV ≈ 9490 mm2 ).
2.5. Registration procedure The process of image registration can be formulated as a problem of minimizing a cost function that quantifies the match between the images of the two modalities. In order to find this function different common features of these images can be used such as: points visible in both (fiducially markers or landmarks), outer contours of different structures (e.g. organ or outer body contour) or the voxel intensity. Our registration process is based on the contour of the urethra. Therapeutic decisions require the delineation of the urethra (OAR) and the prostate (PTV). This segmentation is therefore necessary for the preplanning, that means for planning the ideal positions of the catheters respective seeds and the dose to be delivered. So our first step after the acquisition of both images is to segment the anatomy of the urethra in both modalities. This segmentation step can be performed semi-automatically using an active contouring model.8 These algorithms are also used for acquiring the prostate boundary semi-automatically.9 Using the resulting two sets of points for the urethra (one for each modality) we calculate a transformation matrix that reflects the correlation between the two modalities. There are two different methods we compared to determine the transformation matrix. The first one is based on the Iterative Closest Point Principle10 and the second one on the Distance Map.11 2.5.1. Iterative Closest Point The first method investigated is an iterative process using the point sets of the contour from each modality. For each contour point of the urethra segmented in the CT scan the closest point on the contour of the U/S urethra
is taken using a starting transformation. From this correlation a new transformation matrix using the least square fitting is calculated. With that transformation a new set of points can be determined, which is then used for a next iteration. In each step the mean distance between all points is calculated and the whole procedure is repeated until the difference of two successive distances under-runs a user defined tolerance. This procedure converges to a minimum distance. After the registration a final distance between the urethra delineated in both modalities with respect to the final transformation was measured. This final distance is calculated using the 3D Distance Map of the urethra. 2.5.2. Distance Map The second method uses the 3D Distance Map of the contoured objects to calculate the transformation matrix. I.e. given the 3D Distance Map DMijk with respect to the urethra delineated in the CT scan, we consider the Similarity Measure Value defined by N 1 SM V = DMi j k , (1) N m=1
where N is the amount of points belonging to the contoured object of the U/S volume and (i j k ) correspond to (i, j, k) using the transformation matrix. Employing the optimisation method Simulated Annealing (SA) we minimize that function to calculate the best transformation matrix between the two modalities.
2.6. Evaluation Methods To assess the accuracy of a registration points, landmarks or other structures that are identifiable in both modalities are possibilities to measure the quality of a registration. After a transformation is found the final distance between these points or structures has then to be determined. In our case we used the plastic catheters to assess the accuracy. We therefore segmented them in both modalities and calculated the appropriate positions using the transformation. In case of the phantom study we additionally delineated the prostate and calculated the error of the registration. 2.6.1. Phantom Study Each of the U/S phantom datasets was registered with the CT volume and the results were checked visually.11 For evaluating the study we then just considered the two datasets with six respective seven implanted plastic catheters and measured: 1. The mean distance between each needle and between the prostate of both modalities after registration. 2. The standard deviation of the distances for each catheter and the prostate. Each of these values was calculated for both methods (ICP and Distance Map). Here we differentiated by using a different amount of contours for delineating the urethra. In the phantom dataset with seven catheters the urethra was segmented in each slice and in the other dataset the object was simply segmented every second or third slice to speed up the time required for contouring. 2.6.2. Clinical Study For each of the three patients, the post-planning CT dataset was registered to the post-planning U/S dataset. Using these data, the accuracy of matching CT and U/S scans for the patient was determined. Here we again investigated the following aspects: 1. The mean distance between each needle of both modalities after registration. 2. The standard deviation of the distances for each catheter. Each of these values was calculated for both methods (ICP and Distance Map).
Figure 2. Superposition of the delineated catheters implanted in the phantom after matching of CT and U/S.
3. RESULTS 3.1. Phantom Study Figure 2 illustrates the superposition of the catheters implanted in the phantom after registration. The duration for the calculation of the registration took between 0.65 and 2.44 sec for the ICP method and between 11.03 and 19.35 sec for the Distance Map method. In the latter case, the calculation of the Distance Map for the two modalities required the biggest amount of the calculation time. See Fig. 3 for an example of the matched volumes. Table 1 shows the detailed results of the registration of the post-planning U/S and CT scan of the phantom. The distances between the catheters and the prostate applying the calculated transformation are determined by calculating the Distance Map for each object. The results retrieved using the ICP method shows better results even though the matching of the urethra was comparably good. The results show also that carefully contouring the objects that are used for matching the modalities, significantly improves the whole process such that the deviation then ranges in the size of voxel level.
3.2. Clinical Study Table 2 summarizes the results of registering the post-planning CT and U/S patient datasets. These results were comparable with the other patient datasets examined. The duration for the calculation of the registration required between 0.22 and 6.44 sec for the ICP method and between 18.97 and 30.62 sec for the Distance Map
Figure 3. Prostate phantom after registration with inserted plastic catheters in the coronal view. The upper two images show the fusion of both cutting planes in a sort of chess bord pattern.
Table 1. Calculated results of matching the phantom datasets with implanted catheters. The final distance for the urethra after matching was 0.83 respective 0.84 mm (phantom with 7 catheters) and 0.74 respective 0.60 mm (phantom with 6 catheters). The calculation of the transformation took 0.65 respective 11.03 sec (phantom with 7 catheters) and 1.78 respective 19.35 sec (phantom with 6 catheters)
Method ICP
Distance Map
catheter catheter catheter catheter catheter catheter catheter prostate catheter catheter catheter catheter catheter catheter catheter prostate
1 2 3 4 5 6 7 1 2 3 4 5 6 7
Dataset 1 (7 Catheters) mean distance (mm) 0.96 0.76 1.07 0.94 0.96 1.51 1.28 1.42 0.70 1.38 0.62 1.03 0.58 0.91 1.33 2.62
deviation (mm) 0.89 0.84 0.77 1.84 0.82 1.33 1.22 2.52 0.49 1.60 0.46 1.20 0.56 1.03 1.43 4.61
Dataset 2 (6 Catheters) mean distance (mm) 0.48 0.77 0.61 0.61 0.44 0.74 0.77 0.38 1.08 1.12 1.29 1.03 0.72 2.55
deviation (mm) 0.41 0.64 0.54 0.43 0.43 0.67 0.47 0.26 0.98 0.94 0.98 0.87 0.55 3.20
method. Here again, the calculation of the Distance Map for the two modalities required the biggest amount. See Fig. 4 and Fig. 5 for different examples of the matched volumes. There as well the CT and U/S cuts through the volume are displayed as the superposition of both information after registration. As already seen in the phantom study, the results retrieved using the ICP method are clearly better than the registration results using the Distance Map method. For the patient data the registration using the latter lead to a serious mismatch which could also be detected visually (Fig. 6). This mismatch as well as the differences of the clinical study compared to the phantom study can be explained by the differences in the U/S acquisition in both studies. As described in 2.4 a stepsize of 5 mm was used for the U/S patient volume and a slice thickness of 0.42 mm was used in the phantom study. Thereby the geometry of the urethra looses its characteristics and can not be delineated as precise as in the CT volume. Furtheron may deformations occur during U/S acquisition that lead to a less good registration in the outer parts of the volume. Therefore the comparison of the catheters in that region are less accurate.
4. DISCUSSION Accurate delineation of the ROI and PTV is very important for radiotherapy, but clearly also for multimodal matching when the method is based on one of these objects. To summarize the results of section 3, a comparably good accuracy as in the phantom study could not be achieved for the clinical study. But this process could be improved by a few different factors. Firstly a higher resolutional clinical study (as in the phantom study) could be performed. Then the contrast of the urethra in U/S acquisition could be improved by using a catheter. Furthermore the registration method could be expanded for deformable structures - however only rigid body transformations were used. More patient datasets would be required to yield a significant improvement of the registration. These datasets would also be necessary to validate the algorithms that are currently under development. These developments will include registration methods for CT and U/S based on the pure voxel information and region based elastic
Figure 4. Registration results of the three different patient datasets after needle insertion (axial cuts). Each of them shows on the left hand site the CT scan on the right hand site the appropriate cut through the 3D U/S volume and in the middle an intermixing of both images. Figure 1 displays the contours of the prostate and the urethra drawn within the U/S volume, figure 2 displays the contours of the prostate, urethra and rectum delineated using the CT scan and in figure 3 the delineation of the prostate and the urethra from CT can be seen. The black (CT) respective white (U/S) dots are the catheters used for the treatment. These catheters are herewith also used for measuring the accuracy of the matching.
Table 2. Calculated results of matching the post-planning CT and U/S patient datasets with implanted catheters. The final distance for the urethra after matching was 1.07 (deviation of 0.88) respective 1.76 (deviation of 0.67) mm (depending on the method, i.e. ICP respective Distance Map, see 2.5 for details). The calculation of the transformation required 0.22 respective 18.97 sec for the different methods.
Method ICP
Distance Map
catheter catheter catheter catheter catheter catheter catheter catheter catheter catheter catheter catheter
1 2 3 4 5 6 1 2 3 4 5 6
mean distance (mm) 2.37 3.18 2.57 5.89 3.54 3.80 16.75 15.19 16.34 12.94 19.97 16.11
deviation (mm) 1.95 3.71 2.76 7.89 4.90 6.95 29.26 39.35 14.08 18.69 26.29 34.06
Figure 5. Registration result of one post-planning patient datasets after needle insertion (sagital cuts). The left image shows the CT scan and the right image the appropriate cut through the U/S volume after registration. In the middle image the matching of the volumes is presented. The catheters are visualized as black (CT) respective white (U/S) lines.
registration methods which can probably improve the matching since the U/S scanning process causes deformations of the organ examined.
5. CONCLUSION The introduced system offers a fast tool for segmenting and registering CT volumes with 3D U/S images of the prostate using the geometry of the urethra visible in both modalities. The segmentation is realized semiautomatically using an active contouring model and the registration algorithm based on the chosen geometry provides an estimation of the correlation between the different volumes. Besides developing the fusion software for a quick display of fused images from any modality we implemented different validation methods to present the results of the registration based on the implanted plastic catheters. This validation methods determine the mean distance between the implanted catheters based on the transformation after matching and the deviation. In this report the phantom study as well as the clinical study was presented.
Figure 6. Misregistration of one post-planning patient datasets after needle insertion (axial view) using the Distance Map method. In the middle image the matching of the volumes is presented. The misregistration can be seen by comparing the catheter positions (visualized black in CT respective white in U/S).
ACKNOWLEDGMENTS This work is partially funded by MITTUG a project of the European Commission (beginning 01/2000). The project number is IST-1999-10618. More information can be found on the Internet at http://www.igd.fhg.de/igd-a7.
REFERENCES 1. G. Sakas and S. Walter, “Extracting surfaces from fuzzy 3d ultrasonic data,” in International Conference on Computer Graphics and Interactive Techniques, Proc. SIGGRAPH 22, pp. 6–11, ACM Press, 1995. 2. G. Sakas and M. G. L. A. Schreyer, “Visualization of 3d ultrasonic data,” in IEEE Visualization, Annual IEEE Conference on Visualization 5, pp. 369–373, 1994. 3. G. Sakas, S. Walter, W. Hiltmann, and A. Wischnik, “Foetal visualization using 3d ultrasonic data,” in Computer Assisted Radiology and Surgery, H. U. Lemke, ed., Proc. CAR 95 6, pp. 241–247, 1995. 4. K. H. Hhne and R. Bernstein, “Shading 3d-images from ct using gray level gradients,” IEEE Trans. Med. Imaging 5, pp. 45–47, 1986. 5. R. Ohbuchi, D. Chen, and H. Fuchs, “Incremental volume reconstruction and rendering for 3d Ultrasound Imaging,” in Visualization in Biomedical Computing, Proc. SPIE 1808, pp. 312–323, 1992. 6. D. H. Pretorius and T. R. Nelson, “Opinion: Three-dimensional ultrasound,” Ultrasound Obstet. Gynecol. 5, pp. 219–221, 1995. 7. M. Levoy, “Display of surfaces from volume data,” IEEE Computer Graphics and Applications 8, pp. 29–37, 1988. 8. S. Grokopf, S. Y. Park, and M. H. Kim, “Segmentation of ultrasonic images by application of active contour models,” in Computer Assisted Radiology and Surgery, H. U. Lemke, ed., Proc. CAR 98, p. 871, 1998. 9. E. A. Firle, “Semi-automatische segmentierung der prostata mit hilfe von 3d ultraschallaufnahmen,” in Bildverarbeitung fr die Medizin 2001, H. Handels, A. Horsch, T. Lehmann, and H.-P. Meinzer, eds., Proc. BVM, pp. 262–266, 2001. 10. P. Besl and N. McKay, “A method for registration of 3-d shapes,” IEEE Trans. Patt. Anal. And Mach. Intell. 14, pp. 239–255, 1992. 11. E. A. Firle, W. Chen, and S. Wesarg, “Registration of 3d u/s and ct images of the prostate,” in Computer Assisted Radiology and Surgery, H. U. Lemke, ed., Proc. CAR 02, pp. 527–532, 2002.
