Development of 2D+T tracking algorithm in ultrasound images for radiotherapy Abdenaceur Abdouni, Benoît Presles, Marie Fargier-Voiron, Simon Rit and David Sarrut
Abstract— The aim of this study is to develop and validate a deformable tracking algorithm for monitoring the motion of the target volume on 2D ultrasound (US) images during a radiotherapy fraction. The proposed method is applied on images acquired with a transperineal ultrasound (TP-US) probe on 31 treatment patient’s sessions, treated with a prostate or after a surgery, called a prostatectomy. The developed algorithm is based on Speeded-Up Robust Features (SURF) to find and match the corresponding salient points in the reference and moving images, and Thin Plate Spline (TPS) to warp the image. The results are promising and show that the proposed algorithm performs well with either artificial transforms, or in comparison with a rigid intensity based algorithm used in clinic.
I. INTRODUCTION Cancer is a leading cause of death in economically developed countries [1]. More particularly, prostate cancer is the second leading cause of cancer death [2][3], by causing for instance the death of 29,480 males from 233,000 in North American in 2014[1]. Different treatments can be used for treating this localization, such as surgery or radiotherapy. For the latter, patients can be treated with an intact prostate or after a surgery, called a prostatectomy. It has been shown that the prostate and the prostatic bed can be deformed during a treatment session [4], hence the tracking of the target volume plays an important role. Few solutions exist in radiotherapy to track a target volume: 2D kV, electromagnetic transponders and ultrasound (US) with a transperineal (TP) sensor. Several tracking algorithms have been proposed in the literature, with different US probes and for different organs. Tracking methods can be classified into two main categories. The first category, called intensity-based methods is based on maximizing (or minimizing) a specified similarity measure, between intensities of corresponding pixels. Different measures were developed such as cross correlation for rigid and elastic registration [5][6]. Solutions were proposed for deformable tracking based on block matching algorithm or B-spline method[7][8]. The problem of intensity-based is the large dependence on the image quality, and ultrasound images increase this difficulty. The second category encompasses feature-based methods that are based on finding the correspondence between image features such as points, lines or contours. Thus, a semi-automatic deformable method was proposed using SURF to detect the corresponding points and TPS to warp the image. In this study, the region and object of A. Abdouni is with Léon Bérard Cancer Center; Université de Lyon; F69373 Lyon, France (e-mail:
[email protected]). Benoît Presles, Marie Fargier-Voiron, Simon Rit and David Sarrut are with Université de Lyon, CREATIS, CNRS UMR5220, Inserm U1044,
interests were manually defined[9]. A method based on tracking some inserted markers on the prostate organ was proposed [10], another method based on edge detection and active contour, were used to track the region of interest was realized [11]. The difficulties in the feature based methods are to extract the features in the images and finding correspondences between them. The edge detection and active contour have a drawback of being prone to the errors caused by the existence of spurious (noise-induced) edges. Only one paper in the literature realized an algorithm with TPUS probe for prostate. This paper proposed an intensity-based rigid method that uses normalized cross correlation as similarity measure [12]. It was implemented in the commercial software of the TP-US probe. The prostate volume is tracked and its translations and rotations are measured. However, it has been shown that the prostate is subject to deformation [13][14],which means that the rigid tracking is not the optimal choice for this kind of application. The obtained results with the commercial system [12], showed that the prostate can move on the three axis, but the deformation on the left/right axis is much less comparing to the other axis. Due to this reason a 2D+T tracking method could be a good start in this field. The goal of this work is to develop a deformable 2D+T algorithm that can track the region of interest, i.e. the prostate or the prostatic bed, and therefore improve the system precision. The rest of the paper is organized as follows. In Section II, an overview about the used US system is given. Then, the proposed method is detailed, and the validation and optimization technics are explained, while Section III presents the experimental results. Finally, Section IV summarizes the paper and gives some prospects. II. METHOD AND MATERIALS
A. The US device: In the present study, the US Clarity® Auto-scan system (Elekta) is used to visualize organs of the pelvic zone. The system consists of a TP-US probe that enables to acquire images of the prostate area through the acoustic window of the perineum. A high frequency (5 MHz) can be used due to the short distance between the ultrasound probe and the pelvic zone, so the system collects good quality images.
INSA-Lyon, Université Lyon 1, Lyon F-69621, France. (e-mail:
[email protected],
[email protected],
[email protected] and
[email protected])
The TP-US probe is a 2D probe with a motorized control of the sweeping motion (Fig.1). An image database of 31 sessions from 11 patients (5 prostate and 6 prostatectomy patients) have been created. The number of sessions is different from one patient to another, and varies from 2 to 5. This study was approved by the hospital ethics committee. All included patients signed a letter of consent The 2D+T tracking algorithm is based on the periodicity of the TP-US acquisition. To scan the full region, the system captures t=248 images. The chosen position of the reference image is denoted R (the red line in Fig.1). The reference image corresponds to (𝑇𝑛−1 = (n-1)*t+R) and the moving image corresponds to (𝑇𝑛 =n*t+R) where (n =1,2,..) is the number of sweeps. So, the comparison is done between 𝑇𝑛−1 (reference image) and 𝑇𝑛 (moving image) that have the same position but with some time delay. TP-US probe
Figure 1 The diagram represents the sweep motion of the probe. Where the red line corresponds to the spatial position of the 2D images used for registration.
B. The proposed approach: The proposed algorithm follows the steps illustrated in (Fig.2): Create a mask
calculate SURF points
Filter SURF points
add border points
apply TPS
Figure 2: Schema of algorithm procedure
1) A mask is created (Fig.3b) to differentiate the background from the foreground on the reference and moving images. 2) The salient features are detected in both images by using the Speeded-Up Robust Features (SURF) algorithm (Fig.3c). The SURF is a technique used to find points correspondences [15][16]. This algorithm can be divided into four main steps: interest points detection, interest points localization,
(a) US reference image
(b) Mask of the reference image
orientation assignment, descriptor calculation and matching [17][18]. The SURF depends on some parameters: the number of salient points (denoted α) and the number of octaves (denoted β). 3) The Euclidian distance between the SURF descriptor is used to match the interest points in the two images. 4) To keep only the best interest points, the detected SURF landmarks are filtered in both images to (Fig.3d): • ensure that all the points take information only from the foreground. The SURF points are calculated in the entire image and then the detected mask (step 1) is eroded by the maximum radius of the SURF descriptor. Finally, only the points that are inside the eroded mask are kept. The descriptor radius is denoted r; • remove closed SURF points. In a circle of three pixels around each interest point, only the point that have the lowest descriptor distance is kept, this step ensures that the SURF points cover the whole image and each SURF point has different information. The minimum distance between each two kept points is denoted dmin; • eliminate all the points that have the Euclidian distance between the SURF descriptors larger than a defined threshold. This step ensures that outliers are removed. This threshold is denoted T; • to ensure that the Thin Plate Spline (TPS) will not deform globally the image, border points are added as fixed corresponding points between the two images. 5) The TPS is then applied to transform the moving image thanks to the previous matched points. The TPS is a deformation map of a set of homologous landmarks in one form to the corresponding landmarks of another form. The deformation map is a smooth interpolation function that predicts the positions of points that lie between the known landmarks. A thin plate smoothing is produced by minimizing the bending energy [19][20]. C. Parameters optimization and validation: To optimize and validate the algorithm, two artificial known deformations similar to real deformations were applied on the prostate region: first, bulge transformation that can artificially enlarge the prostate region by deforming the image (Fig.4), second, local translation of the target region. To quantify the proposed algorithm, the mean Target Registration Errors (TRE) around the target region before and after registration were calculated.
(c) SURF points
(d) Filtered SURF points
Figure 3 Illustration of the algorithm process on the reference image
(e) Border + filtered points
(a) Reference image (b) Bulged deformed image Figure 4 Illustration of the bulge deformation
1 The equation of the mean of TREi is TREi= ∑𝑘𝑖=0 ∥ (𝑇̂(𝑝𝑖 ) − 𝑘 𝑝𝑖 ) ∥, where k is the number of points in the region of interest, 𝑇̂ is the estimated transformation by the proposed algorithm and 𝑝𝑖 are the points coordinates in the region of interest. This TRE indicated the amount of deformations before registration. As the applied transformation was known, the final mean of TRE, denoted TREf, between the deformed image by the proposed algorithm and by the artificial transform can be 1 calculated. TREf = ∑𝑘𝑖=0 ∥ (𝑇̂(𝑝𝑖 ) − 𝑇′(𝑝𝑖 )) ∥, where 𝑇̂ is 𝑘 the transformation estimated by the proposed algorithm, T’ is the real transformation. This TRE represented the amount of deformation after registration. In order to have the best results, the algorithm parameters were first optimized. The optimization was done by using the published papers recommendations [9] [15] and by running some tests. The published papers had been used to limit the number of tested values. Then, the experimental tests were done with both artificial transforms on the full database to choose the best parameters.
III. RESULTS
The algorithm was optimized on all the database by using the previous artificial transformations. The results shows that the optimized parameters values were: α= 137, β= 5, r= 12, dmin=3 pixels, T= 0.00002.
Figure 6 TRE value as a function of bulge deformation
Local translation results (Fig.5) showed that the maximum TREf value was less than 2 mm with an initial TREi of 12 mm. According to the clinical algorithm, the maximum prostate translation was around 8 mm, for this value the algorithm showed a TREf less than 1 mm. Bulge deformation results (Fig.6) showed that the maximum TREf value was less than 1.5 mm with a TREi less than 8 mm. In both results a part of the error was due to the interpolation generated by the artificial transforms. The interpolation created a new smooth region to fill the gaps; this new region was not real and it didn’t contain any salient point to extract.
A. Simulated results: Fig.5 and Fig.6 showed the performance of the method (TREi and TREf) as a function of the amount of deformation with the local translation and bulge transform respectively.
B. Results on clinical data: In this section, the proposed algorithm was compared with real data. It was applied on a full patient treatment session and compared with the clinical algorithm. The latter was a local (around the prostate region) intensity-based rigid algorithm that used normalized cross correlation as similarity measure [13][14] and gave the shifts of the prostate center.
Figure 5 TRE value as a function of local translation.
Figure 7 Real deformation performance
Fig. 7 demonstrated 2D+T tracking results of the prostate center by the proposed and clinical algorithms for the same patient session. The graph showed the cumulative displacement of the prostate center as a function of the number of sweeps. The results have shown that the distance between the detected translations with both algorithms was small. In another hand, the amount of translation in both curves was increasing. The aim of this experiment was not to validate the proposed algorithm which has been already validated by using the artificial transforms, but it was to show that the proposed algorithm results were comparable to the clinical results on real deformations. IV.
[6]
M. Baumann, P. Mozer, V. Daanen, and J. Troccaz, “Transrectal ultrasound prostate biopsy tracking with efficient and accurate deformation estimation,” in IEEE International Symposium on Biomedical Imaging, Boston : United States (2009), 2009, pp. 1211– 1214.
[7]
A. Basarab, F. Vidal, R. Abbal, P. Delachartre, C. Vayssiere, P. Leguevaque, and D. Kouame, “Motion estimation in ultrasound imaging applied to the diagnostic of pelvic floor disorders.,” in Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Conference, 2011, pp. 8058–8061.
[8]
V. V. Karnik, A. Fenster, J. Bax, D. W. Cool, L. Gardi, I. Gyacskov, C. Romagnoli, and A. D. Ward, “Assessment of image registration accuracy in three-dimensional transrectal ultrasound guided prostate biopsy,” Med. Phys., vol. 37, no. 2, p. 802, 2010.
[9]
F. Jean, A. B. Albu, and C. Dumoulin, “Feature-based tracking of urethral motion in low-resolution trans-perineal ultrasound.,” Conf. Proc. IEEE Eng. Med. Biol. Soc., vol. 2011, pp. 6639–6642, Jan. 2011.
[10]
N. Hungr, M. Baumann, J.-A. Long, and J. Troccaz, “A 3-D ultrasound robotic prostate brachytherapy system with prostate motion tracking,” Robot. IEEE …, vol. 28, no. 6, p. 16, 2012.
[11]
R. S. Xu, O. Michailovich, and M. Salama, “Information tracking approach to segmentation of ultrasound imagery of prostate,” arXiv Prepr. arXiv0909.5458, vol. 57, no. 8, pp. 1–27, 2009.
[12]
M. Lachaine and T. Falco, “Intrafractional prostate motion management with the clarity autoscan system,” vol. 1, no. 1, pp. 72– 80, 2013.
[13]
V. Lagerburg, M. A. Moerland, J. J. Lagendijk, and J. J. Battermann, “Measurement of prostate rotation during insertion of needles for brachytherapy,” Radiother. Oncol., vol. 77.3, pp. 318–323, 2005.
[14]
Nichol AM, K. Brock, G. Lockwood, D. Moseley, T. Rosewall, P. Warde, C. Catton, and D. Jaffray, “A magnetic resonance imaging study of prostate deformation relative to implanted gold fiducial markers,” Int. J. Radiat. Oncol. Biol. Phys., vol. 67, no. 1, pp. 48– 56, 2007.
[15]
H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up robust features (SURF),” Comput. Vis. image Underst., vol. 110, no. 3, pp. 346–359, 2008.
[16]
P. Viola and M. Jones, “Rapid object detection using a boosted cascade of simple features,” Proc. 2001 IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognition. CVPR 2001, vol. 1, pp. 511–518, 2001.
[17]
P. Y. Simard, L. Bottou, P. Haffner, and Y. LeCun, “Boxlets: a fast convolution algorithm for signal processing and neural networks,” Adv. Neural Inf. Process. Syst., pp. 571–577, 1998.
[18]
D. G. Lowe, “Distinctive Image Features from Scale-Invariant Keypoints,” Int. J. Comput. Vis., vol. 60, no. 2, pp. 91–110, Nov. 2004.
[19]
G. Donato and S. Belongie, “Approximation methods for thin plate spline mappings and principal warps,” Transform. Datasets Linearbased Map Conflation Fram. Surv. L. Inf. Syst., vol. 94063, pp. 1– 13, 2002.
[20]
L. Fred Bookstein, “Principle warp: thin-plate spline and the decomposition of deformations,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 11, no. 6, pp. 1–19, 1989.
CONCLUSION
The aim of this study was to develop an algorithm that can track the deformation of the target region and improve the existing system precision. The proposed algorithm is fully automatic. It is based on SURF to detect and match interest points and TPS to warp the image. Optimization and validation have been done by approximating real deformations with simulated transforms: bulge transform and local translations. The error has been quantified by calculating TRE after registration, TREf. The region of interest was manually defined in the validation and optimization parts with artificial transforms. However, it will not be used in real tracking. The first results are promising, the proposed algorithm is a new elastic algorithm in the literature with TP-US probe that can track the deformations of the target region (prostate and prostatic bed). However, this algorithm suffers from some limitations that must be taken into consideration: without any optimization the calculation time was large, this algorithm was only 2D tracking. At last, this algorithm cannot track organs at the border of the image, since this organ can be lost after some sweeps. Future work will include the optimization of the code to reduce the running time of the algorithm to within an acceptable range for real time applications, as well as extend the algorithm for 3D+T ultrasound images. The algorithm should also be tested for the tracking of other anatomical parts. REFERENCES [1]
R. Siegel, J. Ma, Z. Zou, and A. Jemal, “Cancer statistics, 2014.,” CA. Cancer J. Clin., vol. 64, no. 1, pp. 9–29, 2014.
[2]
E. Bara, M. Duperray, and V. Delavigne, Les traitements du cancer de la prostate. Institut national du cancer, 2010.
[3]
McNeal and E. John, “The zonal anatomy of the prostate,” A Wiley Co., vol. 2, no. 1, pp. 35–49, 1981.
[4]
A. Sehrawat, K. Shimada, and Y. Rabin, “Generating prostate models by means of geometric deformation with application to computerized training of cryosurgery,” Int. J. Comput. Assist. Radiol. Surg., vol. 8, pp. 301–312, 2013.
[5]
M. Baumann, P. Mozer, V. Daanen, and J. Troccaz, “Towards 3D ultrasound image based soft tissue tracking: a transrectal ultrasound prostate image alignment system,” Med. Image Comput. Comput. Interv., pp. 26–33, 2007.
