Available online at www.sciencedirect.com
ScienceDirect Procedia Computer Science 76 (2015) 15 – 20
2015 IEEE International Symposium on Robotics and Intelligent Sensors (IRIS 2015)
Modeling of Interpolation Methods for Robot Assisted and Freehand Ultrasound Imaging System Vedpal Singha, I. Elamvazuthia*, Varun Jeotia, J Georgeb, A.K. Swainc, Dileep Kumara a
Centre for Intelligent Signal and Imaging Research (CISIR), Department of Electrical and Electronic Engineering Universiti Teknologi PETRONAS, 32610, Bandar Seri Iskandar, Perak Darul Ridzuan, Malaysia b Research Imaging Centre, University of Malaya, Kuala Lumpur, Malaysia c University of Auckland, New Zealand
Abstract The 2D ultrasound is one of the popular imaging modality amongst all existing medical imaging methods such as X-Ray, MRI, CT, PET and SPECT. However, 3D ultrasound is also increasingly being used in clinical and image guidance applications such as echo cardiology and obstetrics by robot assisted probe and freehand probe, but 3D ultrasound imaging has some limitations such as lack of suitable interpolation methods that leads to wrong interpretation. To overcome this problem, this study presents a modelling approach for interpolation methods to get the better 3D results for enhanced diagnosis. For this modelling, this research mainly uses four (4) interpolation methods such as conventional Distance Weighted (DW), Improved Distance Weighted (IDW), Conventional Olympic (C-Oly) and Improved Olympic (I-Oly) interpolation methods. In order to evaluate the performance, this study used ten (10) test cases, which are testified by each interpolation method. Each method calculates a value, which would be used to fill-out the empty space during interpolation. Therefore, on the basis of obtained results, it is reported that the I-Oly method is producing more optimal results due to the lowest error rate (2.48%) and performance improvement of 26.23% as compared to other existing methods. The outcome of this research could be imparted to post-graduate level for students undertaking sensor and systems either in the robot assisted or freehand ultrasound imaging field. © by Elsevier B.V.by This is an open © 2015 2015Published The Authors. Published Elsevier B.V.access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of organizing committee of the 2015 IEEE International Symposium on Robotics and Intelligent Peer-review under responsibility of organizing committee of the 2015 IEEE International Symposium on Robotics and Intelligent Sensors (IRIS 2015). Sensors (IRIS 2015) Keywords: Interpolation methods; hole filling; 3D ultrasound reconstruction; conventional distance weighted; improved distance weighted interpolation; conventional Olympic; improved Olympic hole filling method
*Corresponding Author: Tel: +605-3687882; Fax: +605-3657443 E-mail address:
[email protected]
1877-0509 © 2015 Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of organizing committee of the 2015 IEEE International Symposium on Robotics and Intelligent Sensors (IRIS 2015) doi:10.1016/j.procs.2015.12.269
16
Vedpal Singh et al. / Procedia Computer Science 76 (2015) 15 – 20
1. Introduction The 2D ultrasound is a popular non-invasive imaging modality with real time capability. It is helpful in the diagnosis of obstetrics, neurosurgery and musculoskeletal (MSK) disorders for last two decades1. Generally, the 2D ultrasound systems are under use around the world, but it has some problems such as low contrast regions and homogeneous textures that lead to wrong interpretation and unsatisfied image quality. However, some researchers are tried to recover these problems through 3D reconstruction of ultrasound images 2. The 3D reconstruction used a specific arrangement of 2D ultrasound images according to their spatial information. The spatial information of each image should be consistent for accurate 3D reconstruction. Inconsistently adjusted ultrasound images lead to inaccurate 3D reconstruction due to geometrical errors3-5. Generally, ultrasound system uses two approaches for data collection, 1) robot assisted probe and, 2) freehand probe. Robot assisted probe helps in the acquisition of approximate accurately oriented data, because it follows the predefined instructions like angle of rotation and speed of movement. However, freehand probe most of time provides the irregular images that are the more challenging rather than robot assisted probe data 6. It should be noted that the acquired data from the both approaches are not perfectly accurate, which are main causes of inaccurate 3D reconstruction2,4. To overcome these problems, this research attempts to models the suitable interpolation methods for enhanced 3D reconstruction to improve the diagnosis. Although, various research studies reported about the numerous interpolation methods for 3D ultrasound reconstruction5,7,8, however, they show limitations such as unconvinced image quality. In order to overcome these problems, there is a need to use the suitable interpolation method for 3D reconstruction, which is modelled in this study. The rest of the paper is organized as follows. Section 2 provides the information about 3D ultrasound reconstruction process flow and used interpolation methods. Section 3 presents the results and corresponding discussion in detail. The section 4 concludes this paper. 2. Methods The 3D ultrasound reconstruction is not a wide spread technology still, but now it is getting more attention due to its additional beneficial features such as enhanced geometrical information, better visualization and accurate volume measurement. In order to reconstruct a 3D image, a model for freehand 3D Reconstruction of ultrasound images is presented in Fig. 1.
Fig. 1. Process flow for 3D ultrasound reconstruction.
According to Fig. 1, initially, data is acquired in video format (e.g., Avi format) and acquired video is converted into ultrasound images. For better visualization acquired images are filtered, which are used in segmentation to extract the desired region4,9,10. The segmented images are further used in 3D reconstruction. However, during 3D reconstruction, image interpolation is an unavoidable step, which used some methods to enhance the results such as DW, IDW, C-Oly and I-Oly methods9-12. However, the challenge of interpolation methods is to reduce the frame gap without loss of valuable information8,13. In current research, the four popular interpolation methods are DW, IDW, C-Oly and I-Oly are being used for interpolation that are illustrated in Fig. 214.
17
Vedpal Singh et al. / Procedia Computer Science 76 (2015) 15 – 20
Fig. 2. Interpolation method categorization14.
The four interpolation methods that are presented in Fig. 2 are demonstrated in below sections for better understanding. 2.1 The Conventional Distance Weighted (DW) Hole-filling method The DW hole filling method for ultrasound frames is represented by Fig. 3 that represents two intersecting frames X1, X2. The frames X1, X2 has slope T and distance d of frames from a certain point p.
Fig. 3. Distance Weighted Interpolation method 13.
In Fig. 3, a point p in search space is oriented onto two nearest iso-planes and resultant projection points are oriented onto straight edge of quadrilaterals respectively. 2 1 d ij d ij 1 2 (1) I ( X ij ) I ij I ij , (i, j 1,2) 1 2 1 2 d ij d ij d ij d ij I (Xi )
I ( p)
di 2 di1 I ( X i1 ) I ( X i 2 ), (i, j 1,2) di di di1 di 2 d2 d1 I ( X1) I(X2) d1 d 2 d1 d 2
(2) (3)
I(Xij), I(Xi) and I(p) in equations 1, 2 and 3 represents image intensities at edge projections, planner projections and interpolation points respectively13,14. 2.2 The Improved Distance Weighted (IDW) Hole-filling method This method employs DW method with specific threshold approach of defined range width of vacant neighbors by mean of range width of all values of voxels in 3D volume. The frame gaps are calculated by the evaluated empty voxel. Algorithmic approach for IDW method is explained as shown in Fig. 415.
Fig. 4. IDW interpolation method process flow.
18
Vedpal Singh et al. / Procedia Computer Science 76 (2015) 15 – 20
N is distance threshold and f ( x, y, z ) represents an empty voxel. Dis indicates the distance, cnt presents the quantity and v is the voxel value. Each current voxel ws is defined by f ( xx, yy, zz ) , where x, y, z are the 3D coordinate axes values respectively and this is also same for coordinate values xx, yy, zz . The algorithm is graphically represented by Fig. 415. 2.3 The Conventional Olympic (C-Oly) Hole-filling method The Conventional Olympic Hole-filling method uses mean value to fill-out the empty space. However, above defined methods could not provide suitable value which may not be laying and filling in accurate positions 16,17. To overcome this problem, the Conventional Olympic Hole-filling method can be a good option which is represented in Equation 4. (4) C Oly avg[nex {sort{I }}] In this equation, I is a set of input neighborhood pixels, sort means a sorting process of all input pixels values, nex is n % extraction from input set and in last take the average (avg) of remaining values. 2.4 The Improved Olympic (I-Oly) Hole-filling method The Improved Olympic Hole-filling method used thresholding and range width average of sorted remaining neighbor in all existing empty voxels Rn with factor k, which used in classification. This classification provides some values which are used in thresholding range. The empty voxel range width is Rn which examines a difference between values from maximum to minimum and compared to obtained threshold value 2,17. To determine the empty voxel by the adjustment of average and range width values. § Rn · , Where p=p1, R d k .R xe xn ¨ n n ¨ p ¸ ¸ © ¹ p=p2, Rn ! k .Rn (5) Where xe indicates estimated empty voxel, xn is average of remaining sorted neighboring voxels and Rn represents sorted remaining neighbor range width and p provides the resultant value which is determined by threshold value of k .Rn . p1 p1 should be p if Rn d k .Rn , and p2 is value of p if Rn ! k .Rn 17. 3. Results and discussion To evaluate the performance, this research tested each interpolation method on 10 test sets, which are prepared based on the pixel values of images. Therefore, Table 1 presents the resultant value of each method for every test case that to be filled in empty space of a 3D image during interpolation. Table 1. Analyses the performance of interpolation methods. Test
Distance Weighted
1 2 3 4 5 6 7 8 9 10
3.5 17.17 4.5 5.5 5.83 5.16 5.83 8.5 7.66 10.5
Improved Distance Weighted 13 14 10 8 10 11 10 9 10 9
Conventional Olympic
Improved Olympic
14 18.18 4.14 4.66 4.57 5.71 6.57 4.95 7.14 7.42
6.64 12.29 3 3.46 3.32 4.46 5.32 3.75 5.99 6.27
Vedpal Singh et al. / Procedia Computer Science 76 (2015) 15 – 20
In test case 4, the value of DW is 5.5, 8 for IDW, 4.66 for C-Oly and 3 for I-Oly. In this case, 3 would be significantly nearest to the four neighboring values of 2, 4, 5, and 6. Thus, I-Oly method is found as an optimal interpolation method. Thus, from Table 1, it can be seen that I-Oly method provides the optimal results for every case compared to other existing methods. In addition, more validation is illustrated in Fig. 5-7 using obtained interpolated value for empty space. Fig. 5 provides a comparison between DW and IDW methods with respect to their nearest pixel value for empty space where IDW provided more optimal value than DW in eight test cases, excluding 2 cases. But in Fig. 6, IDW method is not working efficiently; it could not provide optimal value as compared with C-Oly method. Thereafter, in Fig. 7, I-Oly shows better performance than C-Oly.
Fig. 5. Performance analysis DW vs IDW methods.
Fig. 6. Performance analysis IDW vs C-Oly methods.
Fig. 7. Performance analysis C-Oly vs I-Oly methods.
Fig. 8-10 represents the evaluation of each method. Fig. 8 depicts the performance improvement of IDW to DW method for each test case that concludes in favor of IDW method as compared to DW method.
Fig. 8. Performance analysis DW vs IDW methods.
Fig. 9. Performance analysis IDW vs COly methods.
Fig. 10. Performance analysis C-Oly vs IOly methods.
Fig. 9 has shown the healthy performance of C-Oly method with respect to the IDW method. The performance of I-Oly method is better than C-Oly method as illustrated in Fig. 10. Fig. 10 depicted the higher value of the C-Oly method rather than I-Oly method. Thus, the graphical representation of each figure proved that the I-Oly method is optimal method as compared to other existing methods. Moreover, Table 2 demonstrates the improvement and error rate of each method to determine the performance. Table 2. Performance evaluation of various interpolation methods. Technique Distance Weighted Improved Distance Weighted Conventional Olympic Improved Olympic
Improvement (%)
Mean Absolute Error (MAE) (%)
(base value)
0.98
69.90
3.80
-29.06 *
2.70
-26.23*
2.48
* negative (-) indicates nearest and optimal value to be filled in empty space
19
20
Vedpal Singh et al. / Procedia Computer Science 76 (2015) 15 – 20
The Table 2 presented an analysis based on performance improvement and Mean Absolute Error (MAE) of the four (4) described methods. The IDW reported 69.90% improvement over DW with 3.8% error rate. Similarly, the C-Oly noticed 29.06% better performance than IDW with low error rate of 2.7%. Hence, the I-Oly is proven itself as a better method with 26.23% improvement with lowest error rate of 2.48% than existing methods. The I-Oly method is highly efficient with lower error rate that would helpful in the reconstruction of better quality 3D images due to the presence of lowest geometrical errors. 4. Conclusion and future work The 3D ultrasound is emerging as an important imaging modality used in numerous medical applications. This study presented a modelling of interpolation methods to get the better 3D results, which would be more helpful in enhanced diagnosis of injuries and abnormalities with efficient quantitative analysis. Therefore, on the basis of obtained results, it is concluded that the I-Oly method is an optimal method for ultrasound images due to its lowest error rate and high performance improvement as compared to existing methods. In future, this study would be applicable in the modelling of interpolation methods for various applications after some amendments. This study paves the way for it to be taught at post-graduate level for students undertaking sensor and systems either in the robot assisted or freehand ultrasound imaging field. Acknowledgement The authors would like to thank UTP for their assistance and Ministry of Education (MOE) for sponsoring the project under grant entitled ‘Formulation of Mathematical Model for 3-D Reconstruction of Ultrasound Images of MSK Disorders’ (Grant no. 0153AB-I55). References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17.
P. Coupé, P. Hellier, X. Morandi, and C. Barillot, "Probe trajectory interpolation for 3d reconstruction of freehand ultrasound," Medical Image Analysis, vol. 11, pp. 604-615, 2007. R. Rohling, A. Gee, and L. Berman, "A comparison of freehand three-dimensional ultrasound reconstruction techniques," Medical image analysis, vol. 3, pp. 339-359, 1999. I. Elamvazuthi, M. L. B. M. Zain, and K. Begam, "Despeckling of ultrasound images of bone fracture using multiple filtering algorithms," Mathematical and Computer Modelling, vol. 57, pp. 152-168, 2013. V. Singh, I. Elamvazuthi, V. Jeoti, and J. George, "Automatic Ultrasound Image Segmentation Framework Based on Darwinian Particle Swarm Optimization," in Proceedings of the 18th Asia Pacific Symposium on Intelligent and Evolutionary Systems, Volume 1, 2015, pp. 225-236. V. Singh, I. Elamvazuthi, V. Jeoti, and J. George, "3D reconstruction of ATFL ligament using ultrasound images," in Intelligent and Advanced Systems (ICIAS), 2014 5th International Conference on, 2014, pp. 1-5. Z. Wei, G. Wan, L. Gardi, G. Mills, D. Downey, and A. Fenster, "Robot-assisted 3D-TRUS guided prostate brachytherapy: system integration and validation," Medical physics, vol. 31, pp. 539-548, 2004. I. L. Herlin and N. Ayache, "Features extraction and analysis methods for sequences of ultrasound images," in Computer Vision— ECCV'92, 1992, pp. 43-57. H. Ludvigsen, "Real-time GPU-based 3D ultrasound reconstruction and visualization," 2010. R. J. Housden, A. H. Gee, G. M. Treece, and R. W. Prager, "Sensorless reconstruction of freehand 3d ultrasound data," in Medical Image Computing and Computer-Assisted Intervention–MICCAI 2006, ed: Springer, 2006, pp. 356-363. T. Qiu, T. Wen, W. Qin, J. Gu, and L. Wang, "Freehand 3D ultrasound reconstruction for image-guided surgery," in Bioelectronics and Bioinformatics (ISBB), 2011 International Symposium on, 2011, pp. 147-150. U. Scheipers, S. Koptenko, R. Remlinger, T. Falco, and M. Lachaine, "3-d ultrasound volume reconstruction using the direct frame interpolation method," Ultrasonics, Ferroelectrics, and Frequency Control, IEEE Transactions on, vol. 57, pp. 2460-2470, 2010. R. W. Prager, A. Gee, and L. Berman, "Stradx: real-time acquisition and visualization of freehand three-dimensional ultrasound," Medical Image Analysis, vol. 3, pp. 129-140, 1999. S. Ji, D. W. Roberts, A. Hartov, and K. D. Paulsen, "Real-time interpolation for true 3-dimensional ultrasound image volumes," Journal of Ultrasound in Medicine, vol. 30, pp. 243-252, 2011. S. Meairs, J. Beyer, and M. Hennerici, "Reconstruction and visualization of irregularly sampled three-and four-dimensional ultrasound data for cerebrovascular applications," Ultrasound in medicine & biology, vol. 26, pp. 263-272, 2000. Y. Zhang, R. Rohling, and D. K. Pai, Direct surface extraction from 3D freehand ultrasound images: IEEE, 2002. D. G. Gobbi and T. M. Peters, "Interactive intra-operative 3D ultrasound reconstruction and visualization," in Medical Image Computing and Computer-Assisted Intervention—MICCAI 2002, ed: Springer, 2002, pp. 156-163. D. Dewi, T. Mengko, I. Purnama, A. Veldhuizen, and M. Wilkinson, "An improved olympic hole-filling method for ultrasound volume reconstruction of human spine," Emerging Communication Technologies for E-Health and Medicine, p. 259, 2012.
