LNCS 7761 - A Flexible Surgical Tool Localization Using a 3D ...

A Flexible Surgical Tool Localization Using a 3D Ultrasound Calibration System for Fetoscopic Tracheal Occlusion (FETO) Rong Xu1, Jun Ohya1, Bo Zhang2, Yoshinobu Sato3, and Masakatsu G. Fujie2 1

Graduate School of Global Information and Telecommunication Studies, Waseda University 2 Faculty of Science and Engineering, Waseda University, Tokyo 3 Graduate School of Medicine, Osaka University, Osaka, Japan

Abstract. In fetoscopic tracheal occlusion (FETO) surgery, the localization of a flexible tool has an important role in easing high demands for surgeons. In this paper, a point-based 3D ultrasound (US) calibration system is proposed for localizing the surgical tool, based on real-time 3D US images, an electromagnetic (EM) tracking device, and a novel cones’ phantom. A 3D US probe attached with an EM sensor is used to acquire the 3D US images of the surgical tool; meanwhile, an EM transmitter is fixed at a certain point in the environment as the origin of the world coordinate system. From these 3D US images, the skeleton of the surgical tool is extracted, and 15 labeled points along the surgical tool are then distinguished, whose coordinates in the world coordinate system are calculated by the proposed 3D US calibration system. The results demonstrate that our proposed approach achieves high localization accuracy, i.e. the average TRE of 1.99 0.49 and the maximum TRE of 2.61 0.49 , which satisfy the requirement of the FETO surgery: less than 3.85 mm. Keywords: Surgical tool localization, 3D ultrasound calibration, real-time 3D ultrasound, electromagnetic tracking device, cones’ phantom.

1

Introduction

Congenital diaphragmatic hernia (CDH) is a birth defect of the diaphragm, and has a prevalence of 1 in 2,000-5,000 newborns. In recent clinical practice, it has reported that severe CDH could be treated in uterus by a minimally invasive surgery (MIS) called fetoscopic tracheal occlusion (FETO) [1] to prevent pulmonary hypoplasia. To perform an FETO surgery, a fiber endoscope having a diameter of 1.3 mm within a cannula (Karl Storz) having a diameter of 3.3 mm is inserted into the amniotic cavity through the abdominal and uterine walls, towards the fetal mouth and fetal trachea, navigated by the ultrasound (US) images and fetoscopic images [2, 3]. However, this operation is so difficult and risky that high-level surgical skills are demanded for surgeons. To address this issue, a flexible wire-driven surgical tool [4, 5], whose posture can be transformed to fit the internal structure between the mouth and trachea, has recently been developed. Moreover, real-time 2D and 3D US images [6, 7] have been widely used to guide fetal MIS surgeries during operation due to K. Drechsler et al. (Eds.): CLIP 2012, LNCS 7761, pp. 17–24, 2013. © Springer-Verlag Berlin Heidelberg 2013

18

R. Xu et al.

their cost efficiency and im mpact-free characteristic compared with preoperative M MRI or CT, yet it is still difficullt for surgeons to localize and operate the flexible surggical tool accurately after being inserted into the fetal mouth and trachea. Therefore, foor a supporting navigation systeem of the FETO surgery, one mission is to localize the ffetal airways and trachea in 3D tracking t space by a 3D US calibration system and the rregistration between a reconsttructed 3D fetal model with the anatomy of fetal airw ways and trachea and the extracteed 3D fetal facial surface from 3D US images, because the anatomy of fetal airways and a trachea cannot be distinguished clearly by 3D US images. In this paper, an appro oach is proposed for another mission to accurately locaalize the flexible surgical tool deespite its different postures (bending patterns) capturedd by real-time 3D US images baased on a 3D US calibration system. In such a system, a 3D US probe attached with a 6-DOF 6 (degree of freedom) EM sensor, is exploited to acquire the images of the surg gical tool, and an EM transmitter is fixed at a certain pooint in the environment as the origin of the world coordinate system. From the 3D US images, the skeletons of th he surgical tool with different postures are extracted, followed by identifying 15 lab beled points along the surgical tool, whose coordinates in the world coordinate system m are calculated by the proposed 3D US calibration systeem. The rest of the paper is organized as follows: Section 2 details a real-time 3D US calibration system and the approach a for localizing the surgical tool; Section 3 preseents the results for validation; an nd finally, Section 4 provides the conclusions.

2

Materials and Methods M

2.1

Background and Overview O

In an FETO surgery, a feto oscope is inserted through the abdominal wall into the uuterus, and a detachable ballo oon is placed in fetal trachea for the tracheal occlusionn, as shown in Fig. 1 (a), to stim mulate lung growth by increasing the pressure of fetal chhest cavity. To insert the balloo on from the fetal mouth to trachea, a flexible and slennder surgical tool has been speccifically developed [4, 5]. However, the localization of the surgical tool in FETO surgeries remains a challenging problem. Here, we proposee an approach for accurate localiization of the surgical tool, as shown in Fig. 1 (b).

(a) FETO surgery

(b) The diagram of the proposed approaach

s and the diagram of the proposed approach Fig. 1. FETO surgery

A Flexible Surgical Tool Localization Using a 3D Ultrasound Calibration System

2.2

19

Flexible Surgical Tool

Zhang et al. [4, 5] developed a wire-driven surgical tool with an outer diameter of 2.4 mm and an inner diameter of 1.0 mm so that the tool can be inserted into the trocar with a diameter of 3 mm. As shown in Fig. 2 (a), the surgical tool consists of three units, each containing 10 ball-joint-shaped arthroses. Each unit also has two-DOFs, which enable the surgical tool to bend in different bending motions (patterns), as shown in Fig. 2 (b).

(a)

(b)

Fig. 2. The structure and bending patterns (motion) of the surgical tool

In order to localize the surgical tool, 15 points are labeled as No.1 to No.15 by selecting one out of every two arthroses (points) along the surgical tool, as shown in Fig. 3 (a). The deformations (bending patterns) of the surgical tool, as shown in Fig. 3 (b), are related to the driving motor’s rotation r (radian), and the motion data with the different bending patterns of the 15 labeled points are obtained by an optical camera (Here, we first consider a simple bending of the surgical tool in one direction).

(a) 15 labeled points

(b) The deformations of the 15 labeled points

Fig. 3. The deformations of the 15 labeled points in the surgical tool

2.3

3D Ultrasound Calibration

3D ultrasound calibration aims to determine the spatial transformation for mapping points from the 3D US image coordinate system to the world coordinate system in

20

R. Xu et al.

3D tracking space. There has been some work on real-time 3D US calibration [8, 9]. In this study, we propose a novel point-based phantom consisting of 12 resin cones for 3D US calibration. It is easy to construct and scan the phantom, and only simple experimental setup is required. 2.3.1 Experimental Devices and Setup In practice, a Philips iU22 system with a V6-2 US probe is employed for collecting 3D US data. The Ascension 3D guidance trakSTAR with four Model 90 6-DOF tracking sensors is used as the EM tracking device. The diameter of the 3D tracking sensor is just 0.9 mm, so it can be inserted into the surgical tool. The static accuracy of the position for the tracking sensor is 1.4 mm RMS and the orientation is 0.5° RMS. The phantom is composed of 12 transparent resin cones of six different types. The bottom diameters of all types are 30 mm, the heights of six types are 30 mm (C-I), 40 mm (C-II), 50 mm (C-III), 60 mm (C-IV), 70 mm (C-V) and 80 mm (C-VI), respectively. Fig. 4 (a) & (b) show two representative types. In the phantom, the cones of No.1 and No.2 are C-I, No.3 and No.4 are C-II, No.5 and No.6 are C-V, No.7 and No.8 are C-VI, No.9 and No.10 are C-III, and No.11 and No.12 are C-IV, as shown in Fig. 4 (c). Also, nine cones are used as a target model for validation purposes in Fig. 4 (d), where the cones from No.4 to No.6 are C-IV, the others are C-I.

(a) C-I

(b) C-IV

(c) cones’ phantom

(d) target model

Fig. 4. Point-based cones’ phantom and target model

Fig. 5. The experimental setup

The experimental setup is shown in Fig. 5, where 12 plastic cones arranged as shown in Fig. 4 (c) are settled on the bottom of water tank. The US probe with a 3D tracking sensor is fixed above the phantom by a plastic holder, and the bean surface contacts the water. The transmitter of the 3D EM device is settled near the water tank.


21

2.3.2 Calibration Matrix The position of each tip in the phantom in the world coordinate system is meeasured by a pen probe. The voxel location of each tip , is distinguished manuaally from the 3D US volumes. The T calibration matrix , and the reference trackeer’s transformation matrix , have the following relationship: ·

·

(1)

, , , 1 , x and y are the column and row indices of the pixel on the where extracted slice from X-Y pllane, and z is the index of the extracted slice along z-aaxis. The scale factors are integrrated with the calibration matrix , so the extra sccale factor in the computation iss not required. After multiplied by the inverse of the reeference transformation matrix from left, Eq. (1) can be written as : ·

/

·

(2)

where is a vector in the 3D US probe coordinate system. Considering n poosi/ p the following equation is obtained from Eq. (22): tions of the vertices in the phantom,

/

,

,···,

/

·

,

· 1

1

where

(3)

. Then, the calibration matrix is calculated by Horrn’s

1 1 method [10, 11] as follows. /

2.4

,

,···,

/

,

·

·

·

(4)

Proposed Approach h

To accurately localize the surgical s tool, we propose an approach to estimate the cooordinates of the 15 labeled po oints along the surgical tool in the world coordinate system. The procedure is as followss:

(a) 3D US data

(b) 3D ROI

(c) Skeleton

he procedure of the surgical tool localization Fig. 6. Th

(d) 15 labeled points

22

R. Xu et al.

(1) After a global threshold of the 3D US volume, the surgical tool is detected by selecting a region of interest (ROI), and the 3D distance map in the ROI is calculated; (2) The skeleton is extracted by a fast marching minimal path extraction in ITK [12], where the start, the end, and several way-points in the skeleton are required before extraction; (3) The coordinates of the 15 labeled points in 3D US space are measured in the extracted skeleton given knowing the coordinates of the start and end points, because the distance between each two labeled points is the same. (4) The coordinates of the 15 labeled points in the world coordinate system are estimated by the proposed 3D US calibration system.

3

Experimental Results

3.1

Validation for 3D US Calibration

The validation of the calibration is crucial to evaluate the performance of the estimated calibration matrix in reconstructing the 3D plane in the tracking space. There are two common methods for measuring calibration errors: (1) fiducial registration error (FRE) is the root mean square (RMS) distance between the localized position of each fiducial as transformed from image space to tracking space and the position of that corresponding fiducial localized in tracking space, which is used to evaluate how well the EM and US points fit together. (2) target registration error (TRE) is the same measurement as FRE, but the points for TRE were not used to estimate the calibration matrix, and thus provide a better indication of the accuracy of the calibration. 24 fiducial points in 3D US images and tracking information were recorded. 12 points are detected from one US volume by using the proposed phantom, and the other 12 points are detected from one more US volume by using the proposed phantom whose base is rotated by 180°. Additionally, 18 points in 3D US images are used for TRE measurement. 9 points are detected from one US volume by using the target model in Fig. 4 (d), and the other 9 points are detected from one more US volume by using the same target model with some level translation. To examine the convergence of the calibration error, 6 points (the vertices in the proposed phantom from No.1 to No.6) are used initially, increasing by one point a time up to 24 points (the point is added sequentially as marked number increases from small to large, where the rotated phantom are marked in the same order) for calibration and FRE validation. For each set of calibration points, 18 points based on the target model are used for TRE validation. Totally, we record the data of ten groups and each group is corresponding to 24 fiducial points and 18 target points. The mean and standard deviations of the ten groups are calculated and illustrated in Fig. 7 (a), with the FREs and TREs at each increased number of calibration points. Accordingly, we find out that: (1) the mean and standard deviations of FREs and TREs have a sudden decline as the number of calibration points n = 9 (first black vertical line in Fig. 7 (a)). (2) those values start to converge as n = 22 (second black vertical line in Fig. 7 (a)). As a result, the minimum number of the points used for our 3D US calibration system is 9. Moreover, more points (23 or more) will not bring much improvement


23

for calibration results. In that way, 24 fiducial points from two US volumes are used to estimate the calibration matrix for localizing the surgical tool, and the corresponding FRE of 1.60 0.54 and TRE of 1.97 0.74 are achieved in this case.

(a) Mean of FRE and TRE (calibration)

(b) Validation of the 15 labeled points

Fig. 7. The validation results of 3D US calibration and surgical tool localization

3.2

Validation for Localizing the Surgical Tool

We calculate the mean and standard deviation of average and maximum TREs for the 15 labeled points corresponding to each bending pattern in Fig. 3 (b), as shown in Fig. 7 (b). Herein, TREs denote the RMS distance errors between the coordinates of the 15 labeled points transformed from 3D US image space to 3D tracking space and the corresponding coordinates localized in 3D tracking space. For each bending pattern from 0 to 10 , the TREs of individual labeled points show slight differences because the standard deviations of average and maximum TREs (blue and red dashed lines in Fig. 7 (b)) are both smaller than 0.5 mm. The fluctuations on the means of average and maximum TREs (blue and red solid lines in Fig. 7 (b)) are also small. Besides, considering TREs of all points for all bending patterns, an average TRE of 1.99 0.49 mm and a maximum TRE of 2.61 0.49 mm are obtained, which satisfy the requirement of the FETO surgery: less than 3.85 mm. In FETO surgeries, the surgical tool is inserted into fetal airways until the entrance of fetal trachea, and the narrowest part of the passage is the fetal pharynx where the surgical tool is supposed to go through. In addition, FETO surgery is operated at about 26-29 weeks gestational age (GA) [3], and the average diameter of fetal pharynx is 7.7 mm at about 26-30 weeks [13]. Thus, 3.85 mm - half of the average diameter of fetal pharynx is the maximum distance error tolerable for this surgery. Therefore, the localization of the surgical tool in our experiments achieves sufficiently high accuracy for FETO surgeries.

4

Conclusion

In this study, we propose an approach to localize the surgical tool for FETO surgeries based on a real-time 3D ultrasound calibration. A new point-based phantom consisting of 12 resin cones is also presented. Validated in various experiments, the 3D US

24

R. Xu et al.

calibration achieves an FRE of 1.60 0.54 and a TRE of 1.97 0.74 by using 24 fiducial points from two US volumes. The localization of the surgical tool achieves an average TRE of 1.99 0.49 and a maximum TRE of 2.61 0.49 , which satisfy the requirement of the FETO surgery: less than 3.85 mm. The experimental results demonstrate that our proposed approach can achieve sufficiently high accuracy for localizing the surgical tool. To further improve the calibration consistency and accuracy, one potential of our future work is to design new algorithms to automatically identify the vertices in the cones’ phantom and extract the skeletons of the surgical tool. Furthermore, the effect of speed of sound would be taken into consideration in the future.

References 1. Harrison, M.R., et al.: A Randomized Trial of Fetal Endoscopic Tracheal Occlusion for Severe Fetal Congenital Diaphragmatic Hernia. New England Journal of Medicine 349(20), 1916–1924 (2003) 2. Deprest, J., Gratacos, E., Nicolaides, K.: Fetoscopic Tracheal Occlusion (FETO) for Severe Congenital Diaphragmatic Hernia: Evolution of a Technique and Preliminary Results. Ultrasound in Obstetrics & Gynecology 24(2), 121–126 (2004) 3. Jani, J., et al.: Severe Diaphragmatic Hernia Treated by Fetal Endoscopic Tracheal Occlusion. Ultrasound in Obstetrics & Gynecology 34(3), 304–310 (2009) 4. Zhang, B., Kobayashi, Y., Maeda, Y., Chiba, T., Fujie, M.G.: Development of 6-DOF Wire-Driven Robotic Manipulator for Minimally Invasive Fetal Surgery. In: IEEE International Conference on Robotics and Automation (ICRA), Shanghai, pp. 2892–2897 (2011) 5. Zhang, B., Maeda, Y., Chiba, T., Kobayashi, Y., Fujie, M.G.: Development of a Robotic Manipulator System for Congenital Diaphragmatic Hernia. In: IEEE International Conference on Systems, Man, and Cybernetics (SMC), Anchorage, AK, pp. 723–728 (2011) 6. Ruano, R., Okumura, M., Zugaib, M.: Four-Dimensional Ultrasonographic Guidance of Fetal Tracheal Occlusion in a Congenital Diaphragmatic Hernia. Journal of Ultrasound in Medicine 26(1), 105–109 (2007) 7. Tchirikov, M.: Successful Tracheal Occlusion Using Ultrathin Fetoscopic Equipment Combined with Real-Time Three-Dimensional Ultrasound. Eur. Surg. Res. 43(2), 204–207 (2009) 8. Lang, A., Parthasarathy, V., Jain, A.: Calibration of 3D Ultrasound to an Electromagnetic Tracking System. In: Proc. of SPIE, vol. 7968, p. 79680W (2011) 9. Huang, X., Gutiérrez, L.F., Stanton, D., Kim, P.C.W., Jain, A.: Image Registration Based 3D TEE-EM Calibration. ISBI 2010, 1209–1212 (2010) 10. Horn, B.K.P.: Closed-Form Solution of Absolute Orientation Using Unit Quaternions. Journal of the Optical Society of America A 4(4), 629–642 (1987) 11. Zhang, H., Banovac, F., White, A., Cleary, K.: Freehand 3D Ultrasound Calibration Using an Electromagnetically Tracked Needle. In: Proceedings of SPIE, vol. 6141, p. 61412M (2006) 12. Mueller, D.: Fast Marching Minimal Path Extraction in Itk. The Insight Journal, 1–9 (2008) 13. Tez, S., Köktener, A., Aksoy, F.G., Turhan, N.Ö., Dilmen, G.: Ultrasound Evaluation of the Normal Fetal Pharynx. Early Human Development 81(7), 629–633 (2005)