Development of a Wireless Hybrid Navigation System for ...

Medicine Meets Virtual Reality 18 J.D. Westwood et al. (Eds.) IOS Press, 2011 © 2011 The authors. All rights reserved. doi:10.3233/978-1-60750-706-2-479

479

Development of a Wireless Hybrid Navigation System for Laparoscopic Surgery Hongliang REN a , Denis RANK b , Martin MERDES b , Jan STALLKAMP and Peter KAZANZIDES a,1

b

a

Dept. of Computer Science, Johns Hopkins University, Baltimore, MD USA {hlren,pkaz}@jhu.edu b Fraunhofer Institute for Manufacturing Engineering and Automation, Stuttgart, Germany {rank,martin,stallkamp}@ipa.fhg.de Abstract. Navigation devices have been essential components for ImageGuided Surgeries (IGS) including laparoscopic surgery. We propose a wireless hybrid navigation device that integrates miniature inertial sensors and electromagnetic sensing units, for tracking instruments both inside and outside the human-body. The proposed system is free of the constraints of line-of-sight or entangling sensor wires. The main functional (sensor) part of the hybrid tracker is only about 15 mm by 15 mm. We identify the sensor models and develop sensor fusion algorithms for the proposed system to get optimal estimation of position and orientation (pose). The proof-of-concept experimental results show that the proposed hardware and software system can meet the defined tracking requirements, in terms of tracking accuracy, latency and robustness to environmental interferences. Keywords. Image guided surgery, Surgical navigation, Electromagnetic tracking, Inertial measurement unit, Sensor fusion

Introduction Real-time tracking of surgical instruments inside the human body poses unique challenges in developing tracking devices for minimally invasive surgeries. Optical tracking (OPT), the gold standard for surgical navigation, is bulky and blind when its line-of-sight (LOS) between the cameras and the markers is occluded [7]. Electromagnetic Tracking (EMT) [4] is feasible for laparoscopic surgery but notorious for its susceptibility to surrounding metallic or conductive surgical tools [6] and its reliance on a wired connection to the markers (coils). In addition, both of them have a limited working volume and OPT further has the restriction of angle of view relative to the optical camera. Ultrasound based [9,5] navigation or 1 Corresponding Author: Peter Kazanzides, Department of Computer Science, Johns Hopkins University.

480

H. Ren et al. / Development of a Wireless Hybrid Navigation System for Laparoscopic Surgery

mechanical tracking [1] usually have handling inconveniences. Some prior work, such as [2][7], combined information from an optical tracker and EM tracker, which makes the operating room even more crowded with two bulky tracking systems, and can still suffer from the constraint of line-of-sight.

1. Objective Our objective is to develop a miniature tracking system that is easy-to-use, free of line-of-sight, angle-of-view, or cabling constraints, and with a reasonable working volume, without affecting surgical workflow. The desired technical specification is to achieve 6DOF target tracking accuracy of about 1 mm in position and 1 degree in orientation, with maximum latency of 100 milliseconds, minimum update frequency of 30 Hz, and with robustness against interference due to metallic objects, electrocautery, etc. Towards this end, we combine inexpensive miniature MEMS inertial sensors to compensate the distortions of the EM tracker, and to improve the dynamic behavior of the tracking system.

2. Material We employ a self-contained Inertial Measurement Unit (IMU), including accelerometer, gyroscope and magnetometer, to provide highly dynamic measurements with respect to global coordinates. For example, the accelerometer and magnetometer together can provide roll, pitch and heading measurements. The miniature electromagnetic sensor is used to provide an external reference to compensate the drift of the IMU. The prototype hardware is shown in Figure 1. It consists of two electromagnetic tracking systems (the commercial Aurora system from NDI [3] and a custom EMT system) and a sensor PCB which has integrated inertial sensors and electronics for signal processing. The commercial Aurora EMT system is included for the purpose of both validation and comparison. The rationale for developing a custom EM system is mainly due to the need for time synchronization and wireless operation. The custom EMT consists of a field generator and up to three receiving coils on the sensor PCB, which is collocated with the inertial sensors. Communication between the sensor PCB and a PC is via Bluetooth (for wireless operation) or USB (for debugging or firmware updates). Note that in this paper, the sensor fusion experiments used the commercial NDI Aurora system, as we are still working on the custom EMT to get comparable performance. The three axis accelerometer, used for detecting movements in the x, y and z directions, is the ST331DLH from STMicroelectronics and its measurement range is set to ±2g with an internal cutoff frequency of 780Hz. Two gyroscope sensors, the two-axis IDG300 and the single-axis ISZ300 from InvenSense, are used to measure the 3-axis angular rate. A three-axis AMR magnetometer, the Honeywell HMC1043 with sensitivity of each axis about 0.3mV/μT, serves as an electronic compass.

H. Ren et al. / Development of a Wireless Hybrid Navigation System for Laparoscopic Surgery

481

Figure 1. The hybrid tracking system consists of a hybrid tracker inside the handle of an endoscope (left), and an external field generator (right). The hybrid tracker is composed of inertial sensors, electromagnetic coils and other supporting electronic components. The field generator is a coil array with 4x4x3 transmitters.

3. Methods Because the raw measurements are from two different sensor-coordinate systems (EMT and IMU), it is necessary to register these coordinate systems before performing sensor fusion. The coordinate registration includes a body-frame registration solved by an AX=XB formulation [8], and base-frame registration solved by a paired-orientation formulation [11]. Figure 2 defines the coordinate systems and illustrates the two unknown transformations, X and Y. Fcoil Ffg X Field Generator

Fimu Y

Fnav : Navigation frame (North East Down)

Figure 2. Definition of the coordinate systems. X and Y are the two unknown transformations, corresponding to body-frame and base-frame transformations, respectively.

The two streams of registered measurements are subsequently fed to a sensor fusion module, which consists of an orientation estimator (rot) and a position estimator (pos), both based on Kalman filters, as shown in the block diagram of Figure 3. For the orientation estimation in Figure 3, the measurements are obtained from the IMU and EMT, as both subsystems can provide orientation measurements. We are weighting them based on the acceleration of the tracker: when the acceleration is small, the measurements from the IMU are more accurate; otherwise, the measurements from the EMT are more accurate.

482

H. Ren et al. / Development of a Wireless Hybrid Navigation System for Laparoscopic Surgery fg

x

fg

q

Coil(s)

nav

Accel (x3) Magnetic (x3)

Gyro (x3)

nav

nav

x  g

q

q

Kalman filter (pos) fgR

~ x

fg

~ x

fg

q~

fg

~ q

nav

Attitude Meas. fgR nav

fgR

fg

fg

qˆ

Kalman filter (rot)

nav

Figure 3. Block diagram of the sensor fusion algorithm; x is position measurement, q is orientation measurement.

The orientation dynamics model is derived in time-derivative quaternion formulation as: q˙ =

1 [ω×] · q, 2

(1)

where q is the quaternion representation of the orientation, and [ω×] is a skew symmetric matrix of the angular velocity, ω, acquired from the gyroscope. We also include the gyroscope bias in the system dynamics, by assuming it is a random walk process. For position estimation, we are using the external position measurements from EMT as the reference and the dynamic model is derived from the kinematic relationship between position, velocity and acceleration, given by, x ¨nav = R · a − R · ω × x˙ nav − g,

(2)

where R is the rotation matrix from IMU frame to navigation frame, xnav , x˙ nav and x ¨nav are the position, velocity and acceleration vectors in the navigation frame, a is the acceleration measured by the accelerometer, ω is the angular velocity measured by the gyroscope, and g is the gravity vector.

4. Results We conducted a series of experiments to validate the proposed hybrid tracking system (HYB). First, we compared the orientation estimate between the hybrid tracker and the commercial Aurora EM tracker. Note that we are using the orientation estimates from the commercial NDI Polaris optical tracker (OPT) as the benchmark. Figure 4 shows the difference in the orientation estimate with respect to the optical tracker (i.e., HYB-OPT vs. EMT-OPT). The overall root-meansquare (RMS) tracking errors of the HYB for the 2 runs were 0.9 degrees, 0.8 degrees, 1.0 degrees, for roll, pitch and yaw, respectively. For EMT, the overall

H. Ren et al. / Development of a Wireless Hybrid Navigation System for Laparoscopic Surgery

483

Estimate difference wrt OPT: HYB vs. EMT 4 HYB EMT

3

2

degrees

1

0

-1

-2

-3

-4

roll

pitch

yaw

roll

pitch

yaw

runs

 Figure 4. Measured orientation errors for proposed hybrid tracker and EMT, using optical tracking as ground truth.

RMS tracking errors were 1.9 degrees, 2.1 degrees, 2.1 degrees, for roll, pitch and yaw, respectively. We also compared the tracking performance in an earlier experiment [10] between HYB and EMT when the same metallic tool was moved around the tracker. The hybrid tracking method demonstrated its resistance to the environmental interference. Thus, the orientation estimation shows superior performance in terms of accuracy and robustness to metallic disturbance, compared to just the use of EM tracking. A trajectory tracking experiment is shown in Figure 5. The dynamic tracking performance of the hybrid tracker is better than just using the external EMT reference. Note that the hybrid system can obtain position measurements even when the EMT signals are missing for a short duration. In order to present the tracking information graphically, we implemented an OpenIGTLink [12] interface to enable rapid integration with IGT platforms such as 3D Slicer, as shown in Figure 6.

5. Conclusions & Outlook The proposed electromagnetic aided inertial navigation system demonstrated improved tracking performance in terms of tracking accuracy, data update rate, and tracking robustness. The integration of inertial sensing with an external reference, such as electromagnetic tracking, provides a promising solution for tracking surgical instruments during laparoscopic surgery. The external reference tracking system can provide

484

H. Ren et al. / Development of a Wireless Hybrid Navigation System for Laparoscopic Surgery

HYB EMT

160 140 120

Y(mm)

100 80 60 40 20 0 −350

−300

−250 X(mm)

−200

Figure 5. Trajectory of position estimation from HYB (blue line) and EMT (red line); HYB demonstrated better dynamic tracking performance.

Figure 6. Graphical representation of the tracking results in 3D Slicer through OpenIGTLink interface

stable correction for inertial sensor drifts and, in turn, the inertial sensor can provide better dynamic tracking results. A important future work is to validate the custom EMT system, including the calibration, localization and integration of the EMT for sensor fusion.

Acknowledgment This project is a joint development between The Johns Hopkins University (JHU) and the Fraunhofer Institute for Manufacturing Engineering and Automation (IPA) and is supported by internal funds from both institutions. The authors gratefully acknowledge the contributions of our colleagues Russell Taylor, Elliot McVeigh, Iulian Iordachita, and Anton Deguet, at JHU.

H. Ren et al. / Development of a Wireless Hybrid Navigation System for Laparoscopic Surgery

485

References [1]

[2]

[3] [4] [5]

[6]

[7]

[8] [9]

[10]

[11]

[12]

J. Bax, D. Cool, L. Gardi, K. Knight, D. Smith, J. Montreuil, S. Sherebrin, C. Romagnoli, and A. Fenster. Mechanically assisted 3D ultrasound guided prostate biopsy system. Medical Physics, 35(12):5397–5410, 2008. W. Birkfellner, F. Watzinger, F. Wanschitz, R. Ewers, and H. Bergmann. Calibration of tracking systems in a surgical environment. IEEE Trans. on Medical Imaging, 17(5):737– 742, Oct. 1998. http://www.ndigital.com/medical. V. Kindratenko. A survey of electromagnetic position tracker calibration techniques. Virtual Reality, 5(3):169–182, Sept. 2000. H.-H. Lin, C.-C. Tsai, and J.-C. Hsu. Ultrasonic localization and pose tracking of an autonomous mobile robot via fuzzy adaptive extended information filtering. IEEE Trans. on Instrumentation and Measurement, 57(9):2024–2034, Sept. 2008. C. Nafis, V. Jensen, and R. von Jako. Method for evaluating compatibility of commercial electromagnetic (EM) microsensor tracking systems with surgical and imaging tables. In SPIE Medical Imaging 2008: Visualization, Image-guided Procedures, and Modeling, volume 6918, pages 691820,1–15. SPIE, 2008. M. Nakamoto, Y. Sato, M. Miyamoto, Y. Nakamjima, K. Konishi, M. Shimada, M. Hashizume, and S. Tamura. 3D ultrasound system using a magneto-optic hybrid tracker for augmented reality visualization in laparoscopic liver surgery. In Medical Image Computing and Computer-Assisted Intervention (MICCAI), pages 148–155, London, UK, 2002. F. Park and B. Martin. Robot sensor calibration: solving AX=XB on the Euclidean group. IEEE Trans. on Robotics and Automation, 10(5):717–721, Oct 1994. J. F. Quinlan, H. Mullett, R. Stapleton, D. FitzPatrick, and D. McCormack. The use of the Zebris motion analysis system for measuring cervical spine movements in vivo. Proceedings of the Institution of Mechanical Engineers – Part H – Journal of Engineering in Medicine, 220(8):889 – 896, 2006. H. Ren and P. Kazanzides. Hybrid attitude estimation for laparoscopic surgical tools: A preliminary study. EMBC 2009. IEEE International Conference on EMBS, pages 5583– 5586, 2009. H. Ren and P. Kazanzides. A paired-orientation alignment problem in a hybrid tracking system for computer assisted surgery. Journal of Intelligent and Robotic Systems, accepted, 2010. J. Tokuda, G. S. Fischer, X. Papademetris, Z. Yaniv, L. Ibanez, P. Cheng, H. Liu, J. Blevins, J. Arata, A. J. Golby, T. Kapur, S. Pieper, E. C. Burdette, G. Fichtinger, C. M. Tempany, and N. Hata. OpenIGTLink: an open network protocol for image-guided therapy environment. The International Journal of Medical Robotics and Computer Assisted Surgery, 5(4):423–434, 2009.