Surg Endosc (2011) 25:681–690 DOI 10.1007/s00464-010-1243-3
Surgery in space: the future of robotic telesurgery Tama´s Haidegger • Jo´zsef Sa´ndor • Zolta´n Benyo´
Received: 25 February 2010 / Accepted: 1 July 2010 / Published online: 22 July 2010 Ó Springer Science+Business Media, LLC 2010
Abstract Background The origins of telemedicine date back to the early 1970s, and combined with the concept of minimally invasive surgery, the idea of surgical robotics was born in the late 1980s based on the principle of providing active telepresence to surgeons. Many research projects were initiated, creating a set of instruments for endoscopic telesurgery, while visionary surgeons built networks for telesurgical patient care, demonstrated transcontinental surgery, and performed procedures in weightlessness. Long-distance telesurgery became the testbed for new medical support concepts of space missions. Methods This article provides a complete review of the milestone experiments in the field, and describes a feasible concept to extend telemedicine beyond Earth orbit. With a possible foundation of an extraplanetary human outpost either on the Moon or on Mars, space agencies are carefully looking for effective and affordable solutions for lifesupport and medical care. The major challenges of surgery in weightlessness are also discussed. Results Teleoperated surgical robots have the potential to shape the future of extreme health care both in space and on Earth. Besides the apparent advantages, there are some T. Haidegger Z. Benyo´ Department of Control Engineering and Information Technology, Budapest University of Technology and Economics, Budapest, Hungary J. Sa´ndor Department of Surgical Education, Semmelweis University, Budapest, Hungary T. Haidegger (&) BME–IIT, Magyar tudosok krt 2, Budapest 1117, Hungary e-mail:
[email protected]
serious challenges, primarily the difficulty of latency with teleoperation over long distances. Advanced virtualization and augmented-reality techniques should help human operators to adapt better to the special conditions. To meet safety standards and requirements in space, a three-layered architecture is recommended to provide the highest quality of telepresence technically achievable for provisional exploration missions. Conclusion Surgical robotic technology is an emerging interdisciplinary field, with a great potential impact on many areas of health care, including telemedicine. With the proposed three-layered concept—relying only on currently available technology—effective support of long-distance telesurgery and human space missions are both feasible. Keywords Robotic surgery Teleoperation Minimally invasive surgery Weightlessness
Technology-enabled telepresence has great scientific and commercial potential in health care. The new field of telemedicine is defined as ‘‘The use of medical information exchanged from one site to another via electronic communications for the health and education of the patient or health care provider and for the purpose of improving patient care. Telemedicine includes consultative, diagnostic and treatment services.’’ [1]. Similarly, the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) defines it as ‘‘The practice of medicine and/or teaching of the medical art, without direct physical physician–patient or physician–student interaction, via an interactive audio-video communication system employing tele-electronic devices.’’ [2]. Probably the most prominent pioneers of the field are Tom Sheridan and Richard Satava, both contributing significantly to the birth of new concepts and technologies.
123
682
Telemedicine generally means only the provision and delivery of clinical services. The term ‘‘telehealth’’ usually refers to clinical and nonclinical services such as education, administration or research. The advantages of telemedicine are various: in the case of short-distance operations, the technology involved can mean great added value, such as an externally controlled tool holder or surgical robot. In long-distance telementoring, the time/cost effectiveness and the higher level of medical care provided are the most important benefits, while in extreme telemedicine, such as for space exploration, it may be the only available form of adequate medical aid. Telemedicine can be online (real-time) or offline, depending on the technical quality of the communication link. Furthermore, it can be broken down into three main categories based on the timing and synchrony of the connection (Fig. 1). Store-and-forward telemedicine means there is only one-way communication at a time, and the remote physician evaluates medical information offline and sends it back to the original site at another time. Next, remote monitoring enables medical professionals to collect information about patients from a distance with different modality sensors. Finally, interactive telepresence provide real-time communication between the two sites, which might be extended with various forms of interactions, allowing for a set of telemedicine services [3]. From the application point of view, telesurgery enables physicians to invasively treat patients geologically separated from themselves. Instant and unlimited remote access to the medical site means that the physician is actually capable of performing operations through robots and other teleoperated devices. When the connection is not reliable enough, or the technical tools are not available, a remote surgeon can still
Fig. 1 Categorization of different telehealth and telemedicine technologies. Telemedicine can be broken down into online and offline categories based on the quality of the connection. Telepresence requires the highest bandwidth and reliability
123
Surg Endosc (2011) 25:681–690
direct the local one based on almost real-time video and voice feed from the operating room. This technique is called telementoring, the spatial extension of classical mentoring and professional guiding. Telementoring may not only be useful for surgical education; it has also been proved that untrained people can satisfactorily perform complex medical procedures with it. When communication quality or latency does not even allow semi-real-time connection, consultancy telemedicine (or telehealth consultancy) can be used. This only requires limited access to the remote site, and as a result, the distant group cannot use real-time services or information updates [4]. Telemedicine and especially telementoring have been widely used around the world. Pilot networks were installed and tested already in the second half of the 20th century, and the first intercontinental procedures were conducted in the 1990s [3]. In the past two decades, several projects were initiated to verify telehealth paradigms. Due to the fact that surgeons navigate mostly based on a camera image, telementoring techniques are highly applicable to laparoscopy and general minimally invasive surgery (MIS). MIS is considered to be one of the most important breakthroughs in medicine in the past decades, and the technology keeps developing in many new directions [5]. The first human telesurgery consultation was reported in 1996, and in 2000, completely remote telesurgical animal trials were conducted [6]. In 1997, laparoscopic colectomy and laparoscopic Nissen fundoplications were the first procedures performed with the aid of professional telementoring from over 8 km away [3]. The same group performed the first international telementoring between the Johns Hopkins Medical Institute (Baltimore, MD) and Innsbruck (Austria) and Bangkok (Thailand) [7]. In 1999, telementoring was used from Maryland for five laparoscopic hernia repairs, performed onboard the USS Abraham Lincoln aircraft carrier in California [8]. Later, several intercontinental telementoring experiments were performed, mainly from the USA, to Italy, France, Singapore, Nepal, and Brazil [9]. Beyond the possibility to observe the remote site, the quality of telepresence has always been paramount for surgeons to be able to perform a procedure. The availability of different modalities combined, such as twodimensional (2D)/three-dimensional (3D) visual, tactile/ haptic, acoustic, etc., has been proved to increase human performance dramatically. Currently in telemedicine the dominant form of sensory feedback is visual, as it provides the highest density of information. Performance of video cameras has been steadily increasing [high-definition (HD) resolution is available with most systems], along with highfidelity 3D stereoscopes. Although haptic feedback was provided with the first telerobot prototypes, the commercially available systems lack this modality due to the
Surg Endosc (2011) 25:681–690
complexity (and additional cost) of the hardware and the challenges to provide realistic tactile feedback.
Extreme telemedicine Health care in space represents a unique field of medicine requiring extreme technological solutions. Space medicine embraces all the different aspects of health care in connection with space exploration. It includes both basic crew selection tests and advanced on-flight surgical support. One of the primary safety criteria has always been to return all crew members safely without any serious injuries or illnesses. Based on the medical events recorded by the National Aeronautics and Space Administration (NASA) (collected through 89 Space Shuttle mission from 1981 to 1998), there have been several dozen medical events and complaints, affecting basically all of the organs, involving the risk of serious consequences. Once in 1982, an astronaut almost had to be evacuated on a Space Shuttle with the symptoms of kidney stones [10]. The idea of telerobotic health care in space dates back to the early 1970s, proposed in a study for NASA to provide surgical care for astronauts with remote-controlled robots [11]. Today, this is particularly relevant and desirable, as specific, high-level medical education of flight surgeons might be impossible to achieve. Proficiency in MIS and laparoscopic surgery requires an extreme amount of practice, and maintenance of skills is only possible with continuous training. Simulated surgical experiments in weightlessness have shown that endoscopic procedures provide a real option to perform surgery in a confined body cavity, handling body fluids and organs. The US Army has always been interested in telesurgery for the battlefield, and currently the Telemedicine and Advanced Technology Research Center (TATRC) supports research to test and extend the reach of remote health care. With the help of mechatronic devices, physicians were first able to effect remote patients with the Green telepresence system in 1991. The US Department of Defense (DoD) aimed to develop a system—Trauma Pod—by 2025 that allows combat surgeons to perform life-saving operations on wounded soldiers from a safe distance [12].
Robotic telesurgery While the desire to create superior surgical robots pushes researchers forward all around the world, present-day commercial systems are more focused on one key target procedure each. In practice, only a few surgical robots are autonomous or cooperatively controlled [13]; most systems are entirely remote-controlled, i.e., the surgeon is absolutely
683
in charge of the motion of the robot. Teleoperated systems typically consist of three parts: one or more slave arms, a master controller, and a sensory (e.g., vision) system providing feedback to the user. Based on the gathered visual (and tactile, acoustic, etc.) information, the surgeon guides the arm by moving the controller and closely watching its effect. Commercialized surgical robot systems The most well-known commercialized robots are the da Vinci Surgical System from Intuitive Surgical Inc. (Sunnyvale, CA) and the discontinued Zeus Telesurgical System from former Computer Motion Inc. (Santa Barbara, CA). While these robots were built with a structure and features that make them capable of performing distant teleoperation, most commonly they are used for on-site surgery. Their primary advantages are easing the complexity of laparoscopic procedures, providing better visualization, control, and ergonomics to the surgeon, higher precision, and less invasiveness to the patient [14]. The market-leading (and only available) complete teleoperated robot is the da Vinci, created with roughly US $500M investment. The patient side consists of two or three tendon-driven, 6 ? 1 degree of freedom (DOF) slave manipulators. The system provides high-quality 3D vision with stereo-endoscopes, adjustable tremor filtering, and motion scaling. In 1995, the newly founded Intuitive Surgical licensed technology from NASA, SRI, IBM, and several universities, and by 1997, the first prototype— Lenny—was developed for animal trials. Next, Mona was made for the very first human trials, involving vascular and gynecological procedures at Saint-Blasius Hospital (Dendermonde, Belgium) in March 1997. As the system was originally intended for cardiovascular (beating-heart) surgery, specific clinical trials were performed in Paris and Leipzig in May 1998 [15]. Based on the initial experience, the market-ready version of the robot (named da Vinci in honor of the great inventor) had upgraded control and ergonomic features. Final clinical tests began in 1999, and the US Food and Drug Administration (FDA) approved the system for general laparoscopic surgery (gallbladder, gastroesophageal reflux, and gynecologic surgery) in July 2000, followed by many other approvals, most recently for transoral otolaryngologic procedures. Once the system was on the market, Intuitive continued perfecting it, and the second generation (da Vinci S) was released in 2006. The latest version (da Vinci Si) became available in 2009 with improved full-HD camera system, advanced ergonomic features, and most importantly, the possibility to use two consoles for assisted surgery. Currently, there are more than 1,500 da Vinci units around the world, most of them in the USA, and the number of procedures performed is over
123
684
half a million. The most successful application of the robot became prostatectomy: around 90% of all radical prostate removal procedures were performed robotically in the USA in 2009, according to Intuitive. The concept of the da Vinci theoretically allows remote teleoperation, but the previous versions of the robot used a proprietary short-distance communication protocol through optic fiber to connect the master and the slave, and only the da Vinci Si system facilitates further displacement of the two units. In 2005, TATRC presented collaborative telerobotic surgery (four nephrectomies on a porcine model) with modified da Vinci consoles, being able to overtake the control of one with the other through a public Internet connection [16]. During the experiment, the average roundtrip latency was 450 ms from Denver to Sunnyvale and 900 ms from Cincinnati to Sunnyvale, which degraded the performance of the physicians [6]. The Canadian Surgical Technologies and Advanced Robotics (CSTAR) in London (Ontario, Canada) used Bell Canada’s core network to test the telesurgery-enabled version of the da Vinci. Six successful telesurgical porcine pyeloplasty procedures were performed in Halifax, Nova Scotia, 1,700 km away. The average network latency was 66 ms, and the overall delay was over five times higher, originating from video signal processing, synchronization, and projection [17]. The Queen Elizabeth II Health Sciences Centre (Halifax, Nova Scotia) reported six successful cases of robotic-assisted telementoring for neurosurgery in 2004 [18]. Another telesurgical robot—Zeus—was developed based on the Automated Endoscopic System for Optimal Positioning (AESOP) camera holder arm (FDA approved in December 1993). Zeus received FDA clearance in 2001. It was also controlled in master–slave setup, and used user datagram protocol over Internet protocol (UDP/IP) for communication, facilitating various telesurgery experiments [19]. In 2003, the whole company was bought by Intuitive, and the product line was stalled. More recently, a Canadian company—Titan Medical Inc. (Toronto, Ontario Canada)—announced its new fourarmed manipulator system, Amadeus [20]. It promises a better view of the surgical field and intelligent tools. The robot is being developed in cooperation with Bell Canada’s Health division to integrate Internet protocol-based, advanced telesurgery capabilities for Amadeus. Clinical trials of the system should begin shortly. Laboratory prototypes Several prototype systems never reached commercialization, although they were created with the aim of facilitating telesurgery in space. The Defense Advanced Research Projects Agency (DARPA) of the DoD initiated the
123
Surg Endosc (2011) 25:681–690
Trauma Pod project in 1994, primarily to enhance battlefield casualty care by developing autonomous and semiautonomous mobile platforms through the integration of telerobotic and robotic medical systems. Extending the framework of the program, robots developed with the help of DARPA have already been tested under extreme circumstances: in weightlessness and at NASA’s Aquarius underwater habitat. NASA’s Jet Propulsion Laboratory (JPL) and MicroDexterity Systems Inc. (Albuquerque, NM) developed the Robot-Assisted Micro-Surgery (RAMS) system [21]. RAMS consists of two arms, equipped with tip-force sensors, providing haptic feedback to the operator. It used the concept of telesurgery for control; however, the operator sat right next to the slave arms. The robot was originally aimed at ophthalmic procedures, capable of achieving 10 micron accuracy, tremor filtering, and eye tracking. The project was discontinued, and RAMS rests idle at JPL. Doctors and scientists at the BioRobotics Lab., University of Washington (Seattle, WA) have developed a portable surgical robot for spacecraft with 22 kg overall mass [22]. The DoD-sponsored robot—called Raven— works along the same principle as the da Vinci system; it has two articulated, tendon-driven arms, each holding a stainless-steel shaft for different surgical tools. It can easily be assembled even by nonengineers, and its communication links have been designed for long-distance remote control. The system participated in multiple field tests and intercontinental trials, and now several units are being built for large-scale clinical trials [23]. Stanford Research International (SRI International, Menlo Park, CA) started to develop the M7 robot in 1998 (Fig. 2A, B). The system is lightweight and able to exert significant forces compared with its size. It is equipped with two seven-DOF manipulators, motion scaling, tremor filtering, and haptic feedback [24]. The robot controller board was designed to operate under extremely different atmospheric conditions (e.g., it only contains solid-state memory drives). The software of the M7 was later updated to better suit the requirements of teleoperation and communication via Ethernet link. The M7 performed the world’s first automated ultrasound-guided tumor biopsy in 2007 (Fig. 2B) [25]. The German Aerospace Center (DLR) Institute of Robotics and Mechatronics (Wessling, Germany) has already built several generations of lightweight robotic arms for ground and space applications. They have also taken part in many telerobotic space experiments in the past decades. The KineMedic and the most recent MIROsurge robots are considered for extreme teleoperation, as one arm is only 10 kg and capable of handling relatively large payload with high accuracy [26].
Surg Endosc (2011) 25:681–690
685
Fig. 2 A, B The M7 robot developed by SRI International. The M7 onboard a NASA parabolic flight. When the DC-9 aircraft flew along parabolic curves to create 20–25 s of weightlessness onboard, surgical knot-tying procedures were tested on phantoms, to evaluate human performance (courtesy of SRI International, Menlo Park, CA)
Fig. 3 The Lindbergh operation. A The surgeon’s console of the Zeus robot during the first transatlantic robot-assisted cholecystectomy in 2001. Marescaux (Institut de Recherche contre les Cancers de L’Appareil Digestif - IRCAD) performed the operation from New
York [32] (photo: IRCAD). B The patient and the slave robotic arms were in Strasbourg, France, approximately 7,000 km from the surgeon’s site. The operation was uneventful [32] (photo: IRCAD)
Small scale, in-body robots offer great advantages, opening the possibility for remote control of operations. Engineers at the University of Nebraska (Lincoln, NE) together with the physicians of the local Medical Center developed a special mobile in vivo wheeled robot for biopsy [27]. Equipped with a camera, the coin-sized robot can enter the abdominal cavity through a small incision or a natural orifice and move around under teleoperation, without causing tissue damage. More recently, the group has developed various swallowable, self-assembling robots that can be controlled using external magnets. The CRIM group at Scuola Superiore Sant’Anna (Pisa, Italy) leads a European Union Seventh Framework Program (FP7)-founded international research collaboration to develop tethered, partially autonomous robots to perform surgery in the endolumen (ARAKNES project) [28]. Another EU project—Vector—aims at the creation of effective capsule robots for local surgical procedures throughout the gastrointestinal tract [29]. More recently, an international collaboration demonstrated the feasibility of intercontinental telesurgery by connecting 14 heterogeneous devices in 28 different configurations around the globe—Plugfest 2009. All happened within 24 h, and basic surgical tasks (Telerobotic Fundamentals of Laparoscopic Surgery [30]) were practiced [31].
Milestone experiments for telesurgery in space Intercontinental surgery The Zeus robot proved to be a solid platform to test different telesurgical scenarios. Between 1994 and 2003, the French Institut de Recherche contre les Cancers de l’Appareil Digestif (IRCAD, Strasbourg, France) and Computer Motion Inc. worked together on several experiments. After six porcine cholecystectomies, the first transatlantic human procedure—the Lindbergh operation— was performed with a Zeus on 7 September, 2001 [32]. Marescaux and colleagues controlled the robot from New York, while the patient lay 7,000 km away in Strasbourg (Fig. 3A, B). A 68-year-old female with history of symptomatic cholelithiasis underwent an uneventful cholecystectomy procedure. Based on previous research, it was estimated that the time delay between the master console and the robot needs to be less than 330 ms to perform the operation safely, while above 700 ms, the operator may have real difficulties controlling the Zeus [9]. A highquality, dedicated asynchronous transfer mode (ATM) fiber-optic link was provided by France Telecom, and an average of only 155 ms communication lag time was experienced.
123
686
In Canada, the world’s first regular telerobotic surgical service network was built and managed routinely between the Center for Minimal Access Surgery (CMAS), a McMaster University Centre (Hamilton, Ontario), and a community hospital in North Bay (some 400 km away), using the Zeus robot [33]. The average latency recorded was about 150 ms using a commercial high-speed Internet link over virtual private network (VPN). CMAS performed 22 telerobotic cases with North Bay General Hospital and over 35 telementoring cases with North Bay General Hospital, Ontario and the Complexe Hospitalier La Sagamie, Quebec. The average latency was 135–140 ms. The network was later extended to include more centers in Canada. While the FDA only permitted the single case of telesurgery of the Lindbergh operation in the USA, Canadian health authorities cleared the method for routine procedures. The CSTAR group in London, Ontario evaluated satellite connection versus ground communication to Halifax (Nova Scotia), while performing internal mammary artery dissection on 15 pigs. The average latency was 55 ms over ground communication channels and around 600 ms via satellite connection [34]. A remotely controlled, robot-actuated catheter guiding device was used in Milan in 2006 to automatically perform heart ablation, initiated and supervised by a group of professionals from Boston, MA. The robot used high magnetic fields to direct the catheter to the desired location, taking advantage of preoperative computed tomography (CT) scans of the patient and real-time electromagnetic navigation. Initial trials were performed on 40 patients before the telesurgical experiment took place. The novelty of the system was that it could create the surgical plan on its own, relying on an anatomical atlas based on 10,000 patients [35]. Underwater telerobotic experiments NASA conducted several experiments to examine the effect of latency on human performance in the case of telesurgery and telementoring. The NASA Extreme Environment Mission Operations (NEEMO) always take place in Florida, on the world’s only permanent undersea laboratory—Aquarius. A special buoy provides connections for electricity, life support, and communication, and a shore-based control center supports the habitat with a maximum crew of six people. Fourteen NEEMO projects have been organized since 2001, three of which were focused on telemedicine. The 7th NEEMO project took place in October 2004. The mission objectives included a series of simulated medical procedures with an AESOP robot, using teleoperation and telementoring [36]. The four crew members (one with surgical experience, one physician without significant experience, and two aquanauts without any medical background) had to perform five test procedures:
123
Surg Endosc (2011) 25:681–690
ultrasonic examination of abdominal organs and structures, ultrasound-guided abscess drainage, repair of vascular injury, cystoscopy, and renal stone removal and laparoscopic cholecystectomy. The AESOP was controlled from the Canadian CMAS, 2,500 km away. The signal delay was tuned between 100 ms and 2 s to observe the effects of latency. High latency resulted in extreme degradation of performance: a single knot-tying took 10 min to accomplish. The nontrained crew members were able to perform satisfactorily by exactly following the guidance of a skilled mentor, outperforming the nonsurgeon physician. Based on a set of measurements, greater effectiveness of teleoperation was shown compared with telementoring; however, the latter took less time to complete [25]. During the 9th NEEMO in April 2006, the crew had to assemble and install an M7 robot, and perform real-time abdominal surgery on a patient simulator. A microwave satellite connection was used, and time delay went up to 3 s to mimic Moon–Earth communication links. In addition, each of the four astronauts had to train for at least 2 h with the wheeled in vivo robots designed at the University of Nebraska. In another experiment, pre-established twoway telecommunication links were used for telementoring. The crew had to prove the effectiveness of telemedicine through the assessment and diagnosis of extremity injuries and surgical management of fractures. The effects of fatigue and different stressors on the human crew’s performance in extreme environments were also measured. Latency was set at up to 750 ms in these experiments. Significant performance degradation of the microwave connection was noticed during stormy weather, causing a jitter in latency of up to 1 s [25]. The 12th NEEMO ran in May 2007, and one of its primary goals was to measure the feasibility of telesurgery with the Raven and the M7 robots (Figs. 2, 4). NASA sent a flight surgeon, two astronauts, and a physician into the ocean. Sewing operations were performed on a phantom in a simulated zero-gravity environment to measure the capabilities of surgeons controlling the robots from Seattle. This time, the average latency was 70 ms over wireless connection. A group of three professionals guided the robot using a commercial Internet connection, and the communication lag time was increased up to 1 s. Several simple tasks were performed, as part of the Fundamentals of Laparoscopic Surgery (FLS) [37]. Operations in weightlessness Surgical experiments (laparotomy and celiotomy on rabbits) were first reported by Russian cosmonauts in 1967. The first survival procedure was performed on the STS-90 Neurolab mission on rats in 1998 [38]. The world’s first human operation was a cyst removal from a patient’s arm,
Surg Endosc (2011) 25:681–690
687
Fig. 4 The Raven robot developed at the University of Washington. A The master console of the Raven robot, while a surgeon accomplishing the Fundamentals of Laparoscopic Surgery (FLS) tasks. The robot performed the directed movements 4,500 km away
onboard the NASA Aquarius underwater habitat in Florida [25] (courtesy of University of Washington, WA). B The Raven in action onboard Aquarius, 19 m below the sea, guided by a surgeon from Seattle during NEEMO 12 (courtesy of NASA)
onboard the European Space Agency’s Airbus A-300 ZeroG aircraft. The plane performed 25 parabolic curves, providing 20–25 s of weightlessness every time [39]. ESA had plans to perform teleoperation in 2008 with a robot controlled through satellite connection, but the mission was postponed. NASA carried out its first zero-gravity robotic surgery experiment in late September 2007 [24]. On a DC9 aircraft, suturing tasks were performed using the M7 (Fig. 2). The performance of classical and teleoperated robotic knob-tying was measured. Both the master and the slave devices were equipped with acceleration compensators to help task execution. The experiments showed that humans can better adapt to extreme environments; however, advanced robotic solutions perform comparably.
typically. Understandably, designated military satellites can provide a lot faster communication channels; the minimum latency per satellite hop is expected to be 4.3–7.8 ms one way [40]. Despite the recent improvement in ground-based transmission, satellite communication has the potential to overcome wired lines primarily in speed, with reasonable quality of service and availability. Beyond Earth orbit, radio and microwave frequency signals propagate at almost the speed of light; however, already in the range of long-distance manned space missions, several minutes of latency can be expected. The planet Mars orbits 56–399 million km from Earth, which means a 6.5–44 min delay in transmission. In addition, for about 2 weeks every synodic period, direct communication may be blocked by the Sun. Most humans are capable of adapting to sensory feedback latency of up to 500 ms [33], and some experiments suggest that individuals might be able to perform tasks even with a consistent 1,000 ms delay [41]. Researchers showed that varying latency significantly reduces operator performance with both robotic telesurgery and virtualreality applications, therefore it is better to use consistent (worst-case) latency to achieve constant performance [42].
Technology limitations The primary difficulty with teleoperation over large distances or low-quality network infrastructure is the communication lag time. Accurate and synchronized sensory feedback is essential to ensure reliable and effective telesurgical treatment. The continuous development of the terrestrial Internet backbone network has resulted in a significant reduction of typical latencies. Using commercial services, delay might be around 85 ms across the USA and anywhere from 20 to 400 ms worldwide. The telerobotic experiment Plugfest 2009 showed 21–112 ms latency for various connections within the USA and 115–305 ms for intercontinental connections [31]. Satellite-based Internet connections can use a fleet of low or medium Earth orbit satellites, where typical roundtrip delays are 40 ms, but bandwidth is very limited. Geosynchronous satellites provide higher latency due to their 36,000 km altitude; round trip latency is 540–700 ms
Achievable solution for space mission support While advanced Internet-based communication enables telesurgery all over the Earth, serious technological problems arise in the case of long-haul space exploration missions. Currently, it seems inevitable to have a flight surgeon onboard the spacecraft, as robots do not have enough autonomy to adapt to any unforeseeable events. Based on the predescribed conditions, difficulties of endoscopic surgery, and system requirements, a three-
123
688
layered mission architecture is proposed to achieve the highest degree of performance possible by combining robotic and human surgery (Fig. 5). Depending on the physical distance between the spacecraft and the ground control center, different telepresence technologies may provide the best performance [43]. Basically, with increasing latency, the effectiveness of real-time control strategies and communication techniques significantly decreases. Mainly within the range of 380,000 km (average Earth– Moon distance), semi-real-time telesurgery techniques can be used to provide medical support in the case of an emergency. On leaving Earth orbit, special control engineering algorithms have to be applied (e.g., virtual coupling of the remote environment [44], predictive displays projecting the intended motion of the tools ahead in time [45]) to extend the feasibility of telesurgery up to a maximum of 2 s delay. With robot-assisted surgery, a shared control approach should be followed, integrating highfidelity automated functions into the robot to extend the capabilities of the human surgeon through image processing and force sensing. This concept could be most beneficial for long-duration on-orbit missions, primarily onboard the International Space Station. Presently, there is no option other than immediate evacuation of the affected astronaut, which poses larger health risk and huge costs. Flying further from the Earth, and reaching the limits of semi-real-time communication, procedures should be performed by the flight surgeon under detailed telementoring guidance of master surgeons on the ground. Telementoring requires exchange of still images, motion video, voice conferencing, electronic chat, and data file transfer. As shown by the NEEMO experiments, telementoring can be an effective alternative to direct teleoperation, allowing Fig. 5 General concept for telemedicine support of longduration space flights. The fundamental goal of telehealth support is to provide the maximum level of available medical care to astronauts during space missions. An onboard teleoperated surgical robot should assist procedures controlled by experts on the ground, while the spacecraft is in the proximity of the Earth. With increasing latency, telementoring and consultancy telemedicine could provide the most effective support to the flight surgeon (reprinted with permission [43])
123
Surg Endosc (2011) 25:681–690
remote staff to perform tasks based on commands from the ground center [36]. Telementoring may extend the boundaries of telepresence, as it can still be effective with a 50–70 s delay (within the range of approximately 10 million km). The built-in, semi-autonomous functions of the surgical robot will play a major role in improving the overall quality of surgery: motion scaling, adaptive tremor filtering, automated following of organ movement, or automated suturing could significantly improve less practiced crew members’ performance. On the other hand, defining virtual boundaries for the robot (spatial tool limitations and speed constraints) may reduce the risk of accidents. Astronauts should also benefit from advanced imaging technologies (e.g., surgical navigation and augmented-reality systems). Telementoring transforms into offline consultancy telemedicine above a certain signal delay. The terrestrial medical support crew will not be able to react promptly to unforeseeable events during the procedure, and the flight surgeon will be left alone for longer and longer periods. Around 1 min of delay, it is inconvenient and impractical for the crew to wait for the guidance of the ground after every procedural step, and in some cases, it would endanger the success of the operation. For these missions, the flight surgeon must be trained to conduct the operation and make decisions alone. However, the ground staff can provide high-value support by patient-specific simulations and thorough consultancy beforehand the operation [43]. These three forms of telemedicine are currently the most feasible alternatives to support human space missions, mainly relying on existing technology. Telementoring and telehealth consultancy are technologically available in shorter ranges as well. It was shown that telesurgery should
Surg Endosc (2011) 25:681–690
be preferred over telementoring depending on availability [36]. The different paradigms can substitute and support each other (e.g., in the case of signal loss) or facilitate the work of the astronauts during the operation. Surgical robots onboard would have further advantages. The astronauts would be able to conduct several material and life science experiments and research, using the robots for micromanipulation tasks, and flight surgeons could continuously develop their skills through practicing.
Discussion Minimally invasive procedures provide new means to handle extreme environmental conditions in surgery, such as weightlessness. Invasive treatment of astronauts may only be possible within the enclosed cavities of the human body. Significant research effort has been invested in the last two decades to develop robotic devices and communication infrastructure facilitating telemedicine. Technology now enables the delivery of high-quality health care services not just in remote rural areas but also in space—on orbit and beyond. The recent NEEMO missions of NASA and the Space Shuttle and parabolic flight surgery experiments have focused on assessment of the technology and human skills necessary to operate with remotely guided surgical robots in weightlessness. The major conclusion is that, beyond communication lag time (originating from signal propagation), most of the technical challenges are manageable with minimally invasive techniques. However, already in the case of intercontinental teleoperation—assuming use of commercial communication links—latency can be on the order of several hundred milliseconds, potentially disrupting the surgeon. There is a strong need for a complete teleoperated robot onboard spacecraft to provide highquality surgical assistance for long-duration space missions. Advanced system design will help to deal with signal delays and to extend human capabilities, allowing humanity to travel further than ever before in history. Acknowledgment The research was supported by the National Office for Research and Technology (NKTH), Hungarian National Scientific Research Foundation grants OTKA T69055, CK80316. Disclosures Drs. Benyo´, Sa´ndor, and Haidegger have no conflicts of interest or financial ties to disclose.
References 1. Pease RWJ (ed) (2003) Medical dictionary. Merriam-Webster, USA 2. SAGES Group (2000) Guidelines for the surgical practice of telemedicine. Society of American Gastrointestinal Endoscopic Surgeons. Surg Endosc 14(10):975–979
689 3. Rosser JC Jr, Young SM, Klonsky J (2007) Telementoring: an application whose time has come. Surg Endosc 21(8):1458–1463. doi:10.1007/s00464-007-9263-3 4. Challacombe B, Kavoussi L, Patriciu A, Stoianovici D, Dasgupta P (2006) Technology Insight: telementoring and telesurgery in urology. Nat Clin Practice Urol 3:611–617 5. Ballantyne GH (2007) The future of telerobotic surgery. In: Patel VR (ed) Robotic urologic surgery. Springer, Columbus, US, pp 199–206 6. Sterbis JR, Hanly EJ, Herman BC, Marohn MR, Broderick TJ, Shih SP, Harnett B, Doarn C, Schenkman NS (2008) Transcontinental telesurgical nephrectomy using the da Vinci robot in a porcine model. Urology 71(5):971–973. doi:10.1016/j.urology. 2007.11.027 7. Lee BR, Caddedu JA, Janetschek G, Schulam P, Docimo SG, Moore RG, Partin AW, Kavoussi LR (1998) International surgical telementoring: our initial experience. Stud Health Technol Inform 50:41–47 8. Cubano M, Poulose BK, Talamini MA, Stewart R, Antosek LE, Lentz R, Nibe R, Kutka MF, Mendoza-Sagaon M (1999) Longdistance telementoring: a novel tool for laparoscopy aboard the USS Abraham Lincoln. Surg Endosc 13(7):673–678 9. Fabrizio M, Lee B, Chan D, Stoianovici D, Jarrett T, Yang C, Kavoussi LR (2000) Effect of time delay on surgical performance during telesurgical manipulation. J Endourol 14(2):133–138 10. Allen CS, Burnett R, Charles J, Cucinotta F, Fullerton R, Goodman JR, Griffith AD, Kosmo JJ, Perchonok M, Railsback J, Rajulu S, Stilwell D, Thomas G, Tri T (2003) Guidelines and capabilities for designing human missions, NASA/Johnson Space Center, TM-2003-210785 11. Alexander AD (1973) Impacts of telemation on modern society. Proc. of Human Factors and Ergonomics Society Annual Meeting 17(2):299–304 12. Satava RM (1995) Virtual reality, telesurgery, and the new world order of medicine. J Image Guided Surg 1:12–16 13. Nathoo N, Cavusoglu MC, Vogelbaum MA, Barnett GH (2005) In touch with robotics: neurosurgery for the future. Neurosurgery 56(3):421–433. doi:10.1227/01.NEU.0000153929.68024 14. Eadie LH, Seifalian aM, Davidson BR (2003) Telemedicine in surgery. Br J Surg 90(6):647–658. doi:10.1002/bjs.4168 15. Ballantyne GH, Marescaux J, Giulianotti PC (eds) (2004) Primer of robotic & telerobotic surgery. Lippincott Williams & Wilkins, Philadelphia, PA 16. Flynn E (2005) Telesurgery in the United States. J Homeland Defense 6:24–28 17. Nguan C, Miller B, Patel R, Luke PP, Schlachta CM (2008) Preclinical remote telesurgery trial of a da Vinci telesurgery prototype. Int J Med Robotics Comput Assist Surg 4:304–309. doi: 10.1002/rcs.210 18. Mendez I, Hill R, Clarke D, Kolyvas G, Walling S (2005) Robotic long-distance telementoring in neurosurgery. Neurosurgery 56(3):434–440. doi:10.1227/01.NEU.0000153928.51881.27 19. Kumar S, Marescaux J (eds) (2008) Telesurgery. Springer, Berlin 20. Rayman R (2009) Is surgery a remote possibility? Robotic surgical system under development has telesurgery capabilities. Health Technol Trends 21(7):5–7 21. Das H, Ohm TI, Boswell C, Steele RO, Rodriguez G (2001) Robot-assisted microsurgery development at JPL. In: Akay M, Marsh A (eds) Information technologies in medicine, Vol. II: rehabilitation and treatment. Wiley, New York, pp 85–99 22. Rosen J, Hannaford B (2006) Doc at a distance. IEEE Spectrum 8(10):34–39 23. Lum MJ, Friedman DC, Sankaranarayanan G, King H, Fodero K, Leuschke R, Hannaford B (2009) The RAVEN: design and validation of a telesurgery system. Int J Robot Res 28(9):1183–1197. doi:10.1177/0278364909101795
123
690 24. Kamler K (2007) How I Survived a Zero-G Robot Operating Room: Extreme Surgeon. Popular Mechanics—online edition. Available: www.popularmechanics.com/science/robotics/4230102.html 25. Doarn CR, Anvari M, Low T, Broderick TJ (2009) Evaluation of teleoperated surgical robots in an enclosed undersea environment. Telemed J e-Health 15(4):325–335. doi:10.1089/tmj.2008.0123 26. Hagn U, Tobergte RK, Jo¨rg MN, Gro¨ger GP, Seibold FF, Hacker AN et al (2010) DLR MiroSurge : a versatile system for research in endoscopic telesurgery. Int J CARS 5:183–193. doi:10.1007/ s11548-009-0372-4 27. Rentschler ME, Dumpert J, Platt SR, Oleynikov D, Farritor SM, Iagnemma K (2006) Mobile In Vivo Biopsy Robot. Proc. of the 2006 IEEE International Conference on Robotics and Automation, Orlando, pp 4155–4160 28. Menciassi A, Dario P (2009) Miniaturized robotic devices for endoluminal diagnosis and surgery: a single-module and a multiple-module approach. Proc. 31st Int Conf of the IEEE Engineering in Medicine and Biology Society, pp 6842–6845. doi: 10.1109/IEMBS.2009.5334474 29. Eirik L, Johansen B, Gjelsvik T, Langø T (2009) Ultrasound based localization of wireless microrobotic endoscopic capsule for the GI tract. Proc. 21st Conference of the Society for Medical Innovation and Technology 30. Lum M, Friedman D, Sankaranarayanan G, King H, Wright A, Sinanan M, Lendvay T, Rosen J, Hannaford B (2008) Objective assessment of telesurgical robot systems: Telerobotic FLS. Medicine Meets Virtual Reality (MMVR). Long Beach, CA, pp 263–265 31. King H, Hannaford B, Kwok K, Yang G, Griffiths P, Okamura A, et al (2010) Plugfest 2009: global interoperability in telerobotics and telemedicine. IEEE International Conference on Robotics and Automation, Anchorage, AK, pp 1733–1738 32. Marescaux J, Leroy J, Rubino F, Smith M, Vix M, Simone M, Mutter D (2002) Transcontinental robot-assisted remote telesurgery: feasibility and potential applications. Ann Surg 235(4): 487–492 33. Anvari M (2004) Robot-assisted remote telepresence surgery. Surg Innov 11(2):123–128. doi:10.1177/107155170401100209 34. Rayman R, Croome K, Galbraith N, Mcclure R, Morady R, Peterson S, Smith S, Subotic V, Van Wynsberghe A, Patel R, Primak S (2007) Robotic telesurgery: a real-world comparison of ground- and satellite-based Internet performance. Int J Med Robotics Comput Assist Surg 3:111–116. doi:10.1002/rcs.133
123
Surg Endosc (2011) 25:681–690 35. Pappone C, Vicedomini G, Manguso F, Gugliotta F, Mazzone P, Gulletta S, Sora N, Sala S, Marzi A, Augello A, Livolsi L, Santagostino A, Santinelli V (2006) Robotic magnetic navigation for atrial fibrillation ablation. J Am Coll Cardiol 47(7): 1390–1400. doi:10.1016/j.jacc.2005.11.058 36. Thirsk R, Williams D, Anvari M (2007) NEEMO 7 undersea mission. Acta Astronautica 60(4–7):512–517. doi:10.1016/ j.actaastro.2006.09.015 37. Peters J, Fried G, Swanstrom L, Soper N, Sillin L (2004) Development and validation of a comprehensive program of education and assessment of the basic fundamentals of laparoscopic surgery. Surgery 135(1):21–27 38. Campbell MR, Kirkpatrick AW, Billica RD, Johnston SL, Jennings R, Short D, Hamilton D, Dulchavsky SA (2001) Endoscopic surgery in weightlessness: the investigation of basic principles for surgery in space. Surg Endosc 15(12):1413–1418. doi:10.1007/s004640080178 39. Doctors remove tumor in first zero-g surgery. New Scientist (September 2006). Available: http://www.newscientist.com/article/ dn10169-doctors-remove-tumour-in-first-zerog-surgery.html 40. Berlocher G (2009) Minimizing Latency in Satellite Networks. Via Satellite, Published at: www.viasatellite.com 41. Lum MJ, Rosen J, Lendvay TS, Wright AS, Sinanan MN, Hannaford B (2008) TeleRobotic fundamentals of laparoscopic surgery (FLS): effects of time delay-pilot study. Proceedings of the international conference of the IEEE engineering in medicine and biology society, pp 5597–600. doi:10.1109/IEMBS.2008.4650483 42. Campbell MR, Billica RD (2008) Surgical capabilities. In: Barratt MR, Pool SL (eds) Principles of clinical medicine for space flight. Springer, Berlin, Ch. 6., pp 123–138 43. Haidegger T, Benyo Z (2008) Surgical robotic support for long duration space missions. Acta Astronautica 63(7–10):996–1005. doi:10.1016/j.actaastro.2008.01.005 44. Thompson JM, Ottensmeyer MP, Sheridan TB (1999) Human factors in telesurgery: effects of time delay and asynchrony in video and control feedback with local manipulative assistance. Telemed J 5(2):129–137. doi:10.1089/107830299312096 45. Rayman R, Croome K, Galbraith N, McClure R, Morady R, Peterson S, Smith S, Subotic V, Van Wynsberghe A, Primak S (2006) Long-distance robotic telesurgery: a feasibility study for care in remote environments. Int J Med Robotics Comput Assist Surg 2:216–224. doi:10.1002/rcs.99
