First Author A.Ramkumar et al.
1
Exploring input devices for contouring in external radiotherapy Anjana Ramkumar1*, Edit Varga1, Wiro J. Niessen2 and Adinda Freudenthal1 1
*
Faculty of Industrial Design Engineering, Delft University of Technology 2 Biomedical Imaging Group, Erasmus MC - University Medical Center, Rotterdam
[email protected]
Abstract The past decade has witnessed major improvements in Human-Computer Interaction (HCI) in healthcare. However, in the field of radiotherapy not many advances have happened in HCI. Rapid and accurate contouring of tumor volumes is now highly important in radiotherapy, and this key step influences the subsequent steps in the radiotherapy workflow. In order to attain reliable contours, good interaction is necessary. The main goal of this paper is to identify the possibilities of adding new interaction methods to the field of radiotherapy contouring by exploring interaction types available in healthcare, especially in the fields of radiology and radiotherapy. This study explores five different interaction methods: mouse- pen, tablet-PC, multi-touch table, gesture based and speech/voice. It was found that the tablet-PC, multi-touch and gesture based interaction methods are the ones that could be included in radiotherapy contouring along with the already existing pen and mouse inputs. Hence, future research will focus on developing prototypes of the above-mentioned combinations and evaluating these by radiation oncologists. . Index Terms — human-computer interaction, radiology, radiotherapy, contouring.
INTRODUCTION Human-Computer Interaction (HCI) lies at the crossroad of several research areas including computer vision, psychology and many others [1]. Computers are as ubiquitous in healthcare as in modern society [2]. HCI plays a very important role among all healthcare professionals and clerical staffs to keep the track and view of patient records, for making appointments etc. [2]. The main expectations are user friendliness, user-acceptance and user-competence [3]. Many interactions and input devices are available, including mouse, pen-tablet, keyboard, joystick, trackball, tactile, gesture, speech, eye-control. Even though there has been a drastic advancement in technology, most of the hospitals still use the traditional input devices such as computer keyboards and display monitors [3]. Healthcare as such is a vast term, so this paper limits the focus mainly on interactions available with imaging data in the fields of radiology and radiotherapy. A large number of imaging procedures are performed every day at the radiology department resulting in a large number of interactions between the human and computer. Interventional radiological procedures require SUMMER Project (PITN-GA-2011-290148) is funded by the 7th Framework Programme of the European Commission (FP7-PEOPLE-2011-ITN).
2
Exploring input devices
sterile conditions where the physician cannot interact with their computers the regular way to inspect the images. They instruct other staff to control the computer for them [4]. Imaging data (MRI, CT) are used for treatment planning in radiotherapy. Rapid and accurate delineation of tumor volumes and organs at risk is hence highly important in radiotherapy, and this key step influences the subsequent steps in the radiotherapy workflow [6]. Even though it is a possibility to contour automatically or semi-automatically but still there is a need for user input to validate and correct the contours. Hence, in order to attain reliable contours, good interaction is necessary. The currently most commonly used means of interacting with these images is via pen and mouse input. The main goal of this paper is to explore the potential benefits and disadvantages of different interaction methods with imaging data in the fields of radiology and radiotherapy. This will help to identify the possibilities of adding new interactions to the field of radiotherapy contouring. RESULTS
Pen-mouse interaction In radiotherapy and radiology, the mouse and the pen are the most commonly used interaction tools. Reliability, accuracy, and user satisfaction, with both interaction tools have been tested in radiology and radiotherapy contouring [7, 8]. It was found that the performance of using a pen was better than the mouse in all studies. It was also found that the overall error and the time taken for segmentation was less with pen than with the mouse. A different study was done by Kentaro et al. by comparing the muscular load performance of the two interactions [9]. The performance of the pen exceeded the performance of the mouse. Tablet-PC Tablet PC is the general platform for the pen interaction and it provides a natural way to interact with the computer [10]. The tablet PC in general is a rather vague term as it is used by individuals to refer to devices of varied sizes in their respective fields. But usually in radiotherapy it is a form of personal computer with a large touch screen operated with pen, stylus or finger input. Even though the big screen has initially been received as a good idea, with the advancement in technology, it is increasingly being replaced by small sized tablets. Tablets have many advantages in clinical practice, as they are portable and the physicians can accomplish tasks anywhere. In radiology, tablets are being used by physicians to effectively communicate, to access scan results from anywhere in the hospital, and to refer to books, journals, encyclopedia and manuals [11, 12]. Tablets also have proved useful for surgeons during surgery. Surgeons do not have to turn away from their sterile area under operation in order to view relevant images. . The Food and Drug Administration (FDA) felt the need to approve many applications [13]. Even the viewing of DICOM images in tablet from the PACS is not restricted anymore. However, its small size could be a limiting factor in certain situations. Multi-touch table Touch screens are increasingly used in the medical context [14], and many applications for image review are available in touch-controlled versions. There are many benefits during decision making and collaborative work. A touch interface is very intuitive.[14]. It is found to be more Innovative imaging to improve radiotherapy treatments Summer-school of SUMMER project (July 2013)
First Author A.Ramkumar et al.
3
efficient than the mouse for selecting and sorting tasks [15] and also helpful in per-operative planning [14]. It is of interest to investigate the potential of this option in radiotherapy, as it could potential improve interaction in the pre-operative treatment planning . In interventional radiology, there is a need to maintain boundaries between the sterile and non-sterile environment and multitouch is one of the common way of spreading infection. A sterile cover over the input devices is impractical in a surgical environment and hence gesture-based interactions will likely play an important role [16]. Gesture-based interaction With a wave of a hand or lift of a finger it is possible to change the way we interact with computers. The gesture-based interaction approach can be divided into glove-based method and vision-based approach. With the advancement of technology, the glove-based approach have largely been replaced by vision-based techniques. Visible techniques are favorable during interventional procedures, when physicians cannot wear extra equipment on their hands. Visionbased interaction has already shown to be a rapid and intuitive interaction approach in brain biopsy procedures for navigation and manipulation of images [16]. New technology is emerging in the market, known as the Leap Motion Controller [17], which has been claimed to be more accurate than the Kinect technology. One of the main differences between Kinect and Leap Motion is that with the former the user needs to be far from the screen whereas with the latter the user needs to be very close. The Leap Motion allows to manipulate the screen via a series of hand and finger movements in the air. Moreover, it is not only able to detect finger movements, but also a pen or a pencil could be used for drawing. Distance, however, remains a limiting factor for both technologies, as user have to operate at a certain distance from the screen. Speech/voice interaction Physicians are generally reluctant to use interfaces that require a considerable amount of typing [18]. Speech recognition technology has the advantage that it does not tie the user down to the keyboard and it provides some freedom to interact with the system even at some distance away [3]. The user can enter or receive data while engaged in other work. Similarly, the doctor can examine a patient and could record his findings using voice input [3]. There are many software solutions existing where the report can be directly dictated into the computer system [4]. Furthermore, it could also be used to control system tasks [18]. Speech interaction is also being used in combination with other interaction methods like gesture-based interaction, which is useful in a sterile environment for interventional radiology procedures [4]. Even though this interaction has many benefits, the negative side of it is that speech interaction needs a quiet environment and continuous speaking leads to fatigue at some point [4]. DISCUSSION From the literature, we can conclude that most of the interactions that are currently available are useful in radiology and radiotherapy. The Pen interaction, which already exists in radiotherapy, has several benefits and is preferred by physicians as well. Tablet is another possible interaction that could be introduced for contouring of tumor and target volumes, and for evaluating and checking the dosimetric plans [13]. Hence, tablets may be able to drastically change the daily clinical practice of radiation oncologists. It is still a question whether the radiation oncologist will adopt this change, which is likely to depend on the usability of the technologies that will be developed. Gesture-based interaction may be an option in radiotherapy, for example the Leap SUMMER Project (PITN-GA-2011-290148) is funded by the 7th Framework Programme of the European Commission (FP7-PEOPLE-2011-ITN).
4
Exploring input devices
Motion Controller could be used for rapid contouring without touching the screen. On the other hand, it might be difficult for the physicians to keep their hand in the air for a long time. Multitouch interaction could be a faster way to interact, but then contouring should be done with fingers. If this is going to be the future, then accuracy of finger-based contouring has to be tested. Another possibility could be to combine it with pen interaction. Speech interaction could be used in radiotherapy for entering patient reports or searching for a patient planning file, but it is hard to imagine it as an input method for contouring. Hence, future research will focus on the abovementioned combinations and test it among radiation oncologists. CONCLUSION A number of interaction methods with imaging data are available within radiology and radiotherapy, other than the regular pen and mouse input. Tablet-PC, multi-touch and gesturebased interaction could be beneficial in radiotherapy therapy treatment planning, and hence there is a need for research to investigate its potential. ACKNOWLEDGMENT
This research is part of a European Union project called Software for the Use of Multi-Modality images in External Radiotherapy (SUMMER) project and which is part of Marie Curie Research Training Network (PITN-GA-2011-290148), which is funded by the 7th Framework Programme of the European Commission (FP7-PEOPLE-2011-ITN).We would like to thank whole of the consortium members for all their support. A special thanks to Aquilab for publishing my study.
REFERENCES [1] Jaimes A, & Sebe N. Multimodal human–computer interaction: A survey. Computer Vision and Image Understanding, (2007); 108(1-2), 116–134. [2] Fromme EK, Kenworthy-Heinige T, & Hribar M. Developing an easy-to-use tablet computer application for assessing patient-reported outcomes in patients with cancer. Supportive care in cancer : official journal of the Multinational Association of Supportive Care in Cancer, 2011; 19(6), 815–22. [3] Lun KC. New user interfaces. International Journal of Bio Medical Computing, 1995, 39(1), 147–150. [4] Hötker AM, Pitton MB, Mildenberger P, & Düber C. Speech and motion control for interventional radiology: requirements and feasibility. International journal of computer assisted radiology and surgery. International Journal of Computer Assisted Radiology and Surgery, April 2013 [5] Harders M, Member S, Székely G, & Member A. Enhancing Human – Computer Interaction in Medical Segmentation, PROCEEDINGS OF THE IEEE,2003; VOL. 91, NO. 9, [6] Whitfield G., Price P, Price G. J, & Moore C. J. Automated delineation of radiotherapy volumes: are we going in the right direction? The British journal of radiology, 2013; 86(1021), 20110718. Innovative imaging to improve radiotherapy treatments Summer-school of SUMMER project (July 2013)
First Author A.Ramkumar et al.
5
[7] Chen JY, Seagull FJ, Nagy P, Lakhani, P, Melhem ER, Siegel EL, & Safdar N. M. Computer input devices: neutral party or source of significant error in manual lesion segmentation? Journal of digital imaging, 2011; 24(1), 135–41. [8] Perandini S, Faccioli N, Inama M, & Pozzi Mucelli R. Freehand liver volumetry by using an electromagnetic pen tablet: accuracy, precision, and rapidity. Journal of digital imaging : the official journal of the Society for Computer Applications in Radiology, 2011; 24(2), 360–5. [9] Kotani K, & Horii K. “An Analysis of Muscular Load and Performance in Using a Pen-tablet System”, J Physiol Anthropol 2003; 22 (2): 89–95. [10] Kim H., Cho YS, & Do EY. Using Pen-Based Computing in Technology for Health, Part IV, HCII 2011; 4, LNCS 6764, 192–201. [11] Baumgart DC, Smartphones in clinical practice, Medical Education and Research, Arch intern med, 2011; 171 (NO. 14). [12] Székely A, Talanow R, & Bágyi P. Smartphones, tablets and mobile applications for radiology. European Journal of Radiology, 2013; 82(5), 829–836. [13] Gomez-Iturriagaa A, Bilbaoa P, Casqueroa F, Cacicedoa J, Crook J. Smartphones and tablets Reshaping radiation oncologists’, reports of practical oncology and radiotherapy, 2012; 1 7, 276–280 [14] Lundström C, Rydell T, Forsell C, Persson, A., & Ynnerman A. Multi-touch table system for medical visualization: application to orthopedic surgery planning. IEEE transactions on visualization and computer graphics, 2011; 17(12), 1775–84. [15] Kin K, Agrawala M, and DeRose T. Determining the benefits of direct- touch, bimanual, and multifinger input on a multitouch workstation. In Proceedings of Graphics Interface, 2009, pages 119–124. [16] Wachs J P, Stern H I, Edan Y, Gillam M, Handler J, Feied C, Smith M. A Gesture-based Tool for Sterile Browsing of Radiology Images, Journal of the American Medical Informatics Association, 2008; 15 (3), 321-23. [17] https://www.leapmotion.com/ [18] Shiffman S, Wu A W, Poon A D, Lane C D, Middleton B, Miller R A, Masarie F E, Cooper G F, Shortliffe E H, Fagan L M. Building a speech interface to a medical diagnostic system, 1991, IEEE Expert, 41-50.
SUMMER Project (PITN-GA-2011-290148) is funded by the 7th Framework Programme of the European Commission (FP7-PEOPLE-2011-ITN).
6
Exploring input devices
Anjana Ramkumar was born in a small town called as Renukoot in 1989, in state called as Uttar Pradesh in north India. Schooling and bachelor’s degree was done in south India. Ramkumar hold a Bachelor’s degree in Medical Radiological Technology (4years) from the Amrita Institute of Medical Sciences and Research Centre, Kochi, India in 2011. After her Bachelors, she moved to the United Kingdom for her Masters in Medical Physics at the University of Surrey in 2012 September. In 2012 November, Ramkumar joined the Faculty of Industrial Design Engineering, Delft University of Technology for her PhD study. Along with her bachelor’s studies she had training at the radiotherapy department, radiology and nuclear medicine departments as a technologist for 3 years. Her bachelor thesis was about “Comparison of gross tumor volumes obtained using autocontoured program PET-VCAR versus manual contouirng”. During her masters she did her internship at the Brighton Sussex cancer center for few weeks and then she worked on her maser’s research at the same hospital on the topic, “commissioning of 4D CT”. At present, her research work is related to contouring, but focusing more on user-centered design. Ms. Ramkumar has won a best paper award for presenting her bachelor’s thesis work at the “Singapore nuclear medicine update2011”.
Innovative imaging to improve radiotherapy treatments Summer-school of SUMMER project (July 2013)
